Multipart upload doesn't work with >200MB objects #535

New issue

Closed

opened 2025-12-28 17:37:43 +00:00 by sami · 0 comments

sami commented

2025-12-28 17:37:43 +00:00

Owner

Originally created by @EESergey on GitHub (Sep 10, 2025).

Originally assigned to: @smallhive on GitHub.

How to reproduce:

warp mixed --host=#REMOVED --access-key=#REMOVED --secret-key=#REMOVED --insecure --concurrent 1 --duration 5m -obj.size 400M --bucket warp --quiet --noclear --debug

HTTP requests:

---------END-HTTP---------
PUT /warp/JQW18E%283/1.blggc1xHHQkm%283ef.rnd?partNumber=23&uploadId=HTTpEJZXSUDYqQu1mLBJXGMDonKobMWJapjMwTLTk5jL HTTP/1.1
Host: #REMOVED
User-Agent: MinIO (linux; amd64) minio-go/v7.0.83 warp/1.0.8
Content-Length: 16800342
Authorization: AWS4-HMAC-SHA256 Credential=#REMOVED/20250910/default/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length,Signature=**REDACTED**
X-Amz-Content-Sha256: STREAMING-AWS4-HMAC-SHA256-PAYLOAD
X-Amz-Date: 20250910T103732Z
X-Amz-Decoded-Content-Length: 16777216
Accept-Encoding: gzip

HTTP/1.1 200 OK
Content-Length: 0
Accept-Ranges: bytes
Date: Wed, 10 Sep 2025 10:37:33 GMT
Etag: aaba8aaf973419c68c3979f42c06c85c13a60461cebc72fcd7cf4d644c4b7ddd
Server: NeoFS-S3-GW/0.41.1
X-Amz-Request-Id: fd50b15f-97f4-4a41-b5f7-49aadb8ed79b
---------END-HTTP---------
---------START-HTTP---------
POST /warp/JQW18E%283/1.blggc1xHHQkm%283ef.rnd?uploadId=HTTpEJZXSUDYqQu1mLBJXGMDonKobMWJapjMwTLTk5jL HTTP/1.1
Host: #REMOVED
User-Agent: MinIO (linux; amd64) minio-go/v7.0.83 warp/1.0.8
Content-Length: 2898
Authorization: AWS4-HMAC-SHA256 Credential=#REMOVED/20250910/default/s3/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
Content-Type: application/octet-stream
X-Amz-Content-Sha256: 0e9ca61b87ba9586e3f2b9648678556cb03a521561abd06735aea67692039c5d
X-Amz-Date: 20250910T103733Z
Accept-Encoding: gzip

HTTP/1.1 400 Bad Request
Content-Length: 501
Accept-Ranges: bytes
Content-Type: application/xml
Date: Wed, 10 Sep 2025 10:37:41 GMT
Server: NeoFS-S3-GW/0.41.1
X-Amz-Request-Id: 6d4b8119-6a70-4175-b49b-c809b31f2e8e

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidPart</Code><Message>One or more of the specified parts could not be found.  The part may not have been uploaded, or the specified entity tag may not match the part&#39;s entity tag.</Message><Key>JQW18E(3/1.blggc1xHHQkm(3ef.rnd</Key><BucketName>warp</BucketName><Resource>/warp/JQW18E(3/1.blggc1xHHQkm(3ef.rnd</Resource><RequestId>6d4b8119-6a70-4175-b49b-c809b31f2e8e</RequestId><HostId>cb7094ed-8662-467d-ac2e-429025b5b0f6</HostId></Error>---------END-HTTP---------
---------START-HTTP---------
DELETE /warp/JQW18E%283/1.blggc1xHHQkm%283ef.rnd?uploadId=HTTpEJZXSUDYqQu1mLBJXGMDonKobMWJapjMwTLTk5jL HTTP/1.1
Host: #REMOVED
User-Agent: MinIO (linux; amd64) minio-go/v7.0.83 warp/1.0.8
Authorization: AWS4-HMAC-SHA256 Credential=#REMOVED/20250910/default/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20250910T103741Z
Accept-Encoding: gzip

HTTP/1.1 204 No Content
Date: Wed, 10 Sep 2025 10:37:44 GMT
X-Amz-Request-Id: 214848a1-6ce3-448d-b461-d8768ab8509a
---------END-HTTP---------
warp: <ERROR> upload error: One or more of the specified parts could not be found.  The part may not have been uploaded, or the specified entity tag may not match the part's entity tag.

In logs:

error	handler/util.go:34	call method	{"status": 400, "request_id": "6d4b8119-6a70-4175-b49b-c809b31f2e8e", "method": "CompleteMultipartUpload", "bucket": "warp", "object": "JQW18E(3/1.blggc1xHHQkm(3ef.rnd", "description": "could not complete multipart upload", "error": "InvalidPart: 400 => One or more of the specified parts could not be found.  The part may not have been uploaded, or the specified entity tag may not match the part's entity tag.", "uploadID": "HTTpEJZXSUDYqQu1mLBJXGMDonKobMWJapjMwTLTk5jL", "Key": "JQW18E(3/1.blggc1xHHQkm(3ef.rnd", "reqId": "6d4b8119-6a70-4175-b49b-c809b31f2e8e"}

Originally created by @EESergey on GitHub (Sep 10, 2025). Originally assigned to: @smallhive on GitHub. How to reproduce: ``` warp mixed --host=#REMOVED --access-key=#REMOVED --secret-key=#REMOVED --insecure --concurrent 1 --duration 5m -obj.size 400M --bucket warp --quiet --noclear --debug ``` HTTP requests: ``` ---------END-HTTP--------- PUT /warp/JQW18E%283/1.blggc1xHHQkm%283ef.rnd?partNumber=23&uploadId=HTTpEJZXSUDYqQu1mLBJXGMDonKobMWJapjMwTLTk5jL HTTP/1.1 Host: #REMOVED User-Agent: MinIO (linux; amd64) minio-go/v7.0.83 warp/1.0.8 Content-Length: 16800342 Authorization: AWS4-HMAC-SHA256 Credential=#REMOVED/20250910/default/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length,Signature=**REDACTED** X-Amz-Content-Sha256: STREAMING-AWS4-HMAC-SHA256-PAYLOAD X-Amz-Date: 20250910T103732Z X-Amz-Decoded-Content-Length: 16777216 Accept-Encoding: gzip HTTP/1.1 200 OK Content-Length: 0 Accept-Ranges: bytes Date: Wed, 10 Sep 2025 10:37:33 GMT Etag: aaba8aaf973419c68c3979f42c06c85c13a60461cebc72fcd7cf4d644c4b7ddd Server: NeoFS-S3-GW/0.41.1 X-Amz-Request-Id: fd50b15f-97f4-4a41-b5f7-49aadb8ed79b ---------END-HTTP--------- ---------START-HTTP--------- POST /warp/JQW18E%283/1.blggc1xHHQkm%283ef.rnd?uploadId=HTTpEJZXSUDYqQu1mLBJXGMDonKobMWJapjMwTLTk5jL HTTP/1.1 Host: #REMOVED User-Agent: MinIO (linux; amd64) minio-go/v7.0.83 warp/1.0.8 Content-Length: 2898 Authorization: AWS4-HMAC-SHA256 Credential=#REMOVED/20250910/default/s3/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED** Content-Type: application/octet-stream X-Amz-Content-Sha256: 0e9ca61b87ba9586e3f2b9648678556cb03a521561abd06735aea67692039c5d X-Amz-Date: 20250910T103733Z Accept-Encoding: gzip HTTP/1.1 400 Bad Request Content-Length: 501 Accept-Ranges: bytes Content-Type: application/xml Date: Wed, 10 Sep 2025 10:37:41 GMT Server: NeoFS-S3-GW/0.41.1 X-Amz-Request-Id: 6d4b8119-6a70-4175-b49b-c809b31f2e8e <?xml version="1.0" encoding="UTF-8"?> <Error><Code>InvalidPart</Code><Message>One or more of the specified parts could not be found. The part may not have been uploaded, or the specified entity tag may not match the part's entity tag.</Message><Key>JQW18E(3/1.blggc1xHHQkm(3ef.rnd</Key><BucketName>warp</BucketName><Resource>/warp/JQW18E(3/1.blggc1xHHQkm(3ef.rnd</Resource><RequestId>6d4b8119-6a70-4175-b49b-c809b31f2e8e</RequestId><HostId>cb7094ed-8662-467d-ac2e-429025b5b0f6</HostId></Error>---------END-HTTP--------- ---------START-HTTP--------- DELETE /warp/JQW18E%283/1.blggc1xHHQkm%283ef.rnd?uploadId=HTTpEJZXSUDYqQu1mLBJXGMDonKobMWJapjMwTLTk5jL HTTP/1.1 Host: #REMOVED User-Agent: MinIO (linux; amd64) minio-go/v7.0.83 warp/1.0.8 Authorization: AWS4-HMAC-SHA256 Credential=#REMOVED/20250910/default/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED** X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 X-Amz-Date: 20250910T103741Z Accept-Encoding: gzip HTTP/1.1 204 No Content Date: Wed, 10 Sep 2025 10:37:44 GMT X-Amz-Request-Id: 214848a1-6ce3-448d-b461-d8768ab8509a ---------END-HTTP--------- warp: <ERROR> upload error: One or more of the specified parts could not be found. The part may not have been uploaded, or the specified entity tag may not match the part's entity tag. ``` In logs: ``` error handler/util.go:34 call method {"status": 400, "request_id": "6d4b8119-6a70-4175-b49b-c809b31f2e8e", "method": "CompleteMultipartUpload", "bucket": "warp", "object": "JQW18E(3/1.blggc1xHHQkm(3ef.rnd", "description": "could not complete multipart upload", "error": "InvalidPart: 400 => One or more of the specified parts could not be found. The part may not have been uploaded, or the specified entity tag may not match the part's entity tag.", "uploadID": "HTTpEJZXSUDYqQu1mLBJXGMDonKobMWJapjMwTLTk5jL", "Key": "JQW18E(3/1.blggc1xHHQkm(3ef.rnd", "reqId": "6d4b8119-6a70-4175-b49b-c809b31f2e8e"} ```