nspcc-dev/neofs-s3-gw
1
0
Fork 0
mirror of https://github.com/nspcc-dev/neofs-s3-gw.git synced 2026-03-01 04:29:15 +00:00
Code Issues 24 Projects Packages Wiki Activity

The gw deletes a null object in versioned bucket from NeoFS if version is not specified in a request #259

New issue
Closed
opened 2025-12-28 17:36:46 +00:00 by sami · 6 comments
sami commented 2025-12-28 17:36:46 +00:00
Owner
Copy link

Originally created by @masterSplinter01 on GitHub (Jun 17, 2022).

Originally assigned to: @KirillovDenis on GitHub.

tree-service branch:

$ aws s3api create-bucket --bucket test7 --endpoint-url http://127.0.0.1:8084

$ aws s3api put-object --key a --body ~/cat.txt  --bucket test7 --endpoint-url http://127.0.0.1:8084 
{
  "ETag": "93d30be4ee6dd625821e8b35f93af305f3f084aa595440a5e696d11dbbc6516a"
  
$ aws s3api put-bucket-versioning --versioning-configuration Status=Enabled  --bucket test7 --endpoint-url http://127.0.0.1:8084

$ aws s3api put-object --key a --body ~/cat.txt  --bucket test7 --endpoint-url http://127.0.0.1:8084
{
  "ETag": "93d30be4ee6dd625821e8b35f93af305f3f084aa595440a5e696d11dbbc6516a",
  "VersionId": "EFf9FAJb48X1gkN5Ca4MjYzWVxkGdA7XWPa6X8d57ZD2"
}

$ neofs-cli container list-objects --wallet ~/work/neofs-dev-env/wallets/wallet.json --rpc-endpoint 192.168.130.71:8080  --cid DdPGrNVp5vu9ja7ayEp7SLKZfi8rRosMsBrqW1KN4CEo
Enter password > 
Gqb3zTbsT9aHPwtgyaHpgwGYqcup6yJmCZ12WDA87zU5
EFf9FAJb48X1gkN5Ca4MjYzWVxkGdA7XWPa6X8d57ZD2

$ aws s3api delete-object --key a --bucket test7  --endpoint-url http://127.0.0.1:8084

$ aws s3api list-object-versions --bucket test7 --endpoint-url http://127.0.0.1:8084| jq                                                      ✔ 
{
  "Versions": [
    {
      "ETag": "93d30be4ee6dd625821e8b35f93af305f3f084aa595440a5e696d11dbbc6516a",
      "Size": 110,
      "Key": "a",
      "VersionId": "EFf9FAJb48X1gkN5Ca4MjYzWVxkGdA7XWPa6X8d57ZD2",
      "IsLatest": false,
      "LastModified": "2022-06-17T23:25:04+00:00",
      "Owner": {
        "DisplayName": "NbUgTSFvPmsRxmGeWpuuGeJUoRoi6PErcM",
        "ID": "NbUgTSFvPmsRxmGeWpuuGeJUoRoi6PErcM"
      }
    }
  ],
  "DeleteMarkers": [
    {
      "Owner": {
        "DisplayName": "NbUgTSFvPmsRxmGeWpuuGeJUoRoi6PErcM",
        "ID": "NbUgTSFvPmsRxmGeWpuuGeJUoRoi6PErcM"
      },
      "Key": "a",
      "VersionId": "CTrY8CcKp1AG1AgjSUtPrxCo8zjgM4EkRraUCBauMotr",
      "IsLatest": true,
      "LastModified": "2022-06-17T23:26:35+00:00"
    }
  ]
}


$ neofs-cli container list-objects --wallet ~/work/neofs-dev-env/wallets/wallet.json --rpc-endpoint 192.168.130.71:8080  --cid DdPGrNVp5vu9ja7ayEp7SLKZfi8rRosMsBrqW1KN4CEo
Enter password > 
EFf9FAJb48X1gkN5Ca4MjYzWVxkGdA7XWPa6X8d57ZD2
Originally created by @masterSplinter01 on GitHub (Jun 17, 2022). Originally assigned to: @KirillovDenis on GitHub. `tree-service` branch: ``` $ aws s3api create-bucket --bucket test7 --endpoint-url http://127.0.0.1:8084 $ aws s3api put-object --key a --body ~/cat.txt --bucket test7 --endpoint-url http://127.0.0.1:8084 { "ETag": "93d30be4ee6dd625821e8b35f93af305f3f084aa595440a5e696d11dbbc6516a" $ aws s3api put-bucket-versioning --versioning-configuration Status=Enabled --bucket test7 --endpoint-url http://127.0.0.1:8084 $ aws s3api put-object --key a --body ~/cat.txt --bucket test7 --endpoint-url http://127.0.0.1:8084 { "ETag": "93d30be4ee6dd625821e8b35f93af305f3f084aa595440a5e696d11dbbc6516a", "VersionId": "EFf9FAJb48X1gkN5Ca4MjYzWVxkGdA7XWPa6X8d57ZD2" } $ neofs-cli container list-objects --wallet ~/work/neofs-dev-env/wallets/wallet.json --rpc-endpoint 192.168.130.71:8080 --cid DdPGrNVp5vu9ja7ayEp7SLKZfi8rRosMsBrqW1KN4CEo Enter password > Gqb3zTbsT9aHPwtgyaHpgwGYqcup6yJmCZ12WDA87zU5 EFf9FAJb48X1gkN5Ca4MjYzWVxkGdA7XWPa6X8d57ZD2 $ aws s3api delete-object --key a --bucket test7 --endpoint-url http://127.0.0.1:8084 $ aws s3api list-object-versions --bucket test7 --endpoint-url http://127.0.0.1:8084| jq  ✔ { "Versions": [ { "ETag": "93d30be4ee6dd625821e8b35f93af305f3f084aa595440a5e696d11dbbc6516a", "Size": 110, "Key": "a", "VersionId": "EFf9FAJb48X1gkN5Ca4MjYzWVxkGdA7XWPa6X8d57ZD2", "IsLatest": false, "LastModified": "2022-06-17T23:25:04+00:00", "Owner": { "DisplayName": "NbUgTSFvPmsRxmGeWpuuGeJUoRoi6PErcM", "ID": "NbUgTSFvPmsRxmGeWpuuGeJUoRoi6PErcM" } } ], "DeleteMarkers": [ { "Owner": { "DisplayName": "NbUgTSFvPmsRxmGeWpuuGeJUoRoi6PErcM", "ID": "NbUgTSFvPmsRxmGeWpuuGeJUoRoi6PErcM" }, "Key": "a", "VersionId": "CTrY8CcKp1AG1AgjSUtPrxCo8zjgM4EkRraUCBauMotr", "IsLatest": true, "LastModified": "2022-06-17T23:26:35+00:00" } ] } $ neofs-cli container list-objects --wallet ~/work/neofs-dev-env/wallets/wallet.json --rpc-endpoint 192.168.130.71:8080 --cid DdPGrNVp5vu9ja7ayEp7SLKZfi8rRosMsBrqW1KN4CEo Enter password > EFf9FAJb48X1gkN5Ca4MjYzWVxkGdA7XWPa6X8d57ZD2 ```
sami 2025-12-28 17:36:46 +00:00
sami commented 2025-12-28 17:36:46 +00:00
Author
Owner
Copy link

@KirillovDenis commented on GitHub (Jun 20, 2022):

As I understand it's expected behavior. https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObject.html (Removes the null version (if there is one) of an object and inserts a delete marker, which becomes the latest version of the object.)

@KirillovDenis commented on GitHub (Jun 20, 2022): As I understand it's expected behavior. https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObject.html (`Removes the null version (if there is one) of an object and inserts a delete marker, which becomes the latest version of the object.`)
sami commented 2025-12-28 17:36:46 +00:00
Author
Owner
Copy link

@masterSplinter01 commented on GitHub (Jun 20, 2022):

Amazon doesn't delete null version:

$ aws s3api list-object-versions --bucket kkekteeva12 | jq                                                                                  
{
  "Versions": [
    {
      "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"",
      "Size": 87422,
      "StorageClass": "STANDARD",
      "Key": "b",
      "VersionId": "s_sJlN890ltOKD1a9Gz0pQ5fm_vHD0Hm",
      "IsLatest": true,
      "LastModified": "2022-06-20T06:23:23+00:00",
      "Owner": {
        "DisplayName": "ops",
        "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b"
      }
    },
    {
      "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"",
      "Size": 87422,
      "StorageClass": "STANDARD",
      "Key": "b",
      "VersionId": "El2XHKZ42Z.2KPm4r79haD2HBJCbZ95D",
      "IsLatest": false,
      "LastModified": "2022-06-20T06:23:17+00:00",
      "Owner": {
        "DisplayName": "ops",
        "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b"
      }
    },
    {
      "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"",
      "Size": 87422,
      "StorageClass": "STANDARD",
      "Key": "b",
      "VersionId": "null",
      "IsLatest": false,
      "LastModified": "2022-06-20T06:22:56+00:00",
      "Owner": {
        "DisplayName": "ops",
        "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b"
      }
    }
  ]
}

$ aws s3api delete-object --key a --bucket kkekteeva12                                                                                  
{
  "DeleteMarker": true,
  "VersionId": "MgyCrFx0AaqQhUfaEIYU56Guw7zrAORP"
}

$ aws s3api list-object-versions --bucket kkekteeva12                                                                              
{
  "Versions": [
    {
      "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"",
      "Size": 87422,
      "StorageClass": "STANDARD",
      "Key": "b",
      "VersionId": "s_sJlN890ltOKD1a9Gz0pQ5fm_vHD0Hm",
      "IsLatest": true,
      "LastModified": "2022-06-20T06:23:23+00:00",
      "Owner": {
        "DisplayName": "ops",
        "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b"
      }
    },
    {
      "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"",
      "Size": 87422,
      "StorageClass": "STANDARD",
      "Key": "b",
      "VersionId": "El2XHKZ42Z.2KPm4r79haD2HBJCbZ95D",
      "IsLatest": false,
      "LastModified": "2022-06-20T06:23:17+00:00",
      "Owner": {
        "DisplayName": "ops",
        "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b"
      }
    },
    {
      "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"",
      "Size": 87422,
      "StorageClass": "STANDARD",
      "Key": "b",
      "VersionId": "null",
      "IsLatest": false,
      "LastModified": "2022-06-20T06:22:56+00:00",
      "Owner": {
        "DisplayName": "ops",
        "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b"
      }
    }
  ],
  "DeleteMarkers": [
    {
      "Owner": {
        "DisplayName": "ops",
        "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b"
      },
      "Key": "a",
      "VersionId": "MgyCrFx0AaqQhUfaEIYU56Guw7zrAORP",
      "IsLatest": true,
      "LastModified": "2022-06-20T06:24:00+00:00"
    }
  ]
}

Also, I don't see any information about deleting of null version here:
https://docs.aws.amazon.com/AmazonS3/latest/userguide/DeleteMarker.html

But in the schemes null version shouldn't be deleted after delete-marker has been placed.

@masterSplinter01 commented on GitHub (Jun 20, 2022): Amazon doesn't delete `null` version: ``` $ aws s3api list-object-versions --bucket kkekteeva12 | jq { "Versions": [ { "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"", "Size": 87422, "StorageClass": "STANDARD", "Key": "b", "VersionId": "s_sJlN890ltOKD1a9Gz0pQ5fm_vHD0Hm", "IsLatest": true, "LastModified": "2022-06-20T06:23:23+00:00", "Owner": { "DisplayName": "ops", "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b" } }, { "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"", "Size": 87422, "StorageClass": "STANDARD", "Key": "b", "VersionId": "El2XHKZ42Z.2KPm4r79haD2HBJCbZ95D", "IsLatest": false, "LastModified": "2022-06-20T06:23:17+00:00", "Owner": { "DisplayName": "ops", "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b" } }, { "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"", "Size": 87422, "StorageClass": "STANDARD", "Key": "b", "VersionId": "null", "IsLatest": false, "LastModified": "2022-06-20T06:22:56+00:00", "Owner": { "DisplayName": "ops", "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b" } } ] } $ aws s3api delete-object --key a --bucket kkekteeva12 { "DeleteMarker": true, "VersionId": "MgyCrFx0AaqQhUfaEIYU56Guw7zrAORP" } $ aws s3api list-object-versions --bucket kkekteeva12 { "Versions": [ { "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"", "Size": 87422, "StorageClass": "STANDARD", "Key": "b", "VersionId": "s_sJlN890ltOKD1a9Gz0pQ5fm_vHD0Hm", "IsLatest": true, "LastModified": "2022-06-20T06:23:23+00:00", "Owner": { "DisplayName": "ops", "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b" } }, { "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"", "Size": 87422, "StorageClass": "STANDARD", "Key": "b", "VersionId": "El2XHKZ42Z.2KPm4r79haD2HBJCbZ95D", "IsLatest": false, "LastModified": "2022-06-20T06:23:17+00:00", "Owner": { "DisplayName": "ops", "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b" } }, { "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"", "Size": 87422, "StorageClass": "STANDARD", "Key": "b", "VersionId": "null", "IsLatest": false, "LastModified": "2022-06-20T06:22:56+00:00", "Owner": { "DisplayName": "ops", "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b" } } ], "DeleteMarkers": [ { "Owner": { "DisplayName": "ops", "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b" }, "Key": "a", "VersionId": "MgyCrFx0AaqQhUfaEIYU56Guw7zrAORP", "IsLatest": true, "LastModified": "2022-06-20T06:24:00+00:00" } ] } ``` Also, I don't see any information about deleting of `null` version here: https://docs.aws.amazon.com/AmazonS3/latest/userguide/DeleteMarker.html But in the schemes `null` version shouldn't be deleted after `delete-marker` has been placed.
sami commented 2025-12-28 17:36:46 +00:00
Author
Owner
Copy link

@KirillovDenis commented on GitHub (Jun 20, 2022):

Sorry, but your output is a little odd because you tried to delete the key a though in bucket you have only key b with null version.

@KirillovDenis commented on GitHub (Jun 20, 2022): Sorry, but your output is a little odd because you tried to delete the key `a` though in bucket you have only key `b` with `null` version.
sami commented 2025-12-28 17:36:46 +00:00
Author
Owner
Copy link

@masterSplinter01 commented on GitHub (Jun 20, 2022):

Sorry, typo, tried to remove b :

$ aws s3api delete-object --key b --bucket kkekteeva12   
{
  "DeleteMarker": true,
  "VersionId": "L.VuItIBg7rDa1WWvoD5wTIK6DCo3k1k"
}

$ aws s3api list-object-versions --bucket kkekteeva12 
{
  "Versions": [
    {
      "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"",
      "Size": 87422,
      "StorageClass": "STANDARD",
      "Key": "b",
      "VersionId": "s_sJlN890ltOKD1a9Gz0pQ5fm_vHD0Hm",
      "IsLatest": false,
      "LastModified": "2022-06-20T06:23:23+00:00",
      "Owner": {
        "DisplayName": "ops",
        "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b"
      }
    },
    {
      "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"",
      "Size": 87422,
      "StorageClass": "STANDARD",
      "Key": "b",
      "VersionId": "El2XHKZ42Z.2KPm4r79haD2HBJCbZ95D",
      "IsLatest": false,
      "LastModified": "2022-06-20T06:23:17+00:00",
      "Owner": {
        "DisplayName": "ops",
        "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b"
      }
    },
    {
      "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"",
      "Size": 87422,
      "StorageClass": "STANDARD",
      "Key": "b",
      "VersionId": "null",
      "IsLatest": false,
      "LastModified": "2022-06-20T06:22:56+00:00",
      "Owner": {
        "DisplayName": "ops",
        "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b"
      }
    }
  ],
  "DeleteMarkers": [
    {
      "Owner": {
        "DisplayName": "ops",
        "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b"
      },
      "Key": "a",
      "VersionId": "MgyCrFx0AaqQhUfaEIYU56Guw7zrAORP",
      "IsLatest": true,
      "LastModified": "2022-06-20T06:24:00+00:00"
    },
    {
      "Owner": {
        "DisplayName": "ops",
        "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b"
      },
      "Key": "b",
      "VersionId": "L.VuItIBg7rDa1WWvoD5wTIK6DCo3k1k",
      "IsLatest": true,
      "LastModified": "2022-06-20T16:06:11+00:00"
    }
  ]
}

Nothing changed

@masterSplinter01 commented on GitHub (Jun 20, 2022): Sorry, typo, tried to remove `b` : ``` $ aws s3api delete-object --key b --bucket kkekteeva12 { "DeleteMarker": true, "VersionId": "L.VuItIBg7rDa1WWvoD5wTIK6DCo3k1k" } $ aws s3api list-object-versions --bucket kkekteeva12 { "Versions": [ { "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"", "Size": 87422, "StorageClass": "STANDARD", "Key": "b", "VersionId": "s_sJlN890ltOKD1a9Gz0pQ5fm_vHD0Hm", "IsLatest": false, "LastModified": "2022-06-20T06:23:23+00:00", "Owner": { "DisplayName": "ops", "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b" } }, { "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"", "Size": 87422, "StorageClass": "STANDARD", "Key": "b", "VersionId": "El2XHKZ42Z.2KPm4r79haD2HBJCbZ95D", "IsLatest": false, "LastModified": "2022-06-20T06:23:17+00:00", "Owner": { "DisplayName": "ops", "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b" } }, { "ETag": "\"bb285fe38300f07d7b755e3b45c0020d\"", "Size": 87422, "StorageClass": "STANDARD", "Key": "b", "VersionId": "null", "IsLatest": false, "LastModified": "2022-06-20T06:22:56+00:00", "Owner": { "DisplayName": "ops", "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b" } } ], "DeleteMarkers": [ { "Owner": { "DisplayName": "ops", "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b" }, "Key": "a", "VersionId": "MgyCrFx0AaqQhUfaEIYU56Guw7zrAORP", "IsLatest": true, "LastModified": "2022-06-20T06:24:00+00:00" }, { "Owner": { "DisplayName": "ops", "ID": "68e12342e5971d09ced74634fc7cc0fc5f01419d06b76353d42ce3108367183b" }, "Key": "b", "VersionId": "L.VuItIBg7rDa1WWvoD5wTIK6DCo3k1k", "IsLatest": true, "LastModified": "2022-06-20T16:06:11+00:00" } ] } ``` Nothing changed
sami commented 2025-12-28 17:36:46 +00:00
Author
Owner
Copy link

@alexvanin commented on GitHub (Jun 23, 2022):

Seems like this is correct behavior when versioning is off, but in those examples versioning is on.

@alexvanin commented on GitHub (Jun 23, 2022): Seems like this is correct behavior when versioning is off, but in those examples versioning is on.
sami commented 2025-12-28 17:36:46 +00:00
Author
Owner
Copy link

@KirillovDenis commented on GitHub (Jul 6, 2022):

Done in #554

@KirillovDenis commented on GitHub (Jul 6, 2022): Done in #554
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
1255-session-token-v2
new-get
logger
experimental/unsigned-responses
experimental/get-perf
debugprint
ezayats/bump-s3-tests
955-after-a-parallel-deletion-of-versioned-objects-list-objects-returns-info-about-them
v0.42.0
v0.41.5
v0.41.4
v0.41.3
v0.41.2
v0.41.1
v0.41.0
v0.40.0
v0.39.0
v0.38.0
v0.37.0
v0.36.1
v0.36.0
v0.35.0
v0.34.1
v0.34.0
v0.33.0
v0.32.0
v0.31.1
v0.31.0
v0.30.1
v0.30.0
v0.29.0
v0.28.2
v0.28.1
v0.28.0
v0.27.1
v0.27.0
v0.26.1
v0.26.0
v0.25.1
v0.25.0
v0.24.0
v0.23.0
v0.22.0
v0.21.1
v0.21.0
v0.20.0
v0.19.0
v0.18.0
v0.17.0
v0.17.0-rc.1
v0.16.0
v0.15.1-pre
v0.15.0
v0.14.2
v0.14.1
v0.14.0
v0.13.0
v0.12.0-rc.3
v0.12.0-rc.2
v0.12.0-rc.1
Labels
Clear labels   
I2

   
I2

   
I3

   
I4

   
S2

   
S3

   
S4

   
S4

   
U0

   
U1

   
U2

   
U2

   
U3

   
U4

   
U4

   
auth-mate

   
blocked

   
bug

   
config

   
dependencies

   
discussion

   
documentation

   
enhancement

   
epic

   
feature

   
go

   
good first issue

   
help wanted

   
performance

   
question

   
security

   
test

   
tree-service

   
tree-service
No labels
I2
I2
I3
I4
S2
S3
S4
S4
U0
U1
U2
U2
U3
U4
U4
auth-mate
blocked
bug
config
dependencies
discussion
documentation
enhancement
epic
feature
go
good first issue
help wanted
performance
question
security
test
tree-service
tree-service
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
nspcc-dev/neofs-s3-gw#259
No description provided.