Local object store behavior for Put of already existing objects #1123

New issue

Open

opened 2025-12-28 17:21:52 +00:00 by sami · 1 comment

sami commented

2025-12-28 17:21:52 +00:00

Owner

Originally created by @roman-khimov on GitHub (Sep 9, 2023).

common.Storage interface exposes a Put() function that's supposed to be pushing objects into the store. One can Put an object and then Put it again. What happens next? Currently:

Put succeeds, returning nil
data is overwritten (FSTree/Peapod/bbcz)

In general we suppose that objects are immutable and we have the same data for the same object. Which means that the first point is likely correct. If that's the case then the second one doesn't matter. However if you're to change the logic of FSTree to preserve an already existing file TestRefillMetabaseCorrupted suddenly starts to fail because it relies on this overwriting semantics.

Describe the solution you'd like

We need to specify what the proper behavior is for all stores.

Describe alternatives you've considered

And it's not that easy, because:

we can have concurrent writers pushing the same data, these prefer to see nil and don't care about overwriting, that's the most common case
old data is supposed to be good, so overwriting is not necessary
we can also have errors leading to different data being pushed for the same address, in this case keeping old data is also more safe
but in this case returning nil looks bad since writer would think that he has successfully pushed something into the store, but any Get would reveal that it's not exactly true (the data is different)
if we're to overwrite then nil makes total sense
old data can be corrupted as well, so we may want to do a forced write (although it can be done with Delete/Put)
checking data consistency and returning different status based on that sounds nice, but can lead to substantial overhead that is not justified for the most common case
always returning EEXIST is an option (let the upper layer handle), but can be daunting for callers

So, what do we really want from our stores?

Originally created by @roman-khimov on GitHub (Sep 9, 2023). ## Is your feature request related to a problem? Please describe. `common.Storage` interface exposes a `Put()` function that's supposed to be pushing objects into the store. One can `Put` an object and then `Put` it again. What happens next? Currently: * `Put` succeeds, returning `nil` * data is overwritten (FSTree/Peapod/bbcz) In general we suppose that objects are immutable and we have the same data for the same object. Which means that the first point is likely correct. If that's the case then the second one doesn't matter. However if you're to change the logic of FSTree to preserve an already existing file `TestRefillMetabaseCorrupted` suddenly starts to fail because it relies on this overwriting semantics. ## Describe the solution you'd like We need to specify what the proper behavior is for all stores. ## Describe alternatives you've considered And it's not that easy, because: * we can have concurrent writers pushing the same data, these prefer to see `nil` and don't care about overwriting, that's the most common case * old data is supposed to be good, so overwriting is not necessary * we can also have errors leading to different data being pushed for the same address, in this case keeping old data is also more safe * but in this case returning `nil` looks bad since writer would think that he has successfully pushed something into the store, but any `Get` would reveal that it's not exactly true (the data is different) * if we're to overwrite then `nil` makes total sense * old data _can_ be corrupted as well, so we may want to do a forced write (although it can be done with Delete/Put) * checking data consistency and returning different status based on that sounds nice, but can lead to substantial overhead that is not justified for the most common case * always returning EEXIST is an option (let the upper layer handle), but can be daunting for callers So, what do we really want from our stores?

sami added the

neofs-storage

discussion

labels

2025-12-28 17:21:52 +00:00

sami commented

2025-12-28 17:21:53 +00:00

Author

Owner

@carpawell commented on GitHub (Sep 18, 2023):

I would say that double writing is not smth really expected (with a really few exceptions) and a separate status and a log record are debug-friendly.

@carpawell commented on GitHub (Sep 18, 2023): I would say that double writing is not smth really expected (with a really few exceptions) and a separate status and a log record are debug-friendly.