Fix WC size estimations #1323

New issue

Closed

opened 2025-12-28 17:22:35 +00:00 by sami · 2 comments

sami commented

2025-12-28 17:22:35 +00:00

Owner

Originally created by @carpawell on GitHub (Jan 29, 2025).

Originally assigned to: @End-rey on GitHub.

Currently, its size (to control cache size and not overflow) is defined as numOfObjs*theBiggestPossibleObj. It is not precise at all. It was done mostly because of uncontrollable bbolt db size for small objects. But there is no bbolt db in WC now: https://github.com/nspcc-dev/neofs-node/pull/3091

Expected Behavior

Taken space is smth like a sum of every object that is currently stored in WC.

Current Behavior

Taken space is a magic product: nspcc-dev/neofs-node@25314f58cb/pkg/local_object_storage/writecache/state.go (L11-L13).

Possible Solution

Store the sum from Expected Behavior and change it accordingly with every PUT and DELETE.

Steps to Reproduce (for bugs)

Look at nspcc-dev/neofs-node@25314f58cb/pkg/local_object_storage/writecache/state.go (L11-L13).

Context

https://github.com/nspcc-dev/neofs-node/pull/3091#pullrequestreview-2580795118

Regression

No.

Your Environment

v0.44.2.

Originally created by @carpawell on GitHub (Jan 29, 2025). Originally assigned to: @End-rey on GitHub. Currently, its size (to control cache size and not overflow) is defined as `numOfObjs*theBiggestPossibleObj`. It is not precise at all. It was done mostly because of uncontrollable bbolt db size for small objects. But there is no bbolt db in WC now: https://github.com/nspcc-dev/neofs-node/pull/3091 ## Expected Behavior Taken space is smth like a sum of every object that is currently stored in WC. ## Current Behavior Taken space is a magic product: https://github.com/nspcc-dev/neofs-node/blob/25314f58cb6d1066b10a405a12240dd830587de2/pkg/local_object_storage/writecache/state.go#L11-L13. ## Possible Solution Store the sum from `Expected Behavior` and change it accordingly with every PUT and DELETE. ## Steps to Reproduce (for bugs) Look at https://github.com/nspcc-dev/neofs-node/blob/25314f58cb6d1066b10a405a12240dd830587de2/pkg/local_object_storage/writecache/state.go#L11-L13. ## Context https://github.com/nspcc-dev/neofs-node/pull/3091#pullrequestreview-2580795118 ## Regression No. ## Your Environment v0.44.2.

sami

2025-12-28 17:22:35 +00:00

closed this issue
added the
bug

neofs-storage

U2

S4

I4
labels

sami commented

2025-12-28 17:22:35 +00:00

Author

Owner

@roman-khimov commented on GitHub (Jan 31, 2025):

Keep object list in memory along with sizes (map address to size), then have an atomic variable with overall size. Add to both when storing, subtract when flushing.

@roman-khimov commented on GitHub (Jan 31, 2025): Keep object list in memory along with sizes (map address to size), then have an atomic variable with overall size. Add to both when storing, subtract when flushing.

sami commented

2025-12-28 17:22:35 +00:00

Author

Owner

@roman-khimov commented on GitHub (Jan 31, 2025):

Or maybe not an atomic var since we'll have to lock the map anyway.

@roman-khimov commented on GitHub (Jan 31, 2025): Or maybe not an atomic var since we'll have to lock the map anyway.

sami referenced this issue

2025-12-28 17:22:44 +00:00

Support Neo N3 account auth #1368