Rework shard logical object metric #1554

New issue

Open

opened 2025-12-28 17:35:50 +00:00 by sami · 2 comments

sami commented

2025-12-28 17:35:50 +00:00

Owner

Originally created by @roman-khimov on GitHub (Dec 17, 2025).

I'm always frustrated when I think of logical object metric for shards. It's meaningless now:

it counts physical objects as they're added
it decrements them as they get inhumed

It can be thought of as a counter of available physical objects, but it fails this definition as well since expired objects won't be decremented from it until they're physically deleted by GC.

Describe the solution you'd like

Ideally it should track ROOT objects only, at least that's the way it's intuitively perceived by node users. It should also somehow react to expirations faster than GC, but that's somewhat problematic since we don't want to scan 1M of expired objects in one pass.

Describe alternatives you've considered

Keeping a broken counter doesn't look like a solution.

Additional context

#1771
#3522
#3615

Originally created by @roman-khimov on GitHub (Dec 17, 2025). ## Is your feature request related to a problem? Please describe. I'm always frustrated when I think of logical object metric for shards. It's meaningless now: * it counts physical objects as they're added * it decrements them as they get inhumed It can be thought of as a counter of available physical objects, but it fails this definition as well since expired objects won't be decremented from it until they're physically deleted by GC. ## Describe the solution you'd like Ideally it should track ROOT objects only, at least that's the way it's intuitively perceived by node users. It should also somehow react to expirations faster than GC, but that's somewhat problematic since we don't want to scan 1M of expired objects in one pass. ## Describe alternatives you've considered Keeping a broken counter doesn't look like a solution. ## Additional context #1771 #3522 #3615

sami added the

enhancement

neofs-storage

labels

2025-12-28 17:35:50 +00:00

sami commented

2025-12-28 17:35:50 +00:00

Author

Owner

@roman-khimov commented on GitHub (Dec 24, 2025):

My proposal is to not count "logical" things at all and just follow the entities we know from the protocol and DB. "Logical" is too fragile. What we can expose is:

phy, as in physical objects of all types, normally it's exactly the same as the number of links in FSTree
root, following the logic of ROOT filter, that's "objects" users care about
tombstone, type-based
lock, type-based
link, type-based
garbage, a counter of objects marked as garbage in the DB, mostly this happens on deletion, but other processes can affect it as well

"Logical" is almost phy-garbage, but it's not exactly this, so this better be left for external analysis.

@roman-khimov commented on GitHub (Dec 24, 2025): My proposal is to not count "logical" things at all and just follow the entities we know from the protocol and DB. "Logical" is too fragile. What we can expose is: * phy, as in physical objects _of all types_, normally it's exactly the same as the number of links in FSTree * root, following the logic of ROOT filter, that's "objects" users care about * tombstone, type-based * lock, type-based * link, type-based * garbage, a counter of objects marked as garbage in the DB, mostly this happens on deletion, but other processes can affect it as well "Logical" is almost `phy-garbage`, but it's not exactly this, so this better be left for external analysis.

sami commented

2025-12-28 17:35:51 +00:00

Author

Owner

@roman-khimov commented on GitHub (Dec 25, 2025):

"Regular" (type-based) and "all" (as in meta 0 prefix) can be considered as well.

"Container volume bucket" needs to be reworked at the same time, it can also use some simple prefix(es?) in meta, current bucket scheme is too complex for this simple container-specific data.

@roman-khimov commented on GitHub (Dec 25, 2025): "Regular" (type-based) and "all" (as in meta 0 prefix) can be considered as well. "Container volume bucket" needs to be reworked at the same time, it can also use some simple prefix(es?) in meta, current bucket scheme is too complex for this simple container-specific data.