Meta: optimize indexing objects #1430

New issue

Open

opened 2025-12-28 17:22:58 +00:00 by sami · 1 comment

sami commented

2025-12-28 17:22:58 +00:00

Owner

Originally created by @carpawell on GitHub (Jun 16, 2025).

It is known that it is possible to PUT objects more often than meta indexing can do: 1K queue is filled quite fast (minutes on real machines with real networks) and even 1M queue does not stand forever.

Expected Behavior

Handing objects from a block should take time that can compete block time itself. In our meta tests it is 50/100ms.

Current Behavior

Handling notifications from block takes longer than the block lasts, block queue fills quite fast.

Possible Solution

Cache HEADers in memory as objects are stored in the storage engine
Optimize indexing storage
...

Steps to Reproduce (for bugs)

Longivity PUT tests with small object sizes.

Regression

Your Environment

0.47.0

Originally created by @carpawell on GitHub (Jun 16, 2025).  It is known that it is possible to PUT objects more often than meta indexing can do: [1K](https://github.com/nspcc-dev/neofs-node/blob/d651e2d3b0a2315d5e8e0f6d8c0fbe899c443b75/pkg/services/meta/meta.go#L137) queue is filled quite fast (minutes on real machines with real networks) and even 1M queue does not stand forever. ## Expected Behavior Handing objects from a block should take time that can compete block time itself. In our meta tests it is 50/100ms. ## Current Behavior Handling notifications from block takes longer than the block lasts, block queue fills quite fast. ## Possible Solution 1. Cache HEADers in memory as objects are stored in the storage engine 2. Optimize indexing storage 3. ... ## Steps to Reproduce (for bugs) Longivity PUT tests with small object sizes. ## Regression No ## Your Environment 0.47.0