client: Optimize signature buffer pool #264

New issue

Open

opened 2025-12-28 18:07:36 +00:00 by sami · 2 comments

sami commented

2025-12-28 18:07:36 +00:00

Owner

Originally created by @cthulhu-rider on GitHub (Dec 11, 2024).

Client allows to specify buffer pool for signing defaulting to nspcc-dev/neofs-sdk-go@b312ddb08b/client/client.go (L96-L105)

to sign any request, client selects an arbitrary buffer from the pool. If this buffer has sufficient size to encode the signed request parts, they are written into it. Otherwise, temp buffer of needed size is allocated for each part via make

i see following drawbacks of this approach:

buffers of different size are mixed in the pool. Some signed entities are empty (like NetmapService request bodies), some are pretty small <=1K (meta headers, bodies of almost all requests), some can be up to 4M (object PUT payload chunks)
make-allocated buffers aint pooled

2 just does not allow reusing the buffer. 1 may worsen caching in some cases:

when next 4M object payload chunk is to-be-signed, we are likely to select small buffer which leads . At the same time, pool can still have big buffers from previous chunks
when small entity is signed, it can acquire big buffer, however, most of its len will not be used. This also affects 1

Describe the solution you'd like

use size-based buffer pool. Size of any signed part is known in advance. We can precalc it and select the buffer of the closest size
if buffer is dynamically allocated for signing anyway, pool it

Describe alternatives you've considered

no except keep as it is

Additional context

blocked by #665 cuz it can make this irrelevant

Originally created by @cthulhu-rider on GitHub (Dec 11, 2024). ## Is your feature request related to a problem? Please describe. `Client` allows to [specify](https://pkg.go.dev/github.com/nspcc-dev/neofs-sdk-go/client#PrmInit.SetSignMessageBuffers) buffer pool for signing defaulting to https://github.com/nspcc-dev/neofs-sdk-go/blob/b312ddb08bc97fec3367351775f05c4d08309512/client/client.go#L96-L105 to sign any request, client selects an arbitrary buffer from the pool. If this buffer has sufficient size to encode the signed request parts, they are written into it. Otherwise, temp buffer of needed size is allocated for each part via `make` i see following drawbacks of this approach: 1. buffers of different size are mixed in the pool. Some signed entities are empty (like NetmapService request bodies), some are pretty small <=1K (meta headers, bodies of almost all requests), some can be up to 4M (object PUT payload chunks) 2. `make`-allocated buffers aint pooled 2 just does not allow reusing the buffer. 1 may worsen caching in some cases: * when next 4M object payload chunk is to-be-signed, we are likely to select small buffer which leads . At the same time, pool can still have big buffers from previous chunks * when small entity is signed, it can acquire big buffer, however, most of its len will not be used. This also affects 1 ## Describe the solution you'd like 1. use size-based buffer pool. Size of any signed part is known in advance. We can precalc it and select the buffer of the closest size 2. if buffer is dynamically allocated for signing anyway, pool it ## Describe alternatives you've considered no except keep as it is ## Additional context * blocked by #665 cuz it can make this irrelevant

sami added the

enhancement

client

blocked

labels

2025-12-28 18:07:36 +00:00

sami commented

2025-12-28 18:07:37 +00:00

Author

Owner

@cthulhu-rider commented on GitHub (Dec 11, 2024):

also wanna mention that when verifying responses, the buffers are always dynamic. They can also be pooled. This deserves a separate issue, but i suggest to create it only if current one is unblocked

@cthulhu-rider commented on GitHub (Dec 11, 2024): also wanna mention that when verifying responses, the buffers are always dynamic. They can also be pooled. This deserves a separate issue, but i suggest to create it only if current one is unblocked

sami commented

2025-12-28 18:07:37 +00:00

Author

Owner

@roman-khimov commented on GitHub (Dec 17, 2024):

We're misusing pools more often than not, my suggestion is to not touch it for now.

@roman-khimov commented on GitHub (Dec 17, 2024): We're misusing pools more often than not, my suggestion is to not touch it for now.