Complex object is not readable through non-container node #1069

New issue

Closed

opened 2025-12-28 17:21:40 +00:00 by sami · 9 comments

sami commented 2025-12-28 17:21:40 +00:00

Owner

Copy link

Originally created by @cthulhu-rider on GitHub (Jul 20, 2023).

Originally assigned to: @cthulhu-rider on GitHub.

i tried to exec several ops with complex objects on https://github.com/nspcc-dev/neofs-dev-env

steps to reproduce:

1. run dev env with tuned object size limit nspcc-dev/neofs-dev-env@a9b92f53f7/neofs-adm.yml (L3)
2. make prepare.ir
3. create container which does not (!) include all 4 storage nodes

$ neofs-cli -c bin/cli.yml container create --policy 'REP 1' --await

4. store object with data bigger than the size limit set in step 1

$ neofs-cli -c bin/cli.yml object put --cid 8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon --file Makefile
[Makefile] Object successfully stored
  OID: 9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm
  CID: 8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon

5. try to read objects from each node

$ neofs-cli -c bin/cli.yml object get --cid 8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon --oid 9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm

3 nodes respond correctly, 1 fails with error

rpc error: read object header: status: code = 2049 message = object not found

log record on the failed node about the operation:

2023-07-20T16:36:03.960Z	debug	get/get.go:87	serving request...	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false}
2023-07-20T16:36:03.960Z	debug	get/local.go:25	local get failed	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"}
2023-07-20T16:36:03.960Z	debug	get/get.go:108	operation finished with error	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"}
2023-07-20T16:36:03.960Z	debug	get/container.go:18	trying to execute in container...	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false, "netmap lookup depth": 0}
2023-07-20T16:36:03.960Z	debug	get/container.go:46	process epoch	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false, "number": 3}
2023-07-20T16:36:03.960Z	debug	get/remote.go:14	processing node...	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false}
2023-07-20T16:36:03.962Z	debug	get/container.go:87	completing the operation	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false}
2023-07-20T16:36:03.962Z	debug	get/get.go:103	requested object is virtual	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false}
2023-07-20T16:36:03.962Z	debug	get/assemble.go:34	trying to assemble the object...	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false}
2023-07-20T16:36:03.962Z	debug	get/assemble.go:83	starting assembling from child	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false, "child ID": "94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf"}
2023-07-20T16:36:03.962Z	debug	get/get.go:87	serving request...	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false}
2023-07-20T16:36:03.962Z	debug	get/local.go:25	local get failed	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"}
2023-07-20T16:36:03.962Z	debug	get/get.go:108	operation finished with error	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"}
2023-07-20T16:36:03.962Z	debug	get/container.go:18	trying to execute in container...	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false, "netmap lookup depth": 0}
2023-07-20T16:36:03.962Z	debug	get/container.go:46	process epoch	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false, "number": 3}
2023-07-20T16:36:03.962Z	debug	get/remote.go:14	processing node...	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false}
2023-07-20T16:36:03.964Z	debug	get/remote.go:34	remote call failed	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "read object header: status: code = 2048 message = access to object operation denied"}
2023-07-20T16:36:03.964Z	debug	get/remote.go:14	processing node...	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false}
2023-07-20T16:36:03.966Z	debug	get/remote.go:34	remote call failed	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "read object header: status: code = 2048 message = access to object operation denied"}
2023-07-20T16:36:03.966Z	debug	get/remote.go:14	processing node...	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false}
2023-07-20T16:36:03.968Z	debug	get/remote.go:34	remote call failed	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "read object header: status: code = 2048 message = access to object operation denied"}
2023-07-20T16:36:03.968Z	debug	get/container.go:63	no more nodes, abort placement iteration	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false}
2023-07-20T16:36:03.968Z	debug	get/get.go:108	operation finished with error	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"}
2023-07-20T16:36:03.968Z	debug	get/assemble.go:76	could not init parent from child	{"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false}

seems like session is lost when out-of-container node spawns new requests or the session doesn't work

Versions

• neofs-cli v0.37.0
• neofs-dev-end a9b92f53f7913fc28d4eb307a671030d87a7ecab (with NeoFS v0.37.0 too)

Originally created by @cthulhu-rider on GitHub (Jul 20, 2023). Originally assigned to: @cthulhu-rider on GitHub. i tried to exec several ops with complex objects on https://github.com/nspcc-dev/neofs-dev-env steps to reproduce: 1. run dev env with tuned object size limit https://github.com/nspcc-dev/neofs-dev-env/blob/a9b92f53f7913fc28d4eb307a671030d87a7ecab/neofs-adm.yml#L3 2. `make prepare.ir` 3. create container which does not (!) include all 4 storage nodes ``` $ neofs-cli -c bin/cli.yml container create --policy 'REP 1' --await ``` 4. store object with data bigger than the size limit set in step 1 ``` $ neofs-cli -c bin/cli.yml object put --cid 8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon --file Makefile [Makefile] Object successfully stored OID: 9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm CID: 8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon ``` 5. try to read objects from each node ``` $ neofs-cli -c bin/cli.yml object get --cid 8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon --oid 9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm ``` 3 nodes respond correctly, 1 fails with error ``` rpc error: read object header: status: code = 2049 message = object not found ``` log record on the failed node about the operation: ``` 2023-07-20T16:36:03.960Z debug get/get.go:87 serving request... {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false} 2023-07-20T16:36:03.960Z debug get/local.go:25 local get failed {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"} 2023-07-20T16:36:03.960Z debug get/get.go:108 operation finished with error {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"} 2023-07-20T16:36:03.960Z debug get/container.go:18 trying to execute in container... {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false, "netmap lookup depth": 0} 2023-07-20T16:36:03.960Z debug get/container.go:46 process epoch {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false, "number": 3} 2023-07-20T16:36:03.960Z debug get/remote.go:14 processing node... {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false} 2023-07-20T16:36:03.962Z debug get/container.go:87 completing the operation {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false} 2023-07-20T16:36:03.962Z debug get/get.go:103 requested object is virtual {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false} 2023-07-20T16:36:03.962Z debug get/assemble.go:34 trying to assemble the object... {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false} 2023-07-20T16:36:03.962Z debug get/assemble.go:83 starting assembling from child {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false, "child ID": "94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf"} 2023-07-20T16:36:03.962Z debug get/get.go:87 serving request... {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false} 2023-07-20T16:36:03.962Z debug get/local.go:25 local get failed {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"} 2023-07-20T16:36:03.962Z debug get/get.go:108 operation finished with error {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"} 2023-07-20T16:36:03.962Z debug get/container.go:18 trying to execute in container... {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false, "netmap lookup depth": 0} 2023-07-20T16:36:03.962Z debug get/container.go:46 process epoch {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false, "number": 3} 2023-07-20T16:36:03.962Z debug get/remote.go:14 processing node... {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false} 2023-07-20T16:36:03.964Z debug get/remote.go:34 remote call failed {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "read object header: status: code = 2048 message = access to object operation denied"} 2023-07-20T16:36:03.964Z debug get/remote.go:14 processing node... {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false} 2023-07-20T16:36:03.966Z debug get/remote.go:34 remote call failed {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "read object header: status: code = 2048 message = access to object operation denied"} 2023-07-20T16:36:03.966Z debug get/remote.go:14 processing node... {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false} 2023-07-20T16:36:03.968Z debug get/remote.go:34 remote call failed {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "read object header: status: code = 2048 message = access to object operation denied"} 2023-07-20T16:36:03.968Z debug get/container.go:63 no more nodes, abort placement iteration {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false} 2023-07-20T16:36:03.968Z debug get/get.go:108 operation finished with error {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/94U6MXjYyiQUeBbt1Lurjjqe1RRG3Ua6M64x6GzBkpnf", "raw": false, "local": false, "with session": false, "with bearer": false, "error": "status: code = 2049 message = object not found"} 2023-07-20T16:36:03.968Z debug get/assemble.go:76 could not init parent from child {"component": "Object.Get service", "request": "GET", "address": "8w2kxjgNyL9kotkebAtXpYU91gtQkF24EuthgBbYURon/9LQSGzpxXmygaUz4gYgWV3x34Uj76sSnuyqafVPa3Fgm", "raw": false, "local": false, "with session": false, "with bearer": false} ``` seems like session is lost when out-of-container node spawns new requests or the session doesn't work ### Versions * neofs-cli [v0.37.0](https://github.com/nspcc-dev/neofs-node/releases/tag/v0.37.0) * neofs-dev-end `a9b92f53f7913fc28d4eb307a671030d87a7ecab` (with NeoFS v0.37.0 too)

sami 2025-12-28 17:21:40 +00:00

• closed this issue
• added the
  bug
  neofs-storage
  S2
  U2
  I2
  labels

sami commented 2025-12-28 17:21:41 +00:00

Author

Owner

Copy link

@cthulhu-rider commented on GitHub (Jul 20, 2023):

• may be related to #2440 (maybe i did the test steps)

@cthulhu-rider commented on GitHub (Jul 20, 2023): * may be related to #2440 (maybe i did the test steps)

sami commented 2025-12-28 17:21:41 +00:00

Author

Owner

Copy link

@cthulhu-rider commented on GitHub (Jul 21, 2023):

we couldn't longer just forward GET request to container node starting from nspcc-dev/neofs-node@907f427b99

@carpawell do u remember why we prevented assembly within local requests? this limit seems pretty strange to me. TTL should control hops of the request only but not it's processing

@cthulhu-rider commented on GitHub (Jul 21, 2023): we couldn't longer just forward GET request to container node starting from https://github.com/nspcc-dev/neofs-node/commit/907f427b99768ff010834621578441ef81dc0ff9 @carpawell do u remember why we prevented assembly within local requests? this limit seems pretty strange to me. TTL should control hops of the request only but not it's processing

sami commented 2025-12-28 17:21:41 +00:00

Author

Owner

Copy link

@cthulhu-rider commented on GitHub (Jul 21, 2023):

despite the fact that the container node refuses to collect the object because of the TTL, why can't the first recipient of the request collect it?

currently it is expected because server cannot associate the requested child object with its parent referenced by the session token. Because of this, in a private container, a out-of-contianer node does not have enough rights, which can be seen from the error log. See #1420

@cthulhu-rider commented on GitHub (Jul 21, 2023): despite the fact that the container node refuses to collect the object because of the TTL, why can't the first recipient of the request collect it? currently it is expected because server cannot associate the requested child object with its parent referenced by the session token. Because of this, in a private container, a out-of-contianer node does not have enough rights, which can be seen from the error log. See #1420

sami commented 2025-12-28 17:21:41 +00:00

Author

Owner

Copy link

@roman-khimov commented on GitHub (Jan 29, 2024):

May be related to #838.

@roman-khimov commented on GitHub (Jan 29, 2024): May be related to #838.

sami commented 2025-12-28 17:21:41 +00:00

Author

Owner

Copy link

@cthulhu-rider commented on GitHub (Feb 27, 2024):

the problem is definitely caused by TTL:

• if TTL=1, object is never assembled, see https://github.com/nspcc-dev/neofs-node/issues/2447#issuecomment-1645468669
• in described scenario, original request has TTL=2, forwarded request wraps the original one with TTL=1
• server processes TTL from the latest hop, ie. 1

therefore, to fix this bug, at least one of following changes is required:

1. process TTL from the original request (2 in this scenario)
2. allow object assembly with TTL=1

to me:

• 1 is legit cuz forwarding should add some addtional meta info only, not change processing of the original request
• 2 is also makes sense, see my comment aboe. 907f427b99 doesnt describe the problem it solved

@roman-khimov @carpawell what do u think?

@cthulhu-rider commented on GitHub (Feb 27, 2024): the problem is definitely caused by TTL: * if `TTL=1`, object is never assembled, see https://github.com/nspcc-dev/neofs-node/issues/2447#issuecomment-1645468669 * in described scenario, original request has `TTL=2`, forwarded request wraps the original one with `TTL=1` * server processes TTL from the latest hop, ie. `1` therefore, to fix this bug, at least one of following changes is required: 1. process TTL from the original request (`2` in this scenario) 2. allow object assembly with `TTL=1` to me: * 1 is legit cuz forwarding should add some addtional meta info only, not change processing of the original request * 2 is also makes sense, see my comment aboe. 907f427b99768ff010834621578441ef81dc0ff9 doesnt describe the problem it solved @roman-khimov @carpawell what do u think?

sami commented 2025-12-28 17:21:41 +00:00

Author

Owner

Copy link

@carpawell commented on GitHub (Feb 27, 2024):

I think 1. is correct in general if it is forwarding without any request changes. But if we change request (even with some minimal meta info) that is not forwarding to me and should decrease TTL. The only TTL non-decreasing operation to me is forwarding that does not allow you to understand that forwarder ever existed.
2. is also correct to me but may break some cases. Mb even test cases that test some think with neofs-cli.

@carpawell commented on GitHub (Feb 27, 2024): I think 1. is correct in general if it is _forwarding_ without any request changes. But if we change request (even with some minimal meta info) that is not _forwarding_ to me and should decrease TTL. The only TTL non-decreasing operation to me is forwarding that does not allow you to understand that forwarder ever existed. 2. is also correct to me but may break some cases. Mb even test cases that test some think with `neofs-cli`.

sami commented 2025-12-28 17:21:41 +00:00

Author

Owner

Copy link

@cthulhu-rider commented on GitHub (Feb 27, 2024):

may break some cases

every change may break some cases in theory. What cases?

@cthulhu-rider commented on GitHub (Feb 27, 2024): > may break some cases every change may break some cases in theory. What cases?

sami commented 2025-12-28 17:21:41 +00:00

Author

Owner

Copy link

@roman-khimov commented on GitHub (Feb 27, 2024):

TTL is there mostly to avoid endless forwarding, to me it's mostly the problem of our approach to it as "local request". Distinguishing small/big object requests just shouldn't rely on TTL to me.

@roman-khimov commented on GitHub (Feb 27, 2024): TTL is there mostly to avoid endless forwarding, to me it's mostly the problem of our approach to it as "local request". Distinguishing small/big object requests just shouldn't rely on TTL to me.

sami commented 2025-12-28 17:21:41 +00:00

Author

Owner

Copy link

@carpawell commented on GitHub (Feb 27, 2024):

What cases?

I do remember that i have tested smth in the object assembly process with TTL 1. But we may consider this as a kludge.

@carpawell commented on GitHub (Feb 27, 2024): > What cases? I do remember that i have tested smth in the object assembly process with TTL 1. But we may consider this as a kludge.

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

master

put-ants-pool

lens-meta-resync

policer-boost-mode

fix-shard-evacuation-for-ec

fstree-head-buffer

initial-placement

test-metadata-contract

head-proxy-buffer

feat/meta-data-native-contract

local-head-buffer

head-local-bin

test/notary-lock

feat/meta-side-chain

notary-set-attribute

search-1000

ec-put-parallel

fix/incorrect-TS-placement

new-object-structure

get-forwarding

test-bolt

logs

decrease-prototick

bugfix/listen-maxobjsize-changes

2684-api-server-optimize-sig-verify

v0.51.1

v0.51.0

v0.50.2

v0.50.1

v0.50.0

v0.49.1

v0.49.0

v0.48.3

v0.48.2

v0.48.1

v0.48.0

v0.47.1

v0.47.0

v0.46.1

v0.46.0

v0.45.2

v0.45.1

v0.45.0

v0.44.2

v0.44.1

v0.44.0

v0.43.0

v0.42.1

v0.42.0

v0.41.1

v0.41.0

v0.40.1

v0.40.0

v0.39.2

v0.39.1

v0.39.0

v0.38.1

v0.38.0

v0.37.0

v0.36.1

v0.36.0

v0.35.0

v0.34.0

v0.33.0

v0.32.0

v0.31.0

v0.30.2

v0.30.1

v0.30.0

v0.29.0

v0.28.3

v0.28.2

v0.28.1

v0.28.0

v0.28.0-rc.3

v0.27.7

v0.27.6

v0.28.0-rc.2

v0.28.0-rc.1

v0.27.5

v0.27.4

v0.27.3

v0.27.2

v0.27.1

v0.27.0

v0.27.0-rc.1

v0.26.1

v0.26.0

v0.25.1

v0.25.0

v0.24.1

v0.24.0

v0.23.1

v0.23.0

v0.22.3

v0.22.2

v0.22.1

v0.22.0

v0.21.1

v0.21.0

v0.20.0

v0.19.0

v0.18.0

v0.17.0

v0.16.0

v0.15.0

v0.14.3

v0.14.2

v0.14.1

v0.14.0

v0.14.0-rc.1

v0.13.2

v0.13.1

v0.13.0

v0.13.0-rc.1

v0.12.1

v0.12.0

v0.12.0-rc3

v0.12.0-rc2

v0.12.0-rc1

v0.11.0

v0.10.0

Labels

Clear labels   

I1

  

I2

  

I3

  

I4

  

S0

  

S1

  

S2

  

S3

  

S4

  

U0

  

U1

  

U2

  

U3

  

U4

  

blocked

  

bug

  

config

  

dependencies

  

discussion

  

documentation

  

enhancement

  

enhancement

  

epic

  

feature

  

go

  

good first issue

  

help wanted

  

neofs-adm

  

neofs-cli

  

neofs-cli

  

neofs-cli

  

neofs-ir

  

neofs-lens

  

neofs-storage

  

neofs-storage

  

performance

  

question

  

security

  

task

  

test

  

windows

No labels

I1

I2

I3

I4

S0

S1

S2

S3

S4

U0

U1

U2

U3

U4

blocked

bug

config

dependencies

discussion

documentation

enhancement

enhancement

epic

feature

go

good first issue

help wanted

neofs-adm

neofs-cli

neofs-cli

neofs-cli

neofs-ir

neofs-lens

neofs-storage

neofs-storage

performance

question

security

task

test

windows

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

nspcc-dev/neofs-node#1069

No description provided.