nspcc-dev/neofs-rest-gw

Fork 0

mirror of https://github.com/nspcc-dev/neofs-rest-gw.git synced 2026-03-01 04:29:14 +00:00

Gateway can deny bearer token accepted by NeoFS itself #78

New issue

Open

opened 2025-12-28 18:00:04 +00:00 by sami · 2 comments

sami commented

2025-12-28 18:00:04 +00:00

Owner

Originally created by @cthulhu-rider on GitHub (Apr 1, 2024).

NeoFS bearer token was recently extended with new field (https://github.com/nspcc-dev/neofs-api/issues/266). Now it has one more field that could be signed and transferred. I tried to use the latest token version w/o REST gateway upgrade (being done in https://github.com/nspcc-dev/neofs-rest-gw/pull/176) and see how it goes

Current Behavior

REST denies bearer token:

{"message":"get bearer token: invalid signature","type":"GW"}

while NeoFS accepts it.

Expected Behavior

REST accepts valid bearer token and op proceeds in the same way as when contacting directly to NeoFS

Possible Solution

this is a function that handles passed bearer token. As we can see, it:

decodes token structure defined in SDK
verifies the signature again via SDK

currently, signature mismatch is expected cuz:

SDK unmarshaler "loses" all token fields that havent been added yet (issuer in this case)
verification checks only decoded fields, i.e. known ones

next can be done to solve the problem:

dont encode bearer token, just forward received token binary to the NeoFS. This is worth tbd regardless of token verification to reduce resource consumption
support unknown fields in signature verification. If the fields are encoded correctly, the gate can order them and this will allow it to check the signature even without interpretation (which is obviously impossible cuz the fields are unknown)

Describe alternatives you've considered

do nothing: keep gateways as much fresh as possible, indirectly this will hide the problem
do not validate tokens by gateway at all. They are checked by NeoFS anyway. For "good" tokens this will increase system performance (no duplicated actions). At the same time, "bad" tokens will reach NeoFS more often in general. This could also be a gateway option in config

Steps to Reproduce

run NeoFS Dev Env with REST gateway (default)
create public container with extendable access rules. Policy can be any (valid) for this scenario

$ neofs-cli -r s01.neofs.devenv:8080 -w ../devenv/services/chain/node-wallet.json container create --policy 'REP 2 CBF 1' --basic-acl eacl-public-read-write --await
Enter password > 
container creation request accepted for processing (the operation may not be completed yet)
container ID: wLb8QEdU2oi8CE3aBCYNPpiS6fgLr7JP1ssgwAZwQRQ
awaiting...
container has been persisted on sidechain

add access rule to deny write ops for OTHERS

$ neofs-cli acl extended create --cid wLb8QEdU2oi8CE3aBCYNPpiS6fgLr7JP1ssgwAZwQRQ --out eacl.json --rule 'deny put others'
$ neofs-cli -r s01.neofs.devenv:8080 -w ../devenv/services/chain/node-wallet.json container set-eacl --cid wLb8QEdU2oi8CE3aBCYNPpiS6fgLr7JP1ssgwAZwQRQ --table eacl.json --await
Enter password > 
Checking the ability to modify access rights in the container...
ACL extension is enabled in the container, continue processing.
eACL modification request accepted for processing (the operation may not be completed yet)
awaiting...
EACL has been persisted on sidechain

assert rule action with REST account which is "other"

$ neofs-cli -r s01.neofs.devenv:8080 -w ../devenv/services/rest_gate/wallet.json object put --cid wLb8QEdU2oi8CE3aBCYNPpiS6fgLr7JP1ssgwAZwQRQ  --file Makefile
Enter password > 
 4218 / 4218 [============================================================================================================================================================================================================================================] 100.00% 0s
rpc error: finish object stream: status: code = 2048 message = access to object operation denied: access to operation OBJECT_PUT is denied by extended ACL check: denied by rule

issue bearer token granting the access to the REST account on behalf of the container creator

$ neofs-cli acl extended create --cid wLb8QEdU2oi8CE3aBCYNPpiS6fgLr7JP1ssgwAZwQRQ --rule 'allow put others' --out bearer_eacl.json
$ neofs-cli -r s01.neofs.devenv:8080 bearer create -i 0 -n 0 -x 102400 -e bearer_eacl.json -o NbUgTSFvPmsRxmGeWpuuGeJUoRoi6PErcM --out bearer_blank.json
$ neofs-cli util sign bearer-token --from bearer_blank.json --to bearer_signed -w ../devenv/services/chain/node-wallet.json 
Enter password > 
signed bearer token was successfully dumped to bearer.json

try the token through NeoFS CLI

$ neofs-cli -r s01.neofs.devenv:8080 -w ../devenv/services/rest_gate/wallet.json object put --cid wLb8QEdU2oi8CE3aBCYNPpiS6fgLr7JP1ssgwAZwQRQ  --file Makefile --bearer bearer_signed
Enter password > 
 4218 / 4218 [============================================================================================================================================================================================================================================] 100.00% 0s
[Makefile] Object successfully stored
  OID: ECqtkxzY6Kq8dvpanoaWgQebqS6odCwcZvqm22WLfL7b
  CID: wLb8QEdU2oi8CE3aBCYNPpiS6fgLr7JP1ssgwAZwQRQ

try the token through REST gateway

$ curl http://rest.neofs.devenv:8090/v1/upload/wLb8QEdU2oi8CE3aBCYNPpiS6fgLr7JP1ssgwAZwQRQ \
-F 'file=@Makefile;filename=Makefile' \
-b 'Bearer=CnYKNAoECAIQDRIiCiAN64Mq1FT0ytPc3Sz+e/uFfSVCFuR8bUgHERnb053N0xoICAMQASICCAMSGwoZNSSLrQB6gIku+V7oMAHXinR47Juhw3qlfhoECICgBiIbChk17p6iLCfjS9AUj8QQjgj3To9QSLLxlcpzEmcKIQKzYiv0AXvf4xfFiu1fTHU/IGt9uJYEb6fXdLvEv3+NwhJA50e/wldmNiqr2t9KSk4qbSdIFRKoTAOpljz/oeQVhYl+CHsS2A7ZNrLxlnhZCDZqty4nfjEZ91XdCiHt+6+XUBgB'
{"message":"get bearer token: invalid signature","type":"GW"}

Context

surfaced with https://github.com/nspcc-dev/neofs-node/pull/2787
problem dissapears (hides actually) if REST pulls SDK changes (https://github.com/nspcc-dev/neofs-rest-gw/pull/176)

btw storage nodes are not forward compatible too. We can do same steps with upgraded REST and "old" nodes. Then output is:

{"code":1024,"message":"writer close: status: code = 1024 message = wrong signature","type":"API"}

but it consistent with NeoFS (CLI) itself:

rpc error: finish object stream: status: code = 1024 message = invalid signature

same problem may be encountered by any app which inherit NeoFS binary models

Regression

no: this worked (and still), but iff SDK revisions are synced

Your Environment

neofs-dev-env@155e022bd96cdde14b4fdf4a6d2c125147c9bcb8 with modified versions in .env file
neofs-node and neofs-cli from https://github.com/nspcc-dev/neofs-node/pull/2787. If merged, latest can be used
neofs-rest-gw@v0.8.3

Originally created by @cthulhu-rider on GitHub (Apr 1, 2024). NeoFS bearer token was recently extended with new field (https://github.com/nspcc-dev/neofs-api/issues/266). Now it has one more field that could be signed and transferred. I tried to use the latest token version w/o REST gateway upgrade (being done in https://github.com/nspcc-dev/neofs-rest-gw/pull/176) and see how it goes ## Current Behavior REST denies bearer token: ``` {"message":"get bearer token: invalid signature","type":"GW"} ``` while NeoFS accepts it. ## Expected Behavior REST accepts valid bearer token and op proceeds in the same way as when contacting directly to NeoFS ## Possible Solution this is a [function](https://github.com/nspcc-dev/neofs-rest-gw/blob/87bcb0271e4162dcee930b6cde1ac3ed95e22f1d/handlers/objects.go#L748) that handles passed bearer token. As we can see, it: 1. decodes token structure defined in SDK 2. verifies the signature again via SDK currently, signature mismatch is expected cuz: 1. SDK unmarshaler "loses" all token fields that havent been added yet ([issuer](https://github.com/nspcc-dev/neofs-api/blob/5c3535423c564fe63991812cb84d8334db1c553a/acl/types.proto#L240-L242) in this case) 2. verification checks only decoded fields, i.e. known ones next can be done to solve the problem: 1. dont encode bearer token, just forward received token binary to the NeoFS. This is worth tbd regardless of token verification to reduce resource consumption 2. support unknown fields in signature verification. If the fields are encoded correctly, the gate can order them and this will allow it to check the signature even without interpretation (which is obviously impossible cuz the fields are unknown) ## Describe alternatives you've considered 1. do nothing: keep gateways as much fresh as possible, indirectly this will hide the problem 2. do not validate tokens by gateway at all. They are checked by NeoFS anyway. For "good" tokens this will increase system performance (no duplicated actions). At the same time, "bad" tokens will reach NeoFS more often in general. This could also be a gateway option in config ## Steps to Reproduce 1. run NeoFS Dev Env with REST gateway (default) 3. create public container with extendable access rules. Policy can be any (valid) for this scenario ``` $ neofs-cli -r s01.neofs.devenv:8080 -w ../devenv/services/chain/node-wallet.json container create --policy 'REP 2 CBF 1' --basic-acl eacl-public-read-write --await Enter password > container creation request accepted for processing (the operation may not be completed yet) container ID: wLb8QEdU2oi8CE3aBCYNPpiS6fgLr7JP1ssgwAZwQRQ awaiting... container has been persisted on sidechain ``` 4. add access rule to deny write ops for OTHERS ``` $ neofs-cli acl extended create --cid wLb8QEdU2oi8CE3aBCYNPpiS6fgLr7JP1ssgwAZwQRQ --out eacl.json --rule 'deny put others' $ neofs-cli -r s01.neofs.devenv:8080 -w ../devenv/services/chain/node-wallet.json container set-eacl --cid wLb8QEdU2oi8CE3aBCYNPpiS6fgLr7JP1ssgwAZwQRQ --table eacl.json --await Enter password > Checking the ability to modify access rights in the container... ACL extension is enabled in the container, continue processing. eACL modification request accepted for processing (the operation may not be completed yet) awaiting... EACL has been persisted on sidechain ``` 5. assert rule action with REST account which is "other" ``` $ neofs-cli -r s01.neofs.devenv:8080 -w ../devenv/services/rest_gate/wallet.json object put --cid wLb8QEdU2oi8CE3aBCYNPpiS6fgLr7JP1ssgwAZwQRQ --file Makefile Enter password > 4218 / 4218 [============================================================================================================================================================================================================================================] 100.00% 0s rpc error: finish object stream: status: code = 2048 message = access to object operation denied: access to operation OBJECT_PUT is denied by extended ACL check: denied by rule ``` 6. issue bearer token granting the access to the REST account on behalf of the container creator ``` $ neofs-cli acl extended create --cid wLb8QEdU2oi8CE3aBCYNPpiS6fgLr7JP1ssgwAZwQRQ --rule 'allow put others' --out bearer_eacl.json $ neofs-cli -r s01.neofs.devenv:8080 bearer create -i 0 -n 0 -x 102400 -e bearer_eacl.json -o NbUgTSFvPmsRxmGeWpuuGeJUoRoi6PErcM --out bearer_blank.json $ neofs-cli util sign bearer-token --from bearer_blank.json --to bearer_signed -w ../devenv/services/chain/node-wallet.json Enter password > signed bearer token was successfully dumped to bearer.json ``` 7. try the token through NeoFS CLI ``` $ neofs-cli -r s01.neofs.devenv:8080 -w ../devenv/services/rest_gate/wallet.json object put --cid wLb8QEdU2oi8CE3aBCYNPpiS6fgLr7JP1ssgwAZwQRQ --file Makefile --bearer bearer_signed Enter password > 4218 / 4218 [============================================================================================================================================================================================================================================] 100.00% 0s [Makefile] Object successfully stored OID: ECqtkxzY6Kq8dvpanoaWgQebqS6odCwcZvqm22WLfL7b CID: wLb8QEdU2oi8CE3aBCYNPpiS6fgLr7JP1ssgwAZwQRQ ``` 8. try the token through REST gateway ``` $ curl http://rest.neofs.devenv:8090/v1/upload/wLb8QEdU2oi8CE3aBCYNPpiS6fgLr7JP1ssgwAZwQRQ \ -F 'file=@Makefile;filename=Makefile' \ -b 'Bearer=CnYKNAoECAIQDRIiCiAN64Mq1FT0ytPc3Sz+e/uFfSVCFuR8bUgHERnb053N0xoICAMQASICCAMSGwoZNSSLrQB6gIku+V7oMAHXinR47Juhw3qlfhoECICgBiIbChk17p6iLCfjS9AUj8QQjgj3To9QSLLxlcpzEmcKIQKzYiv0AXvf4xfFiu1fTHU/IGt9uJYEb6fXdLvEv3+NwhJA50e/wldmNiqr2t9KSk4qbSdIFRKoTAOpljz/oeQVhYl+CHsS2A7ZNrLxlnhZCDZqty4nfjEZ91XdCiHt+6+XUBgB' {"message":"get bearer token: invalid signature","type":"GW"} ``` ## Context * surfaced with https://github.com/nspcc-dev/neofs-node/pull/2787 * problem dissapears (hides actually) if REST pulls SDK changes (https://github.com/nspcc-dev/neofs-rest-gw/pull/176) btw storage nodes are not forward compatible too. We can do same steps with upgraded REST and "old" nodes. Then output is: ``` {"code":1024,"message":"writer close: status: code = 1024 message = wrong signature","type":"API"} ``` but it consistent with NeoFS (CLI) itself: ``` rpc error: finish object stream: status: code = 1024 message = invalid signature ``` same problem may be encountered by any app which inherit NeoFS binary models ## Regression no: this worked (and still), but iff SDK revisions are synced ## Your Environment * [`neofs-dev-env@155e022bd96cdde14b4fdf4a6d2c125147c9bcb8`](https://github.com/nspcc-dev/neofs-dev-env/commit/155e022bd96cdde14b4fdf4a6d2c125147c9bcb8) with modified versions in `.env` file * `neofs-node` and `neofs-cli` from https://github.com/nspcc-dev/neofs-node/pull/2787. If merged, latest can be used * [`neofs-rest-gw@v0.8.3`](https://github.com/nspcc-dev/neofs-rest-gw/releases/tag/v0.8.3)

sami added the

bug

blocked

labels

2025-12-28 18:00:04 +00:00

sami commented

2025-12-28 18:00:05 +00:00

Author

Owner

@roman-khimov commented on GitHub (Apr 1, 2024):

The desired behavior is likely:

for ordinary requests pass token as is, don't mess with it on the gateway
have additional API for token checks at the gateway

This is strongly related to https://github.com/nspcc-dev/neofs-api/issues/241, I'd like to have a token that can be created on the user side without reliance on the REST gateway (which is technically possible, but not trivial at the moment, so REST users like Panel never do this). This would mean that REST should be more transparent, but it can provide some additional API specifically to check tokens as they're known to the gateway.

@roman-khimov commented on GitHub (Apr 1, 2024): The desired behavior is likely: * for ordinary requests pass token as is, don't mess with it on the gateway * have additional API for token checks at the gateway This is strongly related to https://github.com/nspcc-dev/neofs-api/issues/241, I'd like to have a token that can be created on the user side _without_ reliance on the REST gateway (which is technically possible, but not trivial at the moment, so REST users like Panel never do this). This would mean that REST should be more transparent, but it can provide some additional API specifically to check tokens as they're known to the gateway.

sami commented

2025-12-28 18:00:05 +00:00

Author

Owner

@smallhive commented on GitHub (Apr 15, 2024):

There are some hidden things. The problem is deeper than it looks on the first sight. In fact the problem could be mirrored to node, if the gate would be updated first for the new version of SDK. In this case the gate approves the token, but node will say "invalid signature" error.

In the node we have the same place, where we have to unmarshal token to struct from binary or json representation. It is required to be able to pass the token to commands via parameters.

We can try to make this thing more transparent for the gate in case the gate will not interfere inside token. But this require new command API like WithBearerTokenBinary(token []byte) instead of WithBearerToken(t bearer.Token).

@smallhive commented on GitHub (Apr 15, 2024): There are some hidden things. The problem is deeper than it looks on the first sight. In fact the problem could be mirrored to node, if the gate would be updated first for the new version of SDK. In this case the gate approves the token, but node will say "invalid signature" error. In the [node](https://github.com/nspcc-dev/neofs-node/blob/d58dc794a1e3d211f67b458a3f65aaf9fc026afe/cmd/neofs-cli/internal/common/token.go#L26) we have the same place, where we have to `unmarshal` token to struct from binary or json representation. It is required to be able to pass the token to commands via parameters. We can try to make this thing more transparent for the gate in case the gate will not interfere inside token. But this require new command API like `WithBearerTokenBinary(token []byte)` instead of `WithBearerToken(t bearer.Token)`.