rpc error: context deadline exceeded if timeout is default (15s) #813

Open
opened 2025-12-28 17:20:46 +00:00 by sami · 1 comment
Owner

Originally created by @ikovalev1 on GitHub (Oct 18, 2022).

Expected Behavior

"neofs-cli object put" shouldn't fail if one of the nodes is not responding while there are others helthy nodes to store the object.

Current Behavior

neofs-cli object put
rpc error: client failure: context deadline exceeded

Possible Solution

increase default timeot to 30s

Steps to Reproduce (for bugs)

  1. create container with policy REP 3 SELECT 4 so that if one of the nodes hangs, there are 3 nodes to comply with the REP 3 policy
  2. node1: suspend neofs-node: kill -STOP pidof neofs-node
  3. node2: try to put objects to the contaner using default timeout : error
  4. node2: try to put objects to the contaner using timeout=30s: ok

Your Environment

root@az:/home/service# neofs-node -version
NeoFS Storage node
Version: v0.32.0-125-gbcf3df35
GoVersion: go1.18.4

Originally created by @ikovalev1 on GitHub (Oct 18, 2022). ## Expected Behavior "neofs-cli object put" shouldn't fail if one of the nodes is not responding while there are others helthy nodes to store the object. ## Current Behavior neofs-cli object put rpc error: client failure: context deadline exceeded ## Possible Solution increase default timeot to 30s ## Steps to Reproduce (for bugs) 1. create container with policy REP 3 SELECT 4 so that if one of the nodes hangs, there are 3 nodes to comply with the REP 3 policy 2. node1: suspend neofs-node: kill -STOP `pidof neofs-node` 3. node2: try to put objects to the contaner using default timeout : error 4. node2: try to put objects to the contaner using timeout=30s: ok ## Your Environment root@az:/home/service# neofs-node -version NeoFS Storage node Version: v0.32.0-125-gbcf3df35 GoVersion: go1.18.4
Author
Owner

@fyrchik commented on GitHub (Oct 18, 2022):

We could increase the timeout and have this fixed up until the next similar error occurs. And we cannot pick a single default, because it (maximum possible timeout) depends on the timeouts in the configuration of all storage nodes participating in the operation. So 30 can fix this case, but certainly there are others.

I think we can ensure that timeout can be picked from the configuration (I believe it already should). This way any particular deployment can override this default. 15 seconds looks like a reasonable default to have for public deployments.

@cthulhu-rider @realloc @carpawell

@fyrchik commented on GitHub (Oct 18, 2022): We could increase the timeout and have this fixed up until the next similar error occurs. And we cannot pick a single default, because it (`maximum possible timeout`) depends on the timeouts in the configuration of all storage nodes participating in the operation. So 30 can fix this case, but certainly there are others. I think we can ensure that `timeout` can be picked from the configuration (I believe it already should). This way any particular deployment can override this default. 15 seconds looks like a reasonable default to have for public deployments. @cthulhu-rider @realloc @carpawell
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
nspcc-dev/neofs-node#813
No description provided.