nspcc-dev/neofs-node

Fork 0

mirror of https://github.com/nspcc-dev/neofs-node.git synced 2026-03-01 04:29:10 +00:00

`neofs-cli` default timeouts for `object put`/`object get`/etc are implicit #1043

New issue

Open

opened 2025-12-28 17:21:35 +00:00 by sami · 5 comments

sami commented

2025-12-28 17:21:35 +00:00

Owner

Originally created by @notimetoname on GitHub (Jun 19, 2023).

After https://github.com/nspcc-dev/neofs-node/pull/2135 a user can see the following (15-sec default timeout that did not exist before):

▶ neofs-cli object put -r s01.neofs.devenv:8080 -w services/chain/node-wallet.json --cid HrG4qXKc3fpvY1dBay4Jy4qtDgGbwEnDtikDUJTegZJi --file tmp/test.file 
Enter password > 
 276824064 / 10000000000 [=======>-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------]   2.77% 12s
rpc error: client failure: rpc error: code = DeadlineExceeded desc = context deadline exceeded

as a user, I would ask some questions:

"rpc error:"? Really? Network error? The node is down?
I have the progress bar, I see how it works and nothing is frozen, I control it and can send SIGINT if I want to. Why 15-sec timeout is forced? "Any object should be put in 15 seconds" is smth unnatural for a util (at least by default when a progress bar is active).
Why doesn't it work the same as wget/curl?

cc @532910, @MaxGelbakhiani, @aprasolova, @vvarg229

Originally created by @notimetoname on GitHub (Jun 19, 2023). After https://github.com/nspcc-dev/neofs-node/pull/2135 a user can see the following (15-sec default timeout that did not exist before): ```bash ▶ neofs-cli object put -r s01.neofs.devenv:8080 -w services/chain/node-wallet.json --cid HrG4qXKc3fpvY1dBay4Jy4qtDgGbwEnDtikDUJTegZJi --file tmp/test.file Enter password > 276824064 / 10000000000 [=======>-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 2.77% 12s rpc error: client failure: rpc error: code = DeadlineExceeded desc = context deadline exceeded ``` as a user, I would ask some questions: 1. "rpc error:"? Really? Network error? The node is down? 2. I have the progress bar, I see how it works and nothing is frozen, I control it and can send SIGINT if I want to. Why 15-sec timeout is forced? "Any object should be put in 15 seconds" is smth unnatural for a util (at least by default when a progress bar is active). 3. Why doesn't it work the same as `wget`/`curl`? cc @532910, @MaxGelbakhiani, @aprasolova, @vvarg229

sami added the

neofs-cli

discussion

help wanted

labels

2025-12-28 17:21:35 +00:00

sami commented

2025-12-28 17:21:35 +00:00

Author

Owner

@roman-khimov commented on GitHub (Jun 19, 2023):

wget:

       -T seconds
       --timeout=seconds
           Set the network timeout to seconds seconds.  This is equivalent to specifying --dns-timeout, --connect-timeout, and --read-timeout, all at the same time.

           When interacting with the network, Wget can check for timeout and abort the operation if it takes too long.  This prevents anomalies like hanging reads and infinite connects.  The only timeout enabled by default is a 900-second read timeout.  Setting a timeout to 0 disables it altogether.  Unless you know what you are doing, it is best not to change the
           default timeout settings.

           All timeout-related options accept decimal values, as well as subsecond values.  For example, 0.1 seconds is a legal (though unwise) choice of timeout.  Subsecond timeouts are useful for checking server response times or for testing network latency.

       --dns-timeout=seconds
           Set the DNS lookup timeout to seconds seconds.  DNS lookups that don't complete within the specified time will fail.  By default, there is no timeout on DNS lookups, other than that implemented by system libraries.

       --connect-timeout=seconds
           Set the connect timeout to seconds seconds.  TCP connections that take longer to establish will be aborted.  By default, there is no connect timeout, other than that implemented by system libraries.

       --read-timeout=seconds
           Set the read (and write) timeout to seconds seconds.  The "time" of this timeout refers to idle time: if, at any point in the download, no data is received for more than the specified number of seconds, reading fails and the download is restarted.  This option does not directly affect the duration of the entire download.

           Of course, the remote server may choose to terminate the connection sooner than this option requires.  The default read timeout is 900 seconds.

curl:

       --connect-timeout <fractional seconds>
              Maximum time in seconds that you allow curl's connection to take.  This only limits the connection phase, so if curl connects within the given period it will continue - if not it will exit.  Since version 7.32.0, this option accepts decimal values.

              The "connection phase" is considered complete when the requested TCP, TLS or QUIC handshakes are done.

              The decimal value needs to provided using a dot (.) as decimal separator - not the local version even if it might be using another separator.

              If --connect-timeout is provided several times, the last set value will be used.

              Examples:
               curl --connect-timeout 20 https://example.com
               curl --connect-timeout 3.14 https://example.com

              See also -m, --max-time.
...
       -m, --max-time <fractional seconds>
              Maximum time in seconds that you allow each transfer to take.  This is useful for preventing your batch jobs from hanging for hours due to slow networks or links going down.  Since 7.32.0, this option accepts decimal values, but the actual timeout will decrease in accuracy as the specified timeout increases in decimal precision.

              If you enable retrying the transfer (--retry) then the maximum time counter is reset each time the transfer is retried. You can use --retry-max-time to limit the retry time.

              The decimal value needs to provided using a dot (.) as decimal separator - not the local version even if it might be using another separator.

              If -m, --max-time is provided several times, the last set value will be used.

              Examples:
               curl --max-time 10 https://example.com
               curl --max-time 2.92 https://example.com

              See also --connect-timeout and --retry-max-time.
...
       --retry-max-time <seconds>
              The  retry  timer is reset before the first transfer attempt. Retries will be done as usual (see --retry) as long as the timer has not reached this given limit. Notice that if the timer has not reached the limit, the request will be made and while performing, it may take longer than this given time period. To limit a single request's maximum time, use -m,
              --max-time. Set this option to zero to not timeout retries.

              If --retry-max-time is provided several times, the last set value will be used.

              Example:
               curl --retry-max-time 30 --retry 10 https://example.com

              See also --retry.
...
       -y, --speed-time <seconds>
              If a transfer runs slower than speed-limit bytes per second during a speed-time period, the transfer is aborted. If speed-time is used, the default speed-limit will be 1 unless set with -Y, --speed-limit.

              This option controls transfers (in both directions) but will not affect slow connects etc. If this is a concern for you, try the --connect-timeout option.

              If -y, --speed-time is provided several times, the last set value will be used.

              Example:
               curl --speed-limit 300 --speed-time 10 https://example.com

              See also -Y, --speed-limit and --limit-rate.

@roman-khimov commented on GitHub (Jun 19, 2023): wget: ``` -T seconds --timeout=seconds Set the network timeout to seconds seconds. This is equivalent to specifying --dns-timeout, --connect-timeout, and --read-timeout, all at the same time. When interacting with the network, Wget can check for timeout and abort the operation if it takes too long. This prevents anomalies like hanging reads and infinite connects. The only timeout enabled by default is a 900-second read timeout. Setting a timeout to 0 disables it altogether. Unless you know what you are doing, it is best not to change the default timeout settings. All timeout-related options accept decimal values, as well as subsecond values. For example, 0.1 seconds is a legal (though unwise) choice of timeout. Subsecond timeouts are useful for checking server response times or for testing network latency. --dns-timeout=seconds Set the DNS lookup timeout to seconds seconds. DNS lookups that don't complete within the specified time will fail. By default, there is no timeout on DNS lookups, other than that implemented by system libraries. --connect-timeout=seconds Set the connect timeout to seconds seconds. TCP connections that take longer to establish will be aborted. By default, there is no connect timeout, other than that implemented by system libraries. --read-timeout=seconds Set the read (and write) timeout to seconds seconds. The "time" of this timeout refers to idle time: if, at any point in the download, no data is received for more than the specified number of seconds, reading fails and the download is restarted. This option does not directly affect the duration of the entire download. Of course, the remote server may choose to terminate the connection sooner than this option requires. The default read timeout is 900 seconds. ``` curl: ``` --connect-timeout <fractional seconds> Maximum time in seconds that you allow curl's connection to take. This only limits the connection phase, so if curl connects within the given period it will continue - if not it will exit. Since version 7.32.0, this option accepts decimal values. The "connection phase" is considered complete when the requested TCP, TLS or QUIC handshakes are done. The decimal value needs to provided using a dot (.) as decimal separator - not the local version even if it might be using another separator. If --connect-timeout is provided several times, the last set value will be used. Examples: curl --connect-timeout 20 https://example.com curl --connect-timeout 3.14 https://example.com See also -m, --max-time. ... -m, --max-time <fractional seconds> Maximum time in seconds that you allow each transfer to take. This is useful for preventing your batch jobs from hanging for hours due to slow networks or links going down. Since 7.32.0, this option accepts decimal values, but the actual timeout will decrease in accuracy as the specified timeout increases in decimal precision. If you enable retrying the transfer (--retry) then the maximum time counter is reset each time the transfer is retried. You can use --retry-max-time to limit the retry time. The decimal value needs to provided using a dot (.) as decimal separator - not the local version even if it might be using another separator. If -m, --max-time is provided several times, the last set value will be used. Examples: curl --max-time 10 https://example.com curl --max-time 2.92 https://example.com See also --connect-timeout and --retry-max-time. ... --retry-max-time <seconds> The retry timer is reset before the first transfer attempt. Retries will be done as usual (see --retry) as long as the timer has not reached this given limit. Notice that if the timer has not reached the limit, the request will be made and while performing, it may take longer than this given time period. To limit a single request's maximum time, use -m, --max-time. Set this option to zero to not timeout retries. If --retry-max-time is provided several times, the last set value will be used. Example: curl --retry-max-time 30 --retry 10 https://example.com See also --retry. ... -y, --speed-time <seconds> If a transfer runs slower than speed-limit bytes per second during a speed-time period, the transfer is aborted. If speed-time is used, the default speed-limit will be 1 unless set with -Y, --speed-limit. This option controls transfers (in both directions) but will not affect slow connects etc. If this is a concern for you, try the --connect-timeout option. If -y, --speed-time is provided several times, the last set value will be used. Example: curl --speed-limit 300 --speed-time 10 https://example.com See also -Y, --speed-limit and --limit-rate. ```

sami commented

2025-12-28 17:21:35 +00:00

Author

Owner

@notimetoname commented on GitHub (Jun 19, 2023):

@roman-khimov, yeah, I read man before the issue was posted. wget uses a 60(!) times longer default. curl seems not to use a default timeout at all. My question (and the issue) is about the defaults and a message we print when the --timeout is fired.

@notimetoname commented on GitHub (Jun 19, 2023): @roman-khimov, yeah, I read `man` before the issue was posted. `wget` uses a 60(!) times longer default. `curl` seems not to use a default timeout at all. My question (and the issue) is about the defaults and a message we print when the `--timeout` is fired.

sami commented

2025-12-28 17:21:35 +00:00

Author

Owner

@notimetoname commented on GitHub (Jun 19, 2023):

a message we print when the --timeout is fired

and a separate exit code for it?

@notimetoname commented on GitHub (Jun 19, 2023): > a message we print when the --timeout is fired and a separate exit code for it?

sami commented

2025-12-28 17:21:35 +00:00

Author

Owner

@roman-khimov commented on GitHub (Jun 19, 2023):

These are just examples, my point is mostly that timeout handling is entirely app-specific, because wgettish timeout is an "idle timeout" (somewhat like our timeout was before #2135), while curly timeout (while not having a default) behaves like #2135.

While static "max execution time" makes sense for container operations, data can be 1MB, 1GB or 1TB and we should be able to handle any of these cases (over any links). So we can turn --timeout off completely for object put/get (but leave it in options). IIUC there are some heartbeats in grpc to react on connection breakage in this case and still fail download/upload if we can no longer transmit data. If that's not sufficient, then an additional "idle timeout" parameter with some sane (30/60s) default value can be used as well.

@roman-khimov commented on GitHub (Jun 19, 2023): These are just examples, my point is mostly that timeout handling is entirely app-specific, because wgettish timeout is an "idle timeout" (somewhat like our timeout was before #2135), while curly timeout (while not having a default) behaves like #2135. While static "max execution time" makes sense for container operations, data can be 1MB, 1GB or 1TB and we should be able to handle any of these cases (over any links). So we can turn `--timeout` off completely for object put/get (but leave it in options). IIUC there are some heartbeats in grpc to react on connection breakage in this case and still fail download/upload if we can no longer transmit data. If that's not sufficient, then an additional "idle timeout" parameter with some sane (30/60s) default value can be used as well.

sami commented

2025-12-28 17:21:35 +00:00

Author

Owner

@notimetoname commented on GitHub (Jun 19, 2023):

So we can turn --timeout off completely for object put/get (but leave it in options).

That is how I would go OR at least do not make it that short by default (we can turn the progress bar off with a separate flag).

@notimetoname commented on GitHub (Jun 19, 2023): > So we can turn --timeout off completely for object put/get (but leave it in options). That is how I would go OR at least do not make it that short by default (we can turn the progress bar off with a separate flag).