shards evacuate fails with "could not get list of objects: database not open" #793

Closed
opened 2025-12-28 17:20:43 +00:00 by sami · 1 comment
Owner

Originally created by @vkarak1 on GitHub (Oct 12, 2022).

Originally assigned to: @fyrchik on GitHub.

I have been working on testing the following issue, while the problem has been occurred.
Running the "control shards evacuate" command leads to unexpected error:

root@az:/etc/neofs/storage# neofs-cli --config /etc/neofs/storage/config.yml --wallet /etc/neofs/storage/wallet.json --endpoint localhost:8091 control shards evacuate --id 74ckC8iEk4AqhvBrWwPqrr
Enter password >
rpc error: rpc error: code = Internal desc = could not get list of objects: database not open

Expected Behavior

The command finished with amount of moved objects, like an example below:

Objects moved: 1
Shard has successfully been evacuated.

Current Behavior

CLI command fails with the following error description:
rpc error: rpc error: code = Internal desc = could not get list of objects: database not open

Steps to Reproduce (for bugs)

Please find the list of the commands which were completed successfully before I have faced the problem:

  1. neofs-cli --endpoint localhost:8091 -w /etc/neofs/storage/wallet.json control shards list
  2. neofs-cli netmap snapshot -g -r node1.neofs:8080
  3. neofs-cli control set-status --status maintenance --endpoint localhost:8091 -w wallet.json
  4. neofs-cli control shards set-mode --mode read-only --endpoint localhost:8091 -w wallet.json --id F2JTp2PM1dMAsUBHuodYX7

Logs:
evacuate_bug.zip

Versions:

NeoFS CLI
Version: v0.32.0-34-g76cfcc24
GoVersion: go1.18.4

NeoGo
Version: 0.99.3-2-gaa57779d

NeoFS Storage node
Version: v0.32.0-34-g76cfcc24

Your Environment

Server setup and configuration:
cloud, 4 VMs, 4 SN, 4 http qw, 4 s3 gw

Operating System and version (uname -a):
Linux az 5.10.0-18-amd64 #1 SMP Debian 5.10.140-1 (2022-09-02) x86_64 GNU/Linux

Originally created by @vkarak1 on GitHub (Oct 12, 2022). Originally assigned to: @fyrchik on GitHub. <!-- Provide a general summary of the issue in the Title above --> I have been working on testing the following [issue](https://j.yadro.com/browse/OBJECT-1261), while the problem has been occurred. Running the "control shards evacuate" command leads to unexpected error: ``` root@az:/etc/neofs/storage# neofs-cli --config /etc/neofs/storage/config.yml --wallet /etc/neofs/storage/wallet.json --endpoint localhost:8091 control shards evacuate --id 74ckC8iEk4AqhvBrWwPqrr Enter password > rpc error: rpc error: code = Internal desc = could not get list of objects: database not open ``` ## Expected Behavior <!-- If you're describing a bug, tell us what should happen If you're suggesting a change/improvement, tell us how it should work --> The command finished with amount of moved objects, like an example below: ``` Objects moved: 1 Shard has successfully been evacuated. ``` ## Current Behavior <!-- If describing a bug, tell us what happens instead of the expected behavior If suggesting a change/improvement, explain the difference from current behavior --> CLI command fails with the following error description: `rpc error: rpc error: code = Internal desc = could not get list of objects: database not open` ## Steps to Reproduce (for bugs) <!-- Provide a link to a live example, or an unambiguous set of steps to reproduce this bug. --> Please find the list of the commands which were completed successfully before I have faced the problem: 1. neofs-cli --endpoint localhost:8091 -w /etc/neofs/storage/wallet.json control shards list 2. neofs-cli netmap snapshot -g -r node1.neofs:8080 3. neofs-cli control set-status --status maintenance --endpoint localhost:8091 -w wallet.json 4. neofs-cli control shards set-mode --mode read-only --endpoint localhost:8091 -w wallet.json --id F2JTp2PM1dMAsUBHuodYX7 Logs: [evacuate_bug.zip](https://github.com/nspcc-dev/neofs-node/files/9765954/evacuate_bug.zip) **Versions:** ``` NeoFS CLI Version: v0.32.0-34-g76cfcc24 GoVersion: go1.18.4 NeoGo Version: 0.99.3-2-gaa57779d NeoFS Storage node Version: v0.32.0-34-g76cfcc24 ``` ## Your Environment <!-- Include as many relevant details about the environment you experienced the bug in --> Server setup and configuration: cloud, 4 VMs, 4 SN, 4 http qw, 4 s3 gw Operating System and version (uname -a): Linux az 5.10.0-18-amd64 #1 SMP Debian 5.10.140-1 (2022-09-02) x86_64 GNU/Linux
sami 2025-12-28 17:20:43 +00:00
  • closed this issue
  • added the
    bug
    U2
    labels
Author
Owner

@fyrchik commented on GitHub (Nov 5, 2022):

The logs are pretty small here, I guess shard went to degraded because of some errors during startup.
This should be fixed by #1982

@fyrchik commented on GitHub (Nov 5, 2022): The logs are pretty small here, I guess shard went to degraded because of some errors during startup. This should be fixed by #1982
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
nspcc-dev/neofs-node#793
No description provided.