Panic on channel close in RPC waiter #1585

New issue

Open

opened 2025-12-28 17:16:55 +00:00 by sami · 0 comments

sami commented

2025-12-28 17:16:55 +00:00

Owner

Originally created by @roman-khimov on GitHub (Dec 19, 2025).

Current Behavior

2025-12-19T12:45:59.269Z	info	client/notifications.go:230	RPC connection lost, attempting reconnect
2025/12/19 12:45:59.273577 [ants]: worker exits from panic: close of closed channel
goroutine 45049 [running]:
runtime/debug.Stack()
	runtime/debug/stack.go:26 +0x64
github.com/panjf2000/ants/v2.(*goWorker).run.func1.1()
	github.com/panjf2000/ants/v2@v2.11.3/worker.go:62 +0x120
panic({0x10590fa60?, 0x105acf4b0?})
	runtime/panic.go:787 +0x124
github.com/nspcc-dev/neo-go/pkg/rpcclient/waiter.(*EventBased).WaitAny(0x14000370d20, {0x105ae5c88, 0x106517820}, 0x2a6a, {0x140017ee620, 0x1, 0x1})
	github.com/nspcc-dev/neo-go@v0.114.0/pkg/rpcclient/waiter/waiter.go:398 +0x894
github.com/nspcc-dev/neo-go/pkg/rpcclient/waiter.(*EventBased).Wait(0x14000370d20, {0x105ae5c88, 0x106517820}, {0xe9, 0xc0, 0x9d, 0x69, 0x5e, 0xa6, 0x9, ...}, ...)
	github.com/nspcc-dev/neo-go@v0.114.0/pkg/rpcclient/waiter/waiter.go:274 +0x88
github.com/nspcc-dev/neo-go/pkg/rpcclient/actor.(*Actor).WaitSuccess(0xc1e14f19c3e60d0b?, {0x105ae5c88?, 0x106517820?}, {0xe9, 0xc0, 0x9d, 0x69, 0x5e, 0xa6, 0x9, ...}, ...)
	github.com/nspcc-dev/neo-go@v0.114.0/pkg/rpcclient/actor/actor.go:321 +0x48
github.com/nspcc-dev/neofs-node/pkg/morph/client.(*Client).depositNotary(0x140001b45a0, {0x105ae5c88, 0x106517820}, 0x140000eb8c0, 0x1, 0xffffffff)
	github.com/nspcc-dev/neofs-node/pkg/morph/client/notary.go:218 +0x5d0
github.com/nspcc-dev/neofs-node/pkg/morph/client.(*Client).DepositEndlessNotary(0x140001b45a0?, {0x105ae5c88?, 0x106517820?}, 0x14002f4ba80?)
	github.com/nspcc-dev/neofs-node/pkg/morph/client/notary.go:193 +0x48
main.makeNotaryDeposit(0x14000374908)
	github.com/nspcc-dev/neofs-node/cmd/neofs-node/morph.go:59 +0x150
main.initNetmapService.func3({0x105ad4c01?, 0x14001f7aaf0?})
	github.com/nspcc-dev/neofs-node/cmd/neofs-node/netmap.go:239 +0x28
main.initNetmapService.addNewEpochAsyncNotificationHandler.WorkerPoolHandler.func6.1()
	github.com/nspcc-dev/neofs-node/pkg/morph/event/utils.go:75 +0x2c
github.com/panjf2000/ants/v2.(*goWorker).run.func1()
	github.com/panjf2000/ants/v2@v2.11.3/worker.go:73 +0x7c
created by github.com/panjf2000/ants/v2.(*goWorker).run in goroutine 180
	github.com/panjf2000/ants/v2@v2.11.3/worker.go:50 +0x9c

Expected Behavior

No panic.

Possible Solution

Close channels on one side only, if it's WSClient doing it then waiter shouldn't touch them. If channel is unused it'll be garbage-collected anyway.

Steps to Reproduce

Not really known, but can be seen here:
https://rest.fs.neo.org/Aku38wpn2xEqfEPxmwyAa7N3VnAkMV8PEVkpLqwKM1z3/4538-1766148767/index.html#suites/4f5ca4977289636df3b53e1debc15dcf/f14f02e9e604772e/

It looks like some contexts can expire at ~the same time as channel gets closed by WSClient in which case nil checks below are not helping much.

Regression

Not likely.

Your Environment

Version of the product used: 0.114.0

Originally created by @roman-khimov on GitHub (Dec 19, 2025). ## Current Behavior ``` 2025-12-19T12:45:59.269Z info client/notifications.go:230 RPC connection lost, attempting reconnect 2025/12/19 12:45:59.273577 [ants]: worker exits from panic: close of closed channel goroutine 45049 [running]: runtime/debug.Stack() runtime/debug/stack.go:26 +0x64 github.com/panjf2000/ants/v2.(*goWorker).run.func1.1() github.com/panjf2000/ants/v2@v2.11.3/worker.go:62 +0x120 panic({0x10590fa60?, 0x105acf4b0?}) runtime/panic.go:787 +0x124 github.com/nspcc-dev/neo-go/pkg/rpcclient/waiter.(*EventBased).WaitAny(0x14000370d20, {0x105ae5c88, 0x106517820}, 0x2a6a, {0x140017ee620, 0x1, 0x1}) github.com/nspcc-dev/neo-go@v0.114.0/pkg/rpcclient/waiter/waiter.go:398 +0x894 github.com/nspcc-dev/neo-go/pkg/rpcclient/waiter.(*EventBased).Wait(0x14000370d20, {0x105ae5c88, 0x106517820}, {0xe9, 0xc0, 0x9d, 0x69, 0x5e, 0xa6, 0x9, ...}, ...) github.com/nspcc-dev/neo-go@v0.114.0/pkg/rpcclient/waiter/waiter.go:274 +0x88 github.com/nspcc-dev/neo-go/pkg/rpcclient/actor.(*Actor).WaitSuccess(0xc1e14f19c3e60d0b?, {0x105ae5c88?, 0x106517820?}, {0xe9, 0xc0, 0x9d, 0x69, 0x5e, 0xa6, 0x9, ...}, ...) github.com/nspcc-dev/neo-go@v0.114.0/pkg/rpcclient/actor/actor.go:321 +0x48 github.com/nspcc-dev/neofs-node/pkg/morph/client.(*Client).depositNotary(0x140001b45a0, {0x105ae5c88, 0x106517820}, 0x140000eb8c0, 0x1, 0xffffffff) github.com/nspcc-dev/neofs-node/pkg/morph/client/notary.go:218 +0x5d0 github.com/nspcc-dev/neofs-node/pkg/morph/client.(*Client).DepositEndlessNotary(0x140001b45a0?, {0x105ae5c88?, 0x106517820?}, 0x14002f4ba80?) github.com/nspcc-dev/neofs-node/pkg/morph/client/notary.go:193 +0x48 main.makeNotaryDeposit(0x14000374908) github.com/nspcc-dev/neofs-node/cmd/neofs-node/morph.go:59 +0x150 main.initNetmapService.func3({0x105ad4c01?, 0x14001f7aaf0?}) github.com/nspcc-dev/neofs-node/cmd/neofs-node/netmap.go:239 +0x28 main.initNetmapService.addNewEpochAsyncNotificationHandler.WorkerPoolHandler.func6.1() github.com/nspcc-dev/neofs-node/pkg/morph/event/utils.go:75 +0x2c github.com/panjf2000/ants/v2.(*goWorker).run.func1() github.com/panjf2000/ants/v2@v2.11.3/worker.go:73 +0x7c created by github.com/panjf2000/ants/v2.(*goWorker).run in goroutine 180 github.com/panjf2000/ants/v2@v2.11.3/worker.go:50 +0x9c ``` ## Expected Behavior No panic. ## Possible Solution Close channels on one side only, if it's WSClient doing it then waiter shouldn't touch them. If channel is unused it'll be garbage-collected anyway. ## Steps to Reproduce Not really known, but can be seen here: https://rest.fs.neo.org/Aku38wpn2xEqfEPxmwyAa7N3VnAkMV8PEVkpLqwKM1z3/4538-1766148767/index.html#suites/4f5ca4977289636df3b53e1debc15dcf/f14f02e9e604772e/ It looks like some contexts can expire at ~the same time as channel gets closed by WSClient in which case `nil` checks below are not helping much. ## Regression Not likely. ## Your Environment * Version of the product used: 0.114.0