This article describes the troubleshooting procedure when node decommission fails.
During decommissioning, the streaming process starts, and the node streams its data to the other nodes in the Scylla cluster. The process may fail if the node fails to read from the HDD or a network problem occurs.
The node is stuck in the decommission status.
nodetool status command to check the node status. The expected result is
UL (Up Leaving).
Check the node status from the other nodes in the cluster to verify that the status of the decommissioned node status is
nodetool netstats command does not show ongoing streaming.
The following error message will appear in the logs:
nodetool: Scylla API server HTTP POST to URL '/storage_service/decommission' failed: stream_ranges failed
Restart the node you are decommissioning.
sudo systemctl restart scylla-serverdocker exec -it some-scylla supervisorctl restart scylla
(without restarting some-scylla container)
nodetool status command to verify the node is in
nodetool decommission again.