Replace a Running Node in a Scylla Cluster¶
There are two methods to replace a running node in a Scylla cluster.
Add a new node to the cluster and then decommission the old node¶
Adding a new node and only then decommissioning the old node allows the cluster to keep the same level of data replication throughout the process, but at the cost of more data being transferred during the procedure. When adding a new node to a Scylla cluster, existing nodes will give the new node responsibility for a subset of their vNodes, making sure that data is once again equally distributed. In the process, these nodes will stream relevant data to the new node. When decommissioning a node from a Scylla cluster, it will give its vNodes to other nodes, making sure data is once again equally distributed. In the process, this node will stream its data to the other nodes. Hence, replacing a node by adding and decommissioning redistribute the vNodes twice, streaming a node worth of data each time.
Replace a running node - by taking its place in the cluster¶
Stopping a node and taking its place in the cluster is not as safe as the data replication factor is temporarily reduced during the process. However, it is more efficient, as vNode distribution does not change, and only one node worth of data is streamed
Run nodetool drain command (Scylla stops listening to its connections from the client and other nodes).
Stop the Scylla node you want to replace