This document is a step by step procedure for upgrading Scylla Monitoring Stack from version 3.x to 3.y, for example, between 3.0 to 3.0.1.
We recommend installing the new release next to the old one, running both in parallel, and making sure it is working as expected before uninstalling the old version.
Change to the directory you want to install the new Monitoring stack. Download the latest release: You can download the .zip or the .tar.
wget -L https://github.com/scylladb/scylla-monitoring/archive/scylla-monitoring-3.y.zip
unzip scylla-monitoring-3.y.zip
cd scylla-monitoring-scylla-monitoring-3.y/
Replace “y” with the new minor release number, for example, 3.0.1.zip
Copy the scylla_servers.yml and scylla_manager_servers.yml from the version that is already installed.
cp /path/to/monitoring/3.x/prometheus/scylla_servers.yml prometheus/
cp /path/to/monitoring/3.x/prometheus/scylla_manager_servers.yml.yml prometheus/
Starting from Scylla-Monitoring version 2.2, you can run:
./start-all.sh --version
To validate the Scylla-Monitoring version.
To validate that the Monitoring stack starts correctly, first in parallel to the current (3.x) stack.
./start-all.sh -p 9091 -g 3001 -m 9095
Browse to http://{ip}:9091
And check the Grafana dashboard
Note that we are using different port numbers for Grafana, Prometheus, and the Alertmanager.
Caution
Important: do not use the local dir flag when testing!
When you are satisfied with the data in the dashboard, you can shut down the containers.
Caution
Important: Do not kill the 3.x version that is currently running.
Use the following command to kill the containers:
./kill-all.sh -p 9091 -g 3001 -m 9095
You can start and stop the new 3.y version while testing.
Note: migrating will cause a few seconds of blackout in the system.
We assume that you are using external volume to store the metrics data.
At this point you have two monitoring stacks running side by side, you should kill both before continuing.
Kill the newer version that runs in testing mode by following the instructions on how to Killing the new 3.y Monitoring stack in testing mode in the previous section
kill the older 3.x version containers by running:
./kill-all.sh
Start version 3.y in normal mode
From the new root of the scylla-monitoring-scylla-monitoring-3.y run
./start-all.sh -d /path/to/data/dir
Point your browser to http://{ip}:3000 and see that the data is there.
To rollback during the testing mode, follow Killing the new 3.y Monitoring stack in testing mode as explained previously and the system will continue to operate normally.
To rollback to version 3.x after you completed the moving to version 3.y (as shown above). Run:
./kill-all.sh
cd /path/to/scylla-grafana-3.x/
./start-all.sh -d /path/to/data/dir