Audience: Devops professionals, architects
If you have large partitions with lots of tombstones, you can use this workaround to flush the old tombstones.
To avoid data resurrection, make sure that tables are repaired (either with nodetool repair or Scylla Manager) before the gc_grace_seconds threshold is reached.
After the repair finishes, any tombstone older than the previous repair can be flushed.
Note
Use this article to help you find large partitions.
Run nodetool repair to synchronize the data between nodes. Alternatively, you can use Scylla Manager to run a repair.
nodetool repair <options>;
Set the gc_grace_seconds to the time since last repair was started - For instance, if the last repair was executed one day ago, then set gc_grace_seconds to one day (86400sec). For more information, please refer to this KB article.
Note
To prevent the compaction of unsynched tombstones, it is important to get the timing correctly. If you are not sure what time should set, please contact Scylla support.
ALTER TABLE <keyspace>.<mytable> WITH gc_grace_seconds = <newTimeValue>;
Note
Steps 3 & 4 should be repeated on ALL nodes affected with large tombstone partitions.
Run nodetool flush
nodetool flush <keyspace>.<mytable>;
Run compaction (this will remove big partitions with tombstones from specified table)
Note
By default, major compaction runs on all the keyspaces and tables, so if we want to specyfy e.g. only one table, we should point at it using arguments: <keyspace>.<mytable>. For more information, please refer to this article.
nodetool compact <keyspace> <mytable>;
Alter the table and change the grace period back to the original gc_grace_seconds value.
ALTER TABLE <keyspace>.<mytable> WITH gc_grace_seconds = <oldValue>;