How to flush old tombstones from a table¶
Audience: Devops professionals, architects
If you have large partitions with lots of tombstones, you can use this workaround to flush the old tombstones.
To avoid data resurrection, make sure that tables are repaired (either with nodetool repair or Scylla Manager) before the
gc_grace_seconds threshold is reached.
After the repair finishes, any tombstone older than the previous repair can be flushed.
Use this article to help you find large partitions.
Run nodetool repair to synchronize the data between nodes. Alternatively, you can use Scylla Manager to run a repair.
nodetool repair <options>;
gc_grace_secondsto the time since last repair was started - For instance, if the last repair was executed one day ago, then set
gc_grace_secondsto one day (86400sec). For more information, please refer to this KB article.
To prevent the compaction of unsynched tombstones, it is important to get the timing correctly. If you are not sure what time should set, please contact Scylla support.
ALTER TABLE <keyspace>.<mytable> WITH gc_grace_seconds = <newTimeValue>;
Steps 3 & 4 should be repeated on ALL nodes affected with large tombstone partitions.
Run nodetool flush
nodetool flush <keyspace>.<mytable>;
Run compaction (this will remove big partitions with tombstones from specified table)
By default, major compaction runs on all the keyspaces and tables, so if we want to specyfy e.g. only one table, we should point at it using arguments:
<keyspace>.<mytable>. For more information, please refer to this article.
nodetool compact <keyspace>.<mytable>;
Alter the table and change the grace period back to the original
ALTER TABLE <keyspace>.<mytable> WITH gc_grace_seconds = <oldValue>;