How to remove tombstones with a timestamp in the future

Question

How can tombstones that were inserted with a timestamp set in the very distance future be removed?

Context: There was a bug in some code that inserted entries with a timestamp in the distant future. The objects were manually deleted by created a tombstone further in the future.

Now inserts that have the same partition key are automatically deleted because of the future tombstone entry. These entries are not removed during compaction because of their timestamp. How can these future tombstones be removed so that entries with the same PKs can be inserted correctly?

Answer 1

I reproduced the same issue by setting client server to +30 mins. I ran delete on a partition from the client. It created tombstone with future timestamp. All my update on source database server with correct time are being ignored because Cassandra only cares about most recent timestamp data.

I did these steps to clear tombstones on a table with this issue

1) Set table property gc_grace_seconds to 0

2) nodetool flush keyspace table

3) nodetool compact keyspace table

I have just one node. If you have multiple nodes then you need to do the above steps on all the nodes except setting table property

NOTE: Doing this will clear all the tombstones and also Compaction is resource intensive