Is the Cassandra commit log just a backup of the (in memory) heap

Question

I am new to cassandra and now I am trying to make a production server. In documentation I read that data and commitlog should be on separated drives (btw I use hdd), I tought the commitlog will increase to many Gb of data and I created 2 hard drives (both 100Gb), on first will be data (sstables) on second commitlog. But now in config I see: commitlog_total_space_in_mb: 4096, and I think this shoul be 'the maximum heap size'. If commitlog reached this limit, then it seems that memtables increased in size and needs to be flushed to disk and also the data that was contained in memtables is removed from commitlog.

So tell me please if am I right: commitlog is like a backup of heap and cannot increase to hundreds of Gb?

And I do not need to create 100Gb hard drive for that, will be enough a 4Gb partition (on another hard drive, not the same where the data (sstables) is stored)?

Answer 1

commitlog is like a backup of heap and cannot increase to hundreds of Gb?

The commitlog is temporary storage used for a copy of data stored in the memtables. This is in case power is lost or the server otherwise crashes before the memtables can be written out as SSTables. As soon as a memtable is flushed to disk, the commitlog segments are deleted.

The rest of the data on heap (caches, in-flight data, etc.) is never stored in a commitlog. So your commitlog will normally be significantly smaller than your heap.

And I do not need to create 100Gb hard drive for that, will be enough a 4Gb partition (on another hard drive, not the same where the data (sstables) is stored) ?

A smaller partition will be fine, but you may want to leave a little headroom above 4 GB.