Cassandra Mutation too Large, for small insert

Question

I'm getting these errors:

java.lang.IllegalArgumentException: Mutation of 16.000MiB is too large for the maximum size of 16.000MiB

in Apache Cassandra 3.x. I'm doing inserts of 4MB or 8MB blobs, but not anything greater than 8MB. Why am I hitting the 16MB limit? Is Cassandra batching up multiple writes (inserts) and creating a "mutation" that is too large? (If so, why would it do that, since the configured limit is 8MB?)

There is little documentation on mutations -- except to say that a mutation is an insert or delete. How can I prevent these errors?

Alas, it is really complicated. I'm using phantom-dsl to write to Cassandra from a scala web service. But basically, I'm doing an insert of the form "insert into foo (a, b) values (aa, bb)" where a is of type string and b of type blob. And where aa is a 32-character string and bb is a 4MB+16byte blob. I'm firing off many of these inserts, and not doing any explicit batching. I would expect that this should result in a mutation of no more than 32 bytes + 4MB + 16 bytes. This is a lot smaller than the 16MB limit. — eswenson
I would work to break it down. Either you'll find something in your app that isn't doing what you expected or you'll end up with a useful repro you can use to log a bug. — Nom de plume

root root · Accepted Answer · 2016-11-17T00:01:58

you can increase the commit log size to 64 mb in cassandra.yaml

commitlog_segment_size_in_mb: 64

By default the commitLog size is 32 mb.

By design intent the maximum allowed segment size is 50% of the configured commit_log_segment_size_in_mb. This is so Cassandra avoids writing segments with large amounts of empty space.

you should investigate why the write size has suddenly increased. If it is not expected i.e. due to a planned change then it may well be a problem with the client application that needs further inspection.

Cassandra Mutation too Large, for small insert

1 Answers