Impala memory configuration in cloudera

Question

I'm trying to understand the Impala memory settings on my cluster.

We have hosts having 48 GB memory in the cluster. For a host, when I look at the memory resources for each service in cloudera manager, I see that for Impala daemon 38 Gb of memory is allocated. But, Impala Daemon memory limit is set to O , in the Impala configurations.

So, where is the value 38GB getting assigned to Impala Daemon?

And I believe Impala Daemon memory limit is a node level limit, not a cluster level. Is that right?

Please note that static and dynamic pools are also not configured.

Answer 1

If you don't set a process memory limit, Impala will default to using 80% of the memory on the system as it's process memory limit. (Yes, the process memory limit is a per-node value, not a cluster-wide value.)

Note that this does not mean that 80% of the system memory is actually available, but Impala will limit itself to 80% of the memory. That means that if you have other processes using that memory then you'll see swapping.