Clearly explanation of all kinds of memory in Presto

4
votes

I'm SO confusing about memory-settings in Presto. Please check this out below:

  1. query.max-memory

  2. query.max-memory-per-node (base config) enter image description here

  3. query.max-total-memory (release in 0.205) enter image description here

  4. resources.reserved-system-memory (admin properties) enter image description here

  5. Memory Pools (General Pool & Reserved Pool) enter image description here

That is all I can find out.

Here are my Presto settings:

  1. etc/config.properties

query.max-memory=2.25GB
query.max-total-memory=2.25GB
query.max-memory-per-node=0.75GB

  1. jvm: -Xmx3G -Xms3G

Base on my settings, I found these rules:

  1. query.max-memory-per-node <= jvm * 0.25

  2. General Pool = jvm * 0.4, same as the default value of resources.reserved-system-memory

  3. Reserved Pool = jvm * 0.3

Here are my questions:

  1. The relationship between all kinds of memory-settings. (like resources.reserved-system-memory is General Pool? query.max-total-memory=user memory+system memory, what are user memory and system memory?)

  2. What General Pool and Reserved Pool use for? How can I change their values?

  3. General Pool(40% of jvm), Reserved Pool(30% of jvm), where is the last 30% of jvm? or How Presto assigns my 3GB jvm memory?

1
presto

1 Answers

2
votes
reserved = query.max-memory-per-node 
system = 40% of heap 
general = heap - system - reserved