Unix : Memory mapped files , constraints applicable?

Question

This question is for understanding the kind-off constraints applicable for a Mem-Mapped file in unix environ.

We have an APP running in unix environment that hosts and serves files with Mem-mapped files of Key-Value with read only access, also is capable of refreshing on runtime when a new version of file is copied ( probabaly with more key-value pairs).

What i observe is , since the file is Mem-Mapped , as we refresh the file with more key-value pairs VIRT memory consumption increases with not much RES mem consumption.

  PID    PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
12948    16   0 43240 9936 2996 S  0.0  0.1   0:00.00 lookup_server
12951    16   0  562m  16m 9972 S  0.0  0.1   0:00.09 lookup_server

As i understand this is because the whole file is copied as virtual memory pages in harddrive , and only few pages that are in demand are in RES mem.

Is my assumptions right , that

with Mem-Mapping files , the file size is not limited to available physical RAM , as the files would be paged-in/out by the OS on-demand.
and only limiting factor could be disk space configured for virtual memory. in this case how can i identify the disk space identified by OS for virtual memory extension? where does the virtual memory foot print of the file get stored in harddisk ?

keen to know if virtual memory is limited by disk space, and where on disk unix maintains the virtual memory pages? — Shashi

Unknown Unknown · Accepted Answer · 2014-03-27T21:12:27

I think 2) applies only if you map the file with MAP_PRIVATE and then only if you modify the pages in memory. If you map the pages without MAP_PRIVATE the file is already on disk and does not need to be copied into the swap file. 1) Is correct -- you can map larger files than available memory. But remember, that the OS still has to allocate page tables -- so don't try to map a 1TB file.

Unix : Memory mapped files , constraints applicable?

1 Answers