Ruby 2.0 introduces a copy-on-write friendly garbage collector. My processes don't seem to keep memory shared for more than a few minutes - it seems to move from shared_dirty to private_dirty quite quickly.
Some others have had success getting this to work:
- https://gist.github.com/kenn/5105175
- http://marianposaceanu.com/articles/on-ruby-2-0-memory-usage-unicorn-and-heroku
This program can be used to check memory stats on Linux: https://gist.github.com/kenn/5105061
My unicorn configuration: https://gist.github.com/inspire22/f82c77c0a465f1945305
For some reason my unicorn apps, also with preload_app=true, have much less shared memory. Ruby 2.0-p195, rails 3.2, linux 2.6.18 (centos)
[root@thorn script]# ruby memstats.rb 4946
Process: 4946
Command Line: unicorn_rails worker[4] -c /u/apps/newap/current/lib/unicorn.rb -E production -D
Memory Summary:
private_clean 0 kB
private_dirty 56,324 kB
pss 60,256 kB
rss 83,628 kB
shared_clean 4,204 kB
shared_dirty 23,100 kB
size 108,156 kB
swap 68 kB
If I shutdown the master process entirely (not just a HUP) then restart it and immediately check a worker before any requests have queued, I get a better story:
[root@thorn script]# ruby memstats.rb 5743
Process: 5743
Command Line: unicorn_rails worker[4] -c /u/apps/newap/current/lib/unicorn.rb -E production -D
Memory Summary:
private_clean 0 kB
private_dirty 21,572 kB
pss 27,735 kB
rss 66,296 kB
shared_clean 2,484 kB
shared_dirty 42,240 kB
size 91,768 kB
swap 0 kB
But within 5 seconds of being started up, they're back to ~20MB of shared_clean+shared_dirty.
I suspected that swapping might be causing the problem, but after lowering swappiness and making sure that neither the parent nor child processes are being swapped out (using swapstats.rb), the problem persists.
I don't understand exactly what shared_dirty memory is, and how it gets turned into private memory. I'd also love suggestions for improving the longevity and amount of my shared memory. Thanks!