Varnish RAM problems

0
votes

I have problems with RAM on servers which use varnish and only it (no other apps there). Each machine has 64GB RAM available for caching and has three separated varnish services for different backends. Currently, the sum of RAM allocated to varnish from all services is 24GB RAM on each server. I want to increase this value up to 48GB (75% of the whole memory available) but I have some problems.

When I tried to allocate 8GB more just for one service (32GB from all), the committed memory got a peak to 70GB RAM(?). What's more, the increased service has restarted a few times after getting 100% ram allocated to its limit (error msg: child not responding to CLI, killing it/died signal=6/Panic message: Assert error in vbf_fetch_thread()). In addition, services use a lot of VSZ (virtual memory size), it that okay?

2
varnishvarnish-4

2 Answers

1
votes

This could be Transient memory which is uncapped by default and we'll use malloc as storage. In Transinet stevedore Varnish stores object with a TTL < 10s, therefore if you have many of those that's what you see.

The solution is to either increase the TTLs or cap the Transient storage.

1
votes

I've changed jemalloc parameters lg_dirty_mult and lg_chunk. Now I was able to assign 42 GB ram to varnish and committed memory is around 60 GB now. The main varnish task is to cash images and TTL is set to 365d.