Redis randomly starts complaining about not being able to save RDB snapshots

Question

So I'm running a Rails app that's leveraging redis and sidekiq. The config/sidekiq.rb file contains some information about setting the Redis pool size, but I commented it out so that Sidekiq can determine that dynamically (from what I understand).

So while I have some sidekiq jobs running in the background, everything works just fine. However, after maybe about 30 - 60 minutes, sidekiq seems to still be running, but it spits out this error like every minute (guessing it's the interval for my sidekiq cron job):

2018-01-19T17:52:21.906Z 95551 TID-ovs5kqylg ERROR: CRON JOB: MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this inst ance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error.

2018-01-19T17:52:21.906Z 95551 TID-ovs5kqylg ERROR: CRON JOB: /home/nutella/.rvm/rubies/ruby-2.5.0/lib/ruby/gems/2.5.0/gems/redis-4.0.1/lib/redis/client.rb:119:in `call'

To fix this, I have to kill the sidekiq process and restart it for another 30-60 mins.

Answer 1

You have not enough RAM.

By default, Redis is configured to save it's data on disk. To achive this it creates a fork from the main process. At least for Linux systems a child process iherits used RAM from the parent process. It seems that you have not enoguh left.

The simplest option would be to increase RAM. Sidekiq consumes lots of RAM, therefore the problem has disappeared when you have Sidekiq turned off.

You can change Redis config to not throw an exception when it has not saved but it will not save you memory leak.