Unicorn downs for few seconds after USR2 and QUIT signals (Should not)

Question

I have a very basic app with unicorn. Rails 5.1 app that simply echos/prints "Hello World". For the tests i gave 4 workers.

unicorn.rb:

worker_processes 4
preload_app true

timeout 30
listen '127.0.0.1:3000', :tcp_nopush => true, :backlog => 4096
# Logging
...

Now, I send USR2 signal to master process. Master process successfully received my signal and created another master then marked this as an "old" and write PID to an oldbin file

Problem starts here. When I make sure second master and their childs created, I send QUIT to old master. Problem is exactly here. When old master receives QUIT, The whole new and old masters and childs dropped and re-created. This makes my system down for few seconds.

Basically, New master kill itself and then re-create. Why? What am I doing wrong here? I thought this is zero downtime. I can assure you I have downtime for 3-5 seconds.

I checked, old master creates new master with PID 5 (for example) and their childs as 6, 7, 8, 9. When I quit from old one, new master becomes 10 and workers becomes 11,12,13,14. Simply shows that both old and new ones quit and re-re-created.

I do not use -D command. I use foreman to generate systemd file and this is how it looks like:

[Unit]
PartOf=bp-web.target

[Service]
User=Pratha
WorkingDirectory=/var/www/app/releases/20170628140005
Environment=PORT=%i
ExecStart=/bin/bash -lc 'exec bundle exec unicorn -E production -c config/unicorn.rb'
Restart=always
StandardInput=null
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=%n
KillMode=mixed
TimeoutStopSec=5

I use capistrano for release-cycle and deployment.
If I use puma with threads (even 16 worker 50 threads) i get real "zero downtime".

Dennis Dennis · Accepted Answer · 2017-06-29T08:37:47

This is an expected behavior:

Supervisord requires the unicorn process to not daemonize. Also sending SIGUSR2 to unicorn causes the old master to die. Since supervisord watches the old master this will cause it to consider the application as exited, even tho it's running with a new process id. Finally Supervisor will try to restart the application, and fail to do so because all sockets are in use by the new unicorn master.

Solution:

Either use -D option and start your unicorn rails app yourself without systemd/supervisord. Then USR2 will work.
Or use unicornherder for wrapper. This will do the job for you. No downtime.

See link: Unicorn + Systemd/Supervisord & Zero Downtime = unicornherder

Unicorn downs for few seconds after USR2 and QUIT signals (Should not)

1 Answers