one wsgi app eats all apache clients

Question

We have a SOAP mod_wsgi (apache) app which gets heavy load sometimes. The same Apache servers some other wsgi-apps. Unfortunately you can set MaxClients only at server level, not per wsgi-app.

We get:

server reached MaxClients setting, consider raising the MaxClients setting

Is there a way to stop this wsgi app from eating all apache workers?

I want to return 503 "Service Unavailable" only to the SOAP client who connects to the SOAP wsgi app.

Apache config snippet:

   WSGIDaemonProcess soap_app threads=1 processes=3
   WSGIScriptAlias /soap_app /home/soap_app/django_wsgi.py
   <Location "/soap_app/">
       WSGIProcessGroup soap_app
       WSGIApplicationGroup %{GLOBAL}
   </Location>

There are only 3 wsgi daemon processes for the soap app. But it occupies much more apache workers.

Update: We use apache prefork mpm. There are N apache worker. And for mod_wsgi we use prefork, too. There are M mod_wsgi worker processes. The apache worker count can be controlled by MaxClients. The mod_wsgi worker count is controlled by the above config.

I think you can't handle this inside the python wsgi app (django). I guess it needs to be done by the mod_wsgi or apache config.

Here are the first lines of mod_status:

  Server Version: Apache/2.2.17 (Linux/SUSE) mod_ssl/2.2.17 OpenSSL/1.0.0c
  mod_wsgi/3.3 Python/2.7
  Server Built: 2011-07-26 13:43:36.000000000 +0000
===============================================================================
  Current Time: Thursday, 20-Sep-2012 13:15:11 CEST
  Restart Time: Thursday, 06-Sep-2012 16:30:45 CEST
  Parent Server Generation: 0
  Server uptime: 13 days 20 hours 44 minutes 25 seconds
  Total accesses: 307471 - Total Traffic: 7.7 GB
  CPU Usage: u11.85 s1.56 cu0 cs0 - .00112% CPU load
  .257 requests/sec - 6.8 kB/second - 26.4 kB/request
  127 requests currently being processed, 13 idle workers
WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
WWWWWWWWWWWWWWWWWWWWKWWWWW_WWWWWKWWWWWWWWW_WWWWWW_WW_WWWWWWK._WW
W__WW__._W_W__........
Scoreboard Key:
"_" Waiting for Connection, "S" Starting up, "R" Reading Request,
"W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
"C" Closing connection, "L" Logging, "G" Gracefully finishing,
"I" Idle cleanup of worker, "." Open slot with no current process
Srv  PID   Acc   M CPU  SS   Req  Conn  Child Slot   Client         VHost     Request
0-0  15135 0/27/ W 0.04 8417 0    0.0   0.37  290.12 10.1.1.1       foohost   POST /soap_app/foo HTTP/1.1
           11553
           0/
1-0  15142 125/  W 0.18 7354 0    0.0   2.48  324.82 10.1.1.1       foohost   POST /soap_app/foo HTTP/1.1
           12475
           0/
2-0  18350 157/  W 0.27 4780 0    0.0   4.84  300.09 10.1.1.1       foohost   POST /soap_app/foo HTTP/1.1
           11249
3-0  20112 0/10/ W 0.02 7106 0    0.0   0.29  315.77 10.1.1.1       foohost   POST /soap_app/foo HTTP/1.1
           12714
4-0  16562 0/35/ W 0.07 7853 0    0.0   0.96  328.98 10.1.1.1       foohost   POST /soap_app/foo HTTP/1.1
           12098
5-0  20152 0/25/ W 0.06 6732 0    0.0   0.71  288.17 10.1.1.1       foohost   POST /soap_app/foo HTTP/1.1

Please post a comment, if you don't understand the question. — guettli
do daemon process and apache worker not mean the same thing? — Xuan
did you read this question with it's answers? link maybe helps you to get a clearer view on the situation — Andreas
Here is the same question on the mod_wsgi mailing list: groups.google.com/d/msg/modwsgi/2qFjdjzydrE/bQIRGpkGP1wJ — guettli

barracel barracel · Accepted Answer · 2013-02-21T09:19:19

All the request are served by an Apache child (controlled by MaxClients) but every time a request hits the soap_app url the Apache child will wait for one of the the 3 WSGIDaemonProcess. If you receive request to soap_app faster than you can serve them with 3 processes eventually you will run out of Apache child.

The only way I see to control the number of Apache child dedicated to soap_app is to use mod_proxy and proxy the soap_app request to another "service". The proxy pass directive allows you to define the number of concurrent request to be serve, that will be equal to the number of Apache child you want to use for soap_app.

The "service" to serve the soap_app request could be another VirtualHost of the same Apache (never tested it) or a gunicorn instance with the soap_app application

one wsgi app eats all apache clients

1 Answers