initializing module on django+mod_wsgi+apache

2
votes

My django application is running on apache+wsgi. One of the module in my django app needs to load a Java library via jpype and this Java library takes long time to initialize due to its application nature.

The problem is that, for each http request handled by django in apache+wsgi setup, this Java library is re-loaded. However, this does not happen when I run my same app in development web server (python manager.py runserver 8000). In development web server, it only loads the Java library only once.

Is there any way to change apache or mod_wsgi configuration or my django app so that it won't reload my Java library for every http request?

Many thanks.

Andy

1
pythondjangoapachewsgidjango-wsgi
In which part of your project/app the library is loaded? Project settings? In app-specific files? - Torsten Engelbrecht
The library is loaded in one of my app specific files. This file contains a class static method to load java library if not already loaded. - Andy
The problem is that when using Django on production more an app specific file might be imported more than once so your call to the Java library will be more than once. I had same issues with PyLucene before which was initiallizing the Java VM. A good solution is to load the library in your settings.py (which should only imported once per wsgi thread) and then reference to this in your app. - Torsten Engelbrecht
when I tried this, jpype crashed. File "/Library/Python/2.6/site-packages/jpype/_core.py", line 44, in startJVM _jpype.startup(jvm, tuple(args), True) RuntimeError: Unable to start JVM at src/native/common/jp_env.cpp:54 Invalid memory access of location 0x8 rip=0x10168aed0 - Andy
Then I think I have no working solution for you. For my lucene installation this approach worked. I am not really a Java guy though and try yo avoid using libraries from another language if possible. Good luck finding a solution. - Torsten Engelbrecht

1 Answers

1
votes

You are possibly just getting confused and are actually using as poor Apache/mod_wsgi configuration. Specifically, you are likely using embedded mode with Apache prefork MPM. That is bad because Apache will use lots of single thread processes and so the code has to be loaded in all of them. That is why you probably think it is happening on each request against the same process, where in reality, each request is hitting a different process.

Ensure you are using daemon mode of mod_wsgi and that your code is thread safe and so use single multithreaded process and it shouldn't have the issue.

Edit your question and add your Apache/mod_wsgi configuration snippets from Apache configuration file and state what Apache MPM you are using.