How to control Python's stdout encoding under Apache/WSGI

Question

I'm writing a Django app, and it prints Unicode to stdout. It works perfectly on the development server, but with Apache and modwsgi I get UnicodeEncodeError: 'ascii' codec can't encode characters ....

So, the stdout encoding is forced to ASCII. I've found a number of solutions, all of which seem like they should work according to documentation, but none work for me:

I tried setting the PYTHONIOENCODING variable to utf-8, using the SetEnv directive.
I also tried setting LANG, LC_ALL, LC_CTYPE to en_US.UTF-8 using SetEnv and PassEnv.
Since my Apache server is running under systemd, I tried setting the variables using Environment= directives in the service file.
Finally, I tried adding lang=en_US.UTF-8 locale=en_US.UTF-8 to the WSGIDaemonProcess directive.

The variables are seen in the environment (reflected e.g. on Django error pages in debug mode), but the error still occurs.

Why don't these things work and what should I do? Overriding stdout in the code that works fine outside Apache or encoding everything manually doesn't feel like the right approach.

The error occurs at a plain print statement. Here's the entire view that raises the error:

def test(request):
    import locale
    print locale.getdefaultlocale()
    print locale.getpreferredencoding()

    s = u'привет'
    print s
    return s

Apache error log:

('en_US', 'UTF-8')
UTF-8
Internal Server Error: /app/testview/
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner
    response = get_response(request)
  File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
    response = self._get_response(request)
  File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/web/my_site/my_app/views.py", line 964, in test
    print s
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-5: ordinal not in range(128)

When I print sys.stdout itself, I see that it's actually a mod_wsgi.Log object. In the WSGI file, all I have for logging is:

logging.basicConfig(stream=sys.stderr)

Versions: Python 2.7.14, Apache 2.4.29, mod_wsgi 4.5.20 on Arch Linux

What about to from __future__ import unicode_literals ? is a workaround to you? — dani herrera
Though pretty old, we are facing same issue these days. Did you manage to figure it out? — user1997656

Graham Dumpleton Graham Dumpleton · Accepted Answer · 2017-12-24T10:49:48

The first two will not do anything as SetEnv only sets per request WSGI environ, not process variables.

Adding lang/locale on WSGIDaemonProcess should work in most cases provided you are actually running things in daemon mode. It will not work for embedded mode.

http://blog.dscpl.com.au/2014/09/setting-lang-and-lcall-when-using.html

Have you verified you are actually running in daemon mode.

http://modwsgi.readthedocs.io/en/develop/user-guides/checking-your-installation.html#embedded-or-daemon-mode

How to control Python's stdout encoding under Apache/WSGI

2 Answers