mintty terminal, python, curses, and utf8 : incorrect display

Question

I've asked here but I'm pretty sure I won't get an answer.

copy - paste this python code :
execute it
in a classical utf8 shell it work properly and displays "Coordonnées" but in my mintty terminal at home it displays "CoordonnM-CM-)es". Any idea how I could make it work, or where I should look (mintty has no character encoding options)?

code:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os, sys, time, random, copy, pprint
import termios, tty, fcntl
import thread, curses
import locale

def curses_main(stdscr, code):
    curses.curs_set(0)
    stdscr.addstr(2, 0, "Coordonnées")
    stdscr.refresh()
    # (1) = délai d'attente de 1/10 ème de seconde :
    curses.halfdelay(1)
    while True:
        try:
            c=stdscr.getch()
        except:
            c=-1
        if 0<c<256:
            if chr(c) in 'Qq':
                break
def main():
    locale.setlocale(locale.LC_ALL, '')
    code = locale.getpreferredencoding()
    curses.wrapper(curses_main, code)

if __name__ == '__main__':
    main()

Update

Error: I thought found it: it's written everywhere to begin with that code:

locale.setlocale(locale.LC_ALL, '')

so i tried with that and it worked:

locale.setlocale(locale.LC_ALL, 'fr_FR')

=> force it to French language

... it worked but only for the accents. If i try to print line drawing chars:

stdscr.addstr(20,20, "─ │ ┌ ┘ ┐ └ ├ ┤ ┬ ┴ ┼ ═ ║ ╔ ╝ ╗ ╚ ╠ ╣ ╦ ╩ ╬")

gives me:

M-b~T~@ M-b~T~B M-b~T~L M-b~T~X M-...

what is head banging is that vim displays properly those chars. So it seems like it comes... from python? From my Windows/cygwin/mintty that works:

env | grep -i lang
LANG=C.UTF-8

if I force encoding at home in my Windows/cygwin/mintty that doesn't work:

locale.setlocale(locale.LC_ALL, 'C.UTF-8')

this doesn't work.

Any idea where I could look? python seems to override properly the locale settings... the console is properly configured, and the fonts can display line drawing chars, because I see them with vim. So thinking more about it, it may be either a problem of python or a problem of curses...

Damn! The python code doesn't display properly. If you know how to display it here, may I ask you to go to this link where the code is displayed and copy paste here (and tell me how to make it work here =) Thanks a lot ) — Olivier Pons
What are the LC_ALL, LC_CTYPE, and LANG environment variable settings? What are the locale/charset settings on the Text page of mintty's options? — ak2
I first thought it worked so I posted my own answer... but it just worked for specific French chars (like "é"). Line drawing chars didn't work. Please see my answer, where I put more details of the problem. Maybe it doesn't come from the mintty console itself... — Olivier Pons

Lennart Regebro Lennart Regebro · Accepted Answer · 2011-01-24T10:30:15

Your locales preferred encoding and your tty's encoding doesn't have to be the same.

Check what sys.stdin.encoding and sys.stdout.encoding is and try to use that.

Wild guesses is that the default encoding for mintty is UTF8, but your locales preferred encoding is (since it's windows) cp1252.

mintty terminal, python, curses, and utf8 : incorrect display

Update

2 Answers