0
votes

when using post in django, a ascii string will be automatic transfer into unicode string. for example:

s = '\xe2\x80\x99'

is a str type string. (Which is utf-8 format)

when post this string to django, and then get it from request.POST, it is transferred to unicode string:

u'\xe2\x80\x99'

this may cause decode/encode error, because python thought it was a unicode string, but it is a utf-8 string in fact.

My question is how to FORCE convert unicode string to ascii string? Which means just remove the pre 'u' from u'\xe2\x80\x99' to '\xe2\x80\x99'. The traditional method like decode and encode may not work in this situation.

1
Why can't you just do mystr.decode("utf8")? - Joran Beasley
You need to add a lot more detail. In particular, how are you posting that string to Django? What does your HTML look like, and what encoding is your HTTP request using? - Peter DeGlopper

1 Answers

3
votes

When receiving the request, the encoding of the response is mis-declared as (probably) iso-8859-1, or perhaps not declared at all and defaulting to that encoding. The web site should declare its encoding correctly with a header:

<headers>
<meta http-equiv="content-type" content="text/html;charset=UTF-8">
</headers>

But if that isn't under your control, you can undo the encoding and decode it correctly:

>>> s = u'\xe2\x80\x99'
>>> s.encode('iso-8859-1')
'\xe2\x80\x99'
>>> s.encode('iso-8859-1').decode('utf8')
u'\u2019'