Python Requests: submit JSON in non utf-8 encoding

Question

I must submit data to a web service which uses a MySQL server configured with latin-1 as the tables collation. This web service receives a JSON payload and stores some data in the database. UTF-8 is the correct encoding of JSON data for transport, but the web service does not correctly encodes it to latin-1 to save in the database.

So when a submit a JSON like {'Key': 'ç'}, the web service stores it in the incorrect encoding and displays to me in a web page as

"The value of Key is 'Ã§'"

I'm searching for a workaround.

My client is in Python 3.5 and I'm using the Requests library, which uses urllib3 as plumbing. When I submit a dict to Requests, it uses the json lib to convert it to a string type, and urllib3 submits it as a bytes to send the request through the network.

Can I make Requests submit my JSON object string with a latin-1 encoding?

Sraw Sraw · Accepted Answer · 2018-05-22T01:40:47

There is a way, you can manually send json data:

a = {'Key': 'ç'}
data = json.dumps(a).encode("latin-1")
requests.post(url, data=data)

Then the content you send is a binary string encoded with latin-1. Then you can decode it from the server side:

res.body.decode("latin-1")

Python Requests: submit JSON in non utf-8 encoding

1 Answers