json.decoder.JSONDecodeError: Extra data: line 1 column 5 (char 4) AND raise JSONDecodeError("Extra data", s, end)

1
votes

I'm working on a stock market analysis app using the yahoo finance api on rapidapi. I successfully pull the data from the api using requests and the result is a requests.model.response. I found that when I convert this to a json dict I get the following error (same as title):

json.decoder.JSONDecodeError: Extra data: line 1 column 5 (char 4)/raise JSONDecodeError("Extra data", s, end)

to resolve the issue I tried alternatively converting the model.response to a string then back to a dict with the following:

responset = response.text responseld = json.loads(json.dumps(responset))

I did this based off of some research I did online. No, it doesn't exactly make sense to me because this does merely seem to be a conversion of type then back to the same type. But I've thought that maybe dumps does something more than just a type conversion because I saw in another thread that this approach (that I've shown here) has been used to resolve a similar error for somebody else.

Do you have any advice? Please let me know if you need more information. Thank you.

1
pythonjsonpython-3.x
Can you provide text of response that was failing with conversion? – Arty
Also one possible solution is to parse binary data of response directly using result = json.loads(response.content.decode('utf-8')). Here response.content is raw-bytes data of response. – Arty
Thank you for the responses and possible solution, I will try it out tomorrow. Are you looking for the full text of the requests.model.response? Or something else? I should be able to get that and post it tomorrow. – Benjamin Moon
Basically to investigate your error (if my solution is not enough) would be nice to have both response data or url (if it is not secret) and piece of code that you use to convert this data to json. So that we can reproduce error on our side and try to fix it. – Arty
Also there is a built-in way to decode json response in requests which is result = response.json(). My code mentioned earlier above is more low-level and may be easier debugged. – Arty

1 Answers

1
votes

There are several ways to decode json from response obtained through requests library.

I expect that you got response with code similar to this:

response = requests.get(url)

First, most low-level, but reliable and easy to debug, it uses response.content which is raw body bytes of response:

result = json.loads(response.content.decode('utf-8'))

second uses response.text which is basically response.content automatically converted to text string using best matching encoding by requests library:

result = json.loads(response.text)

and most high-level and simple, where requests takes all care to do the right job of conversion:

result = response.json()

One of this methods should work.

Also after receiving response you always need to check status code, because requests doesn't throw exception on non-200 status code, so you need to do assert response.status_code == 200 or response.raise_for_status() before decoding. Status code not equal to 200 always means that server responded with error and response will not contain json. Here is List of HTTP Status Codes.

All most common features of requests library (including json parsing) are described in short documentation here.