python - JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Question

I am getting error Expecting value: line 1 column 1 (char 0) when trying to decode JSON.

The URL I use for the API call works fine in the browser, but gives this error when done through a curl request. The following is the code I use for the curl request.

The error happens at return simplejson.loads(response_json)

response_json = self.web_fetch(url)
response_json = response_json.decode('utf-8')
return json.loads(response_json)


def web_fetch(self, url):
    buffer = StringIO()
    curl = pycurl.Curl()
    curl.setopt(curl.URL, url)
    curl.setopt(curl.TIMEOUT, self.timeout)
    curl.setopt(curl.WRITEFUNCTION, buffer.write)
    curl.perform()
    curl.close()
    response = buffer.getvalue().strip()
    return response

Traceback:

File "/Users/nab/Desktop/myenv2/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  111.                         response = callback(request, *callback_args, **callback_kwargs)
File "/Users/nab/Desktop/pricestore/pricemodels/views.py" in view_category
  620.     apicall=api.API().search_parts(category_id= str(categoryofpart.api_id), manufacturer = manufacturer, filter = filters, start=(catpage-1)*20, limit=20, sort_by='[["mpn","asc"]]')
File "/Users/nab/Desktop/pricestore/pricemodels/api.py" in search_parts
  176.         return simplejson.loads(response_json)
File "/Users/nab/Desktop/myenv2/lib/python2.7/site-packages/simplejson/__init__.py" in loads
  455.         return _default_decoder.decode(s)
File "/Users/nab/Desktop/myenv2/lib/python2.7/site-packages/simplejson/decoder.py" in decode
  374.         obj, end = self.raw_decode(s)
File "/Users/nab/Desktop/myenv2/lib/python2.7/site-packages/simplejson/decoder.py" in raw_decode
  393.         return self.scan_once(s, idx=_w(s, idx).end())

Exception Type: JSONDecodeError at /pricemodels/2/dir/
Exception Value: Expecting value: line 1 column 1 (char 0)

Answer 1

To summarize the conversation in the comments:

• There is no need to use simplejson library, the same library is included with Python as the json module.

• There is no need to decode a response from UTF8 to unicode, the simplejson / json .loads() method can handle UTF8 encoded data natively.

• pycurl has a very archaic API. Unless you have a specific requirement for using it, there are better choices.

requests offers the most friendly API, including JSON support. If you can, replace your call with:

import requests

return requests.get(url).json()

Answer 2

Check the response data-body, whether actual data is present and a data-dump appears to be well-formatted.

In most cases your json.loads- JSONDecodeError: Expecting value: line 1 column 1 (char 0) error is due to :

• non-JSON conforming quoting
• XML/HTML output (that is, a string starting with <), or
• incompatible character encoding

Ultimately the error tells you that at the very first position the string already doesn't conform to JSON.

As such, if parsing fails despite having a data-body that looks JSON like at first glance, try replacing the quotes of the data-body:

import sys, json
struct = {}
try:
  try: #try parsing to dict
    dataform = str(response_json).strip("'<>() ").replace('\'', '\"')
    struct = json.loads(dataform)
  except:
    print repr(resonse_json)
    print sys.exc_info()

Note: Quotes within the data must be properly escaped

Answer 3

Be sure to remember to invoke json.loads() on the contents of the file, as opposed to the file path of that JSON:

json_file_path = "/path/to/example.json"

with open(json_file_path, 'r') as j:
     contents = json.loads(j.read())

I think a lot of people are guilty of doing this every once in a while (myself included):

contents = json.loads(json_file_path)

Answer 4

With the requests lib JSONDecodeError can happen when you have an http error code like 404 and try to parse the response as JSON !

You must first check for 200 (OK) or let it raise on error to avoid this case. I wish it failed with a less cryptic error message.

NOTE: as Martijn Pieters stated in the comments servers can respond with JSON in case of errors (it depends on the implementation), so checking the Content-Type header is more reliable.

Answer 5

Check encoding format of your file and use corresponding encoding format while reading file. It will solve your problem.

with open("AB.json", encoding='utf-8', errors='ignore') as json_data:
     data = json.load(json_data, strict=False)

Answer 6

I had the same issue trying to read json files with

json.loads("file.json")

I solved the problem with

with open("file.json", "r") as read_file:
   data = json.load(read_file)

maybe this can help in your case

Answer 7

A lot of times, this will be because the string you're trying to parse is blank:

>>> import json
>>> x = json.loads("")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

You can remedy by checking whether json_string is empty beforehand:

import json

if json_string:
    x = json.loads(json_string)
else:
    # Your code/logic here 
    x = {}

Answer 8

I encounterred the same problem, while print out the json string opened from a json file, found the json string starts with '', which by doing some reserach is due to the file is by default decoded with UTF-8, and by changing encoding to utf-8-sig, the mark out is stripped out and loads json no problem:

open('test.json', encoding='utf-8-sig')

Answer 9

There may be embedded 0's, even after calling decode(). Use replace():

import json
struct = {}
try:
    response_json = response_json.decode('utf-8').replace('\0', '')
    struct = json.loads(response_json)
except:
    print('bad json: ', response_json)
return struct

Answer 10

I was getting this error because my json file was empty.

Answer 11

I had exactly this issue using requests. Thanks to Christophe Roussy for his explanation.

To debug, I used:

response = requests.get(url)
logger.info(type(response))

I was getting a 404 response back from the API.

Answer 12

Just check if the request has a status code 200. So for example:

if status != 200:
    print("An error has occured. [Status code", status, "]")
else:
    data = response.json() #Only convert to Json when status is OK.
    if not data["elements"]:
        print("Empty JSON")
    else:
        "You can extract data here"

Answer 13

I had the same issue, in my case I solved like this:

import json

with open("migrate.json", "rb") as read_file:
   data = json.load(read_file)

Answer 14

I was having the same problem with requests (the python library). It happened to be the accept-encoding header.

It was set this way: 'accept-encoding': 'gzip, deflate, br'

I simply removed it from the request and stopped getting the error.

Answer 15

For me, it was not using authentication in the request.

Answer 16

For me it was server responding with something other than 200 and the response was not json formatted. I ended up doing this before the json parse:

# this is the https request for data in json format
response_json = requests.get() 

# only proceed if I have a 200 response which is saved in status_code
if (response_json.status_code == 200):  
     response = response_json.json() #converting from json to dictionary using json library

Answer 17

I received such an error in a Python-based web API's response .text, but it led me here, so this may help others with a similar issue (it's very difficult to filter response and request issues in a search when using requests..)

Using json.dumps() on the request data arg to create a correctly-escaped string of JSON before POSTing fixed the issue for me

requests.post(url, data=json.dumps(data))

Answer 18

If you are a Windows user, Tweepy API can generate an empty line between data objects. Because of this situation, you can get "JSONDecodeError: Expecting value: line 1 column 1 (char 0)" error. To avoid this error, you can delete empty lines.

For example:

 def on_data(self, data):
        try:
            with open('sentiment.json', 'a', newline='\n') as f:
                f.write(data)
                return True
        except BaseException as e:
            print("Error on_data: %s" % str(e))
        return True

Reference: Twitter stream API gives JSONDecodeError("Expecting value", s, err.value) from None

Answer 19

In my case it is because the server is giving http error occasionally. So basically once in a while my script gets the response like this rahter than the expected response:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<h1>502 Bad Gateway</h1>
<p>The proxy server received an invalid response from an upstream server.<hr/>Powered by Tengine</body>
</html>

Clearly this is not in json format and trying to call .json() will yield JSONDecodeError: Expecting value: line 1 column 1 (char 0)

You can print the exact response that causes this error to better debug. For example if you are using requests and then simply print the .text field (before you call .json()) would do.