python eval vs ast.literal_eval vs JSON decode

16
votes

I am converting 2 MB of data as a string into a dict. The input is serialized in JSON.

Anyways I am currently using ast.literal_eval and I get the dictionary I want, but then when I tried just running eval it seems to run faster, and also returns the same result.

Is there any reason to use the ast module or the json module when eval works just fine?

4
python
eval is prone to security threats. Use only when you absolutely control what gets eval'ed - jldupont
If you want to distribute the dictionary alongside with your code, you can laso simply put it in a Python module and import it. That way, you make clear this is Python code. - Sven Marnach

4 Answers

33
votes

I don't really like this attitude on stackoverflow (and elsewhere) telling people without any context that what they are doing is insecure and they shouldn't do it. Maybe it's just a throwaway script to import some data, in that case why not choose the fastest or most convenient way?

In this case, however, json.loads is not only more secure, but also more than 4x faster (depending on your data). 

In [1]: %timeit json.loads(data)
10000 loops, best of 3: 41.6 µs per loop

In [2]: %timeit eval(data)
10000 loops, best of 3: 194 µs per loop

In [3]: %timeit ast.literal_eval(data)
1000 loops, best of 3: 269 µs per loop

If you think about it makes sense json is a such more constrained language/format than python, so it must be faster to parse with an optimized parser.

25
votes

Yes, there's definitely a reason: eval() is evil. Your code might read untrusted data one day, an this would allow an attacker to run arbitrary code on your machine.

You shouldn't use ast.literal_eval() to decode JSON either. It cannot decode every valid JSON string and is not meant to be used for this purpose. Simply use json.loads(), it's reasonably fast.

16
votes

No. Unless you hit one of two scenarios:

  1. That's not JSON!

    Someone puts __import__('os').system('rm -rf /') in the file instead. You are boned.

  2. It's JSON, but not the Python-like part!

    Someone puts true, false, null, or a Unicode escape somewhere in it. Happy birthday.

2
votes

Not an answer exactly, but it should be noted that eval and literal_eval are not the same thing. The ast.literal_eval won't run arbitrary code.

That said, I agree with using JSON; I just wanted to point out that eval != literal_eval