Error installing mrjob on Mac (OS X 10.11.1)

Question

Typing in Terminal

pip install mrjob

gives the error message: "NameError: name 'execfile' is not defined" and "Command "python setup.py egg_info" failed with error code 1 in /private..."

Using

sudo pip install mrjob

also gives the same error message. In addition, it gives this message:

Password: The directory '/Users/.../Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. The directory '/Users/.../Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

How do I successfully install?

Answer 1

Between March 2015 and May 2015, the mrjob setup.py used the Python-2 only built-in function execfile, which was removed from the Python 3 built-in functions.

The NameError you encountered strongly suggests that the problem was due to using the Python 3 interpreter.

The project had a similar bug raised in September 2015, and the fixed setup.py had not yet been released at that time. It appears that this bug was fixed in v0.5.0, released 28 March 2016.

To install, use a version criteria:

pip install --user "mrjob>=5"

Also probably upgrading pip might solve the problems you mention wrt sudo, but that seems like an unrelated problem best addressed in a separate question.