Reinstall python packages after homebrew [email protected] update

5
votes

I have recently updated python to [email protected] via homebrew.

I then noticed that my python scripts were unable to link to the global python packages that I had previously installed with pip (packages installed globally and not within a virtual environment).

This has sent me down the rabbit hole of understanding how python is set up on macos and I have several questions.

  1. pip installed by homebrew will now install all packages in /usr/local/lib/python3.8/site-packages.

    • Does that mean I need to reinstall all my python packages that used to live (and still are) in /usr/local/lib/python3.7? Should I remove the latter directory after? (For that matter I can see I have also a /usr/local/lib/python3.6 directory)
    • What's the efficient way of managing this in the future when the next python update comes along? Virtual envs per project (I'm not very familiar with the use of virtual envs)?

  2. When I run brew list | grep python I get the following list

python
python3
[email protected]

I can see [email protected] is correctly symlinked as /usr/local/bin/python3 but I don't know what the python and python3 in the above list are since looking at /usr/local/Cellar I can see that I have an empty /usr/local/Cellar/python directory and a /usr/local/Cellar/python3 directory that is symlinked to it. Can I juste remove these two folders?

  1. Finally, looking into all this, I was surprised to discover a python3 executable living in /usr/bin. Looking into the sys.path when I execute /usr/bin/python3, I can see it's some sort of python installed by Xcode since it's looking for libraries in various Xcode.app directories. Since my /usr/local/bin is at the top of my PATH envt variable, this python3 will never be called but I wonder why I'd need it at all and whether I can just get rid of it.

I suppose the answer to a lot of these questions is to use virtual environments, which I've miraculously avoided doing for so long... Would love some help understanding the above though.

EDIT: Running brew info python or brew info python3 returns the same thing

[email protected]: stable 3.8.3 (bottled)
Interpreted, interactive, object-oriented programming language
https://www.python.org/
/usr/local/Cellar/[email protected]/3.8.3_2 (4,268 files, 65.5MB) *
  Poured from bottle on 2020-07-12 at 23:19:08
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/[email protected]
1
python-3.xmacoshomebrew
This is not related to virtual environments. A new Python installation is a new Python installation, full stop. It does not (and due to binary incompatibility should not!) use packages from another Python installation. You have to reinstall the package that you need. As for cleaning up Python installations – use brew doctor and the various other brew commands. If brew does not remove something (e.g. usr/bin/python3) there is a good chance you should not do so either. – MisterMiyagi

1 Answers

2
votes

I had a similar issue with a system running Python 3.9

I had initially not run the final command (only the --dry-run), as everything seemed fine, but as suggested during the installation, using brew link to clobber and create new symlinks was needed.

% brew link --overwrite --dry-run [email protected]
Would remove:
% brew link --overwrite [email protected]
Linking /usr/local/Cellar/[email protected]/3.9.1_1... 23 symlinks created
%

As suggested by MisterMiyagi in the comments, this was uncovered via brew doctor

...
Warning: You have unlinked kegs in your Cellar.
Leaving kegs unlinked can lead to build-trouble and cause brews that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
  [email protected]