We've all done it:
pip install numpy
# run into permissions issues
sudo pip install numpy # or "sudo !!" for the power users ;)
~~~{% endraw %}
End of story, it works, it's what is written in the README so what's wrong with it?
You see, when you install a library through pip, you are executing {% raw %}`setup.py`{% endraw %} with root permissions. This file could harbor malicious code. You are also messing with the system libraries and that invariably leads to issues down the line.
Relevant XKCD (thx u/truh):
![xkcd](https://imgs.xkcd.com/comics/python_environment.png)
A much better and secure way would be:{% raw %}
~~~bash
pip install --user numpy # libs will be installed in ~/.local/lib
~~~{% endraw %}
This is better, and can be used for installing applications, but it doesn't solve the problem of having different versions needs for different python projects. Enter [pipenv](https://pipenv.readthedocs.io/en/latest/). {% raw %}`pipenv` is to python what `composer` is to PHP. It lets you **easily** install and use libraries per project. It's not the only tool allowing you to do that, but it's the one I use so it's the one I'm gonna present you. Example:
~~~bash
pipenv install numpy matplotlib pandas
# to start your program
pipenv run ./crunch-data.py
# to install libs from another machine, after a git pull:
pipenv sync
# to get a shell in the env (like `source myenv/bin/activate` for venv)
pipenv shell
~~~
This allows a very reproducible environment for your program, without resorting to Docker and without messing up user or system libraries. Save yourself from future bugs and start using pipenv, venv, conda or virtualenv right away! It's much better than {% raw %}`requirements.txt`{% endraw %} + pip. :)
Cheers,
~Nico
Top comments (21)
sudo pip is for global packages, virtualenv for literally everything else. Neither venv or pipenv will prevent you getting pwnd if you're careless enough to install a malicious package. They are not effective security measures.
Using
sudo pip
on distros that provide native Python packages (apt-get install python-numpy
, etc.) is calling for trouble.Fair enough. I do prefer the native package manager route when it's an option, but in those cases it's generally handled automatically as a dependency anyways. Really, 99.99% of all manual pip interactions should be happening in some sandbox env anyways. Regardless it's better practice to understand and respect root ops rather than fear them, because sometimes they are necessary.
When would a
pip install --user
not suffice instead of a global install?We see it when we're installing an application that will be used by another user account, since ~/.local/lib (or the equivalent) isn't shared. This is pretty rare in a development environment, but it comes up frequently in an administered multi-user setup (say a shared workstation or batch cluster). Sometimes service accounts as well depending on what they're doing.
After running pip with sudo in my arch Linux I messed-up some of my system files. And now the whole system is broken.
Or use conda which solves this problem for Python plus many other languages and arbitrary binaries.
but conda is not per project, is it?
You can create a specific
conda
environment for each proyect, and even specify theconda channel
from which it is installed.I usually create an
environment.yml
file for this, so I only runto create the
conda env
.After this I only do:
(Depending on your Anaconda version, to activate/deactivate the env, you can use
source
instead ofconda
.)Good to know thx.
What about for pip install --upgrade pip
"ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/bin/pip3'
"
How do you get pip into /usr/bin without running it privileged?
You don't, that's the whole point. Unless you give your user permissions to write in /usr/bin, which is not a good idea.
should we install pip itself with sudo?
eg: sudo apt install python3-pip
and then ,
pip install --upgrade pip
is that corrent process?
Here's a simple virtualenv manager that you can use to install Python apps in a safe way: github.com/dvershinin/pip-safe You can then just type
pip-safe install <pypy-name>
and it will take care of installing stuff and making it available on PATH. (you can use it instead of typing "pip install" and breaking things :)I also prefer
pipenv
where I can.But it hasn't worked 100% of the times :(
So had to "fall back to" manually creating
virtualenv
followed bypip install
I use pipenv, works great!
Yes yes but how to reset the nightmare that is my current python environment(s)? Honestly I'd be happy resetting my entire Dev ecosystem (but would like to avoid reformatting). MacOS of course.
Simple: install a real OS like GNU/Linux or Free/OpenBSD on your mac and you're good to go :p
No seriously macos is fucked up in various ways… But if you use pipenv or similar, you don't really have to worry about what is installed on the rest of the computer.
Right, I do use virtual environments these days, kinda... but ... my mac is a barren wasteland of python installs and duplicate packages and all sorts of other mind-boggling environment fuckery that I have been putting off dealing with! Ill get to it one of these days, I guess.
But venv.
how do yu specify which environment to use?