Installing modules and Virtual environments

The standard modules are only a tiny fraction of the vast amount of libraries available for Python. The rich third-party ecosystem is one of the reasons why Python is so widely used. You can find plenty of well made modules for web development, finance applications, scientific computing, machine learning, bioinformatics, data science, GUI, games and so on. There are plenty of alternatives for standard modules as well.

Quoting from pypi.org: Python Packaging Index

The Python Package Index (PyPI) is a repository of software for the Python programming language. PyPI helps you find and install software developed and shared by the Python community.

This chapter will discuss how to use pip for installing modules. You'll also see how to create virtual environments using the venv module.

pip

Modern Python versions come with the pip installer program. The below code shows how to install the latest version of a module (along with dependencies, if any) from PyPI. See pip user guide for documentation and other details like how to use pip on Windows.

$ python3.9 -m pip install --user regex
Collecting regex
  Downloading ...
    100% ...
Installing collected packages: regex
Successfully installed regex-2020.11.13

--user option limits the availability of the module to the current user, see packaging.python: Installing to the User Site for more details. Here's an example with regex module that makes use of possessive quantifiers, a feature not yet supported by the re module.

>>> import regex

# numbers >= 100 with optional leading zeros
# same as: r'\b0*[1-9]\d{2,}\b'
>>> regex.findall(r'\b0*+\d{3,}\b', '0501 035 154 12 26 98234')
['0501', '154', '98234']

See packaging.python: Installing from PyPI for details like constraining package version number, upgrading packages, etc.

uninstall instead of install in the above example will remove the package. See also stackoverflow: how to uninstall a package for details and gotchas.

Unless you really know what you are doing, do NOT ever use pip as a root/admin user. Problematic packages are an issue, see Malicious packages found to be typo-squatting and Hunting for Malicious Packages on PyPI for examples. See also security.stackexchange: PyPI security measures.

venv

Virtual environments allow you to work with specific Python and package versions without interfering with other projects. Modern Python versions come with built-in module venv to easily create and manage virtual environments.

The flow I use is summarized below. If you are using an IDE, it will likely have options to create and manage virtual environments.

$ python3.9 -m venv new_project
$ cd new_project/
$ source bin/activate
(new_project) $ # pip install <modules>
(new_project) $ # do some scripting
(new_project) $ deactivate
$ # you're now out of the virtual environment

Here, new_project is the name of the folder containing the virtual environment. If the folder doesn't already exist, a new folder will be created. source bin/activate will enable the virtual environment. Among other things, python will point to the version you used to create the environment, which is python3.9 in the above example. The prompt will change to the name of the folder, which is an easy way to know that you are in an virtual environment (unless your normal prompt is something similar). pip install will be restricted to this environment.

Once you are done with your work, use deactivate command to exit the virtual environment. If you delete the folder, your installed modules in that environment will be lost as well.

See also:

• meribold: Virtual Environments Demystified
• calmcode.io: virtualenv — video
• realpython: Python Virtual Environments Primer
• stackoverflow: What is the difference between venv, pyvenv, pyenv, virtualenv, virtualenvwrapper, pipenv, etc?

Creating your own package

The packaging.python: Packaging Python Projects tutorial walks you through packaging a simple Python project. See also:

• stackoverflow: What is setup.py?
• Practical Python Programming: Packaging and Distribution