Pipenv: Python's Official Package Management tool

Introduction

Pipenv is Python's officially recommended Package management tool, as the name suggests it combines the power of pip and virtualenv. It is really powerful and gives control of package management for a python project. It also has support for adding dev packages, python versions, smartly recognizing main and derived package installation/uninstallation, and so on. It is the official package management tool for Python.

It is quite similar to npm for Nodejs in Javascript, or bundle for Ruby, cargo for Rust, and so on. It really simple and easy to use as it manages the overhead for package management for us and hence it is also a high-level package management tool as opposed to pip, which is not as powerful as Pipenv. So, in this article, we'll explore Pipenv package manager for Python and how you can use it in your next python project. Let's get started.

Install Pipenv

Pipenv is just like any other package in python, you can install it with pip as normally you install any other package with the command:

pip install pipenv

You can refer to the documentation of Pipenv from here.

Set up a Python Environment

This step is not mandatory but it avoids any mistakes that you can make in the future while installing the package, so to simply create a new python environment for your project, you simply have to write the following command:

pipenv shell

This will create the virtual environment for Python in the local folder. It is quite similar to installing virtualenv and then activating the env/venv folder script. Though pipenv is quite powerful as it automatically detects the environment.

If you look at it carefully, there will be a file generated after the command has been executed successfully. The file called Pipfile without any extension will have been created in your current folder where you executed the command from. The file contains the version of python used in this project along with the list of dependencies(currently empty). Also the source from where it will download and manage the dependencies.

The Pipfile after simply creating the virtualenv via the command pipenv shell looks something like follows:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]

[dev-packages]

[requires]
python_version = "3.8"

If you want to set up a specific version of python for the virtual environment, you can do this using pipenv. The version should be installed in your system though for the pipenv to pick it up, if you have the specific version of python set up with all path variables configured, you can use the below command to set up the version of python in pipenv.

pipenv --python version-number

For example : pipenv --python 3.9 will set up the virtual environment with python version 3.9.

What is Pipfile

Pipfile is basically a TOML file that has all the details of the different dependencies/packages and the version of Python used in the project/directory. A TOML is a simple configuration file that is reader-friendly, it is a map of keys and values as configuration data structures. In Pipenv, we can have keys like package-names and the value as the version-number, certain groups of dependencies like dev-packages and packages, etc. Pipenv is the file that Pipenv implements its package management environment. The file is really important and powerful as it can install all dependencies even if provided with requirements.txt file. Yes, it can automatically detect that if you provide the path to that file.

Pipenv also has additional features like adding dev dependencies/packages in the project as a separate dependency. So if you want to test a feature with a specific package you can add it as a dev package and it will be stored separately. The pipfile will segregate the dependencies so that Pipenv can install/uninstall from the virtual environment. In short, Pipfile lets you have great control over your project's packages management.

Installing Python Packages

Once your Pipenv is initialized as a virtual environment for Python, we can install dependencies with either pip or pipenv. This is the mistake that might get you trapped, if you already have not run the command pipenv shell and installed any dependencies with pip, you will install that dependency globally and make some mess of the project. So, it is advised to use pipenv shell in order to activate your virtual environment. If you do not wanna do that, you can use the command to install any dependency without activating the virtual environment.

pipenv install <package-name>

This will detect any virtual environment in the project, if it's not created already it will create it and install that package in that environment.

Installing any package using pipenv will update the Pipenv file and the package to its packages list.

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
flask = "*"

[dev-packages]

[requires]
python_version = "3.8"

OR

If you wish to install with pip, as usual, you need to be in the virtual subshell. If you already are, then Pipenv will add that dependency to the virtual environment. Note though, if you install any package with pip and not with pipenv, the package won't be added to Pipfile but would be installed in the virtual environment.

Here, if we type the command pipenv graph it will show us a nice detailed list of all the installed dependencies.

pipenv graph

$ pipenv graph
Flask==2.0.3
  - click [required: >=7.1.2, installed: 8.0.4]
    - colorama [required: Any, installed: 0.4.4]
  - itsdangerous [required: >=2.0, installed: 2.1.1]
  - Jinja2 [required: >=3.0, installed: 3.0.3]
    - MarkupSafe [required: >=2.0, installed: 2.1.0]
  - Werkzeug [required: >=2.0, installed: 2.0.3]
requests==2.27.1
  - certifi [required: >=2017.4.17, installed: 2021.10.8]
  - charset-normalizer [required: ~=2.0.0, installed: 2.0.12]
  - idna [required: >=2.5,<4, installed: 3.3]
  - urllib3 [required: >=1.21.1,<1.27, installed: 1.26.8]

As you might have guessed, the above command is equivalent to the pip freeze command, but just compare the details both tools have. Pipenv really shines here.

If you compare the output of Pipfile and pipenv graph you get a bit confused as to why is there are so fewer packages in the file. So, Pipfile doesn't store the sub-packages/dependencies of a certain base package. Let's take, for example, Flask here, we have Flask as the main package, and click, Jinja2, Werkzeug are its sub dependencies, further colorama and MarkupSafe are in turn dependencies of click and Jinja2. So, Pipfile just includes the top-level packages, pipenv is smart enough to install the rest. It also doesn't include requests but it is indeed included in the graph. This is because Pipenv will only put the dependencies in the Pipfile if it has been installed via the pipenv command.

You can use pip freeze here as well as we are technically in a Python virtual environment. And you can clearly see the difference. Pipenv is a high-level tool compared to pip, it technically uses pip and virtualenv and leverages both of them to become a smart package management tool.

What is the Pipfile.lock

• If you are coming from Javascript world, it is similar to package-lock.json file.
• If you are coming from Ruby world, it is similar to the Gemfile.lock file.
• If you are coming from Rust world, it is similar to the cargo.lock file.

Ok, you get the idea, it is a file that more robustly specifies the version of the packages without conflicting with the other version or the Python version itself. If you look at the Pipfile.lock also has hashes that store the sub-packages as well. The file format here is JSON as opposed to TOML for the Pipfile.

Configuring the Pipenv environment

Now, a question you might have is where is the virtual environment? Is it there? Of course, it will be there, it is configured to a different location though. By default, it will be stored in the ~\.virtualenvs\ folder.

You can get the location of the current virtualenv with the following command:

pipenv --venv

$ pipenv --venv
C:\Users\acer\.virtualenvs\pipenv-blog-gHY6vF9t

For Windows, it is in the Admin user(in my case it is named acer) followed by the hidden folder virtualenvs, this folder will contain all the virtualenvs for different projects using pipenv.

If you wished to change this location and keep the virtual environment folder in the same directory as your project, you can set up the environment variable for it as follows:

For Linux/macOS:
Add the following to your ~/.bashrc or other equivalent shell configuration file.

export PIPENV_VENV_IN_PROJECT="enabled"

For Windows:

Add it to the PATH Environment variable.

set PIPENV_VENV_IN_PROJECT="enabled"   

This will make sure the virtualenvs for the project in pipenv are created inside the current folder itself and not in a single ~\.virtualenvs\ folder.

Creating the requirements.txt file

Let's say you also want to create a requirements.txt file for distributing your project to someone else, as not everyone will use Pipenv to manage their dependencies. It is really straightforward and quick to create a requirements.txt file from the Pipenv environment.

The below command will make the requirements.txt file from the existing Pipenv project.

pipenv lock -r >requirements.txt

This will create the requirements.txt file, based on the Pipfile. Pipenv is smart again to provide all of the required dependencies to the requirements.txt in order that pip will be able to install all the required dependencies.

Using requirements.txt in Pipenv

We can install all the dependencies from the requirements.txt file while we are migrating from bare-bones virtualenv and pip to Pipenv. Pipenv will install all the mentioned dependencies and it will also add its checks for the appropriate checks for dependencies.

pipenv install -r requirements.txt

This will install the dependencies mentioned in the requirements.txt file into the Pipenv virtual environment.

Managing Dev Packages

Let's say we want to install a package but we are not sure to use it in production or the actual script, just a test for how it will work. Thus we have dev packages to install optional or testing packages.

To install a dev-dependency or package, you can install via the following command:

pipenv install <package-name> --dev

If we see the Pipfile, the django package that we installed tagged with --dev will be in the dev-packages

$ cat Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
flask = "*"

[dev-packages]
django = "*"

[requires]
python_version = "3.8"

If we wanted to uninstall all the dev-packages, we can simply enter the command:

pipenv uninstall --all-dev

Pipenv uninstall devpackages

This will simply uninstall all the dev dependencies/packages from the pipenv environment and also remove them from the packages list in Pipfile. If you wished to uninstall a specific package in pipenv, you can uninstall it by the simple command:

pipenv uninstall <package-name>

`

Installing/Uninstalling all packages

To install only the default packages and not dev-packages.


pipenv install


To install or configure a project, if you want to test the project with all the dev dependencies, you can install them with:


pipenv install --dev


This will install all the packages both packages and dev-packages.

If you want to uninstall all the packages in pipenv, you can use the command :


pipenv uninstall --all


This will uninstall all the default and dev packages from pipenv. This is like starting a fresh virtual environment.

References:

• Pipenv documentation
• Real Python - Pipenv Guide

Conclusion

So, this was a brief introduction to pipenv which is Python's recommended package management tool. We saw that we have control over the dependencies and packages in a virtual environment which is taken care of automatically for us. We don't have to activate the environment to install or manage any package for a project.