unsplash-logoScott Umstattd

I have used a couple of dev env on python.

1. System-provided Python (Mac)
   As many of you know, the version is too outdated right now. When I used it, it was fine but at that time I wanted to use python3 so I didn't use it for a long time.

2. Pyenv
   Pyenv allows us to install multiple versions of python which is very useful, but when I used it, I didn't use Virtualenv, so my dev envs were very messy.

3. Anaconda
   Then, I started learning about machine learning and one ITP resident recommended us to use Anaconda since it's very easy to create a virtual environment and install some specific modules/packages.
   found out this article(https://jacobian.org/2019/nov/11/python-environment-2020/) that is from Jacob Kaplan-Moss, co-creator of Django.

4. Pyenv + Virtualenv
   When I used 3, I also used Pyenv, but sometimes that made me confused and also Anaconda had some issues when I run flask. Eventually, I uninstalled Anaconda and started using Pyenv and Virtualenv. Actually, I'm still using this. But, the issue is that I heavily use a specific virtual env and it's very messy so it's about to ruin my python env lol

5. Pipenv
   The team I joined used python and the lead engineer recommended to use pipenv But, pipenv is slow and has caused some issues when I update
   modules/packages. I end up using 4 for other things/projects.

Recently, I read the article on python dev env that is written in Japanese lol. The article mentioned 3 three things, Pipenv, Poetry, and Pyflow. It says that Poetry could be a better choice. After reading that, I researched about Poetry a little bit. Then, I found out this article(https://jacobian.org/2019/nov/11/python-environment-2020/) that is from Jacob Kaplan-Moss, co-creator of Django.

Poetry
https://python-poetry.org/

Why do I need to use Poetry?

Before introducing the install steps, I would like to mention the reason why I want to switch Pyenv + Virtualenv to Poetry. The biggest reason is that I can share a project easily.
When I was an ITP resident and led a python workshop, I spent so much time to create an environment for sample codes since most students didn't follow guides I shared before the workshop lol.
Also, there are a few reasons.

What Poetry can do is to allow me to manage python projects like js/nodejs projects which are managed by yarn/npm. This feature enables me to discard a project easily(I will be able to delete the project.) when I don't need it. I can use virtual env with Poetry.
In addition, Poetry has commands that build a package and publish it to PyPI. This is also good for me since I have published a couple of packages to PyPI. Plus Poetry has an export command for requirements.txt.

https://python-poetry.org/docs/cli/#export

I decided to set up Poetry on my Ubuntu since Mac's environment will take some time to switch and I noticed that I haven't used Pyenv on my Ubuntu. (I don't know why I haven't used it.)

In this article, I will share steps to set up Poetry on my Ubuntu.
The env will be Pyenv + Poetry + Pipx.
This setup is very similar to my js dev env. (nvm + yarn)

Step1. Install Pyenv
Step2. Install Pipx for python CLI tool (especially, youtube-dl)
Step3. Install Poetry
Step4. Create a sample project with Poetry
Step5. Allow making venv in a project

Step1 Install Pyenv

There are a couple of ways to install Pyenv on Ubuntu. In this article, I use brew since I'm using brew on Mac and currently we can use brew on Linux too.

https://docs.brew.sh/Homebrew-on-Linux
https://github.com/pyenv/pyenv

$ sh -c "$(curl -fsSL https://raw.githubusercontent.com/Linuxbrew/install/master/install.sh)"

$ brew install pyenv

$ pyenv --version
pyenv 1.2.15

system pythons

$ python --version
Python 2.7.17

$ python3 --version
Python 3.7.5

Let's install python 3.6 and 3.8

$ pyenv install 3.6.5
$ pyenv install 3.8.0
$ pyenv global 3.8.0

Ugh, got BUILD FAILED (Ubuntu 18.04 using python-build 20180424) with 3.6.5...

The solution is the following

sudo apt-get remove libssl-dev
sudo apt-get update
sudo apt-get install libssl1.0-dev

https://github.com/pyenv/pyenv/wiki/Common-build-problems

Now the global Python version is changed by Pyenv

$ python --version
Python 3.8.0

Step2 install pipx

Of course, you can install CLI tools via brew, but this time I'm using pipx to try a new thing lol.

$ pip list
Package    Version
---------- -------
pip        19.2.3 
setuptools 41.2.0 


$ python -m pip install pipx
$ pipx install youtube-dl
$ pipx list
   package youtube-dl 2019.12.25, Python 3.8.0
    - youtube-dl
# Actually the command is a little bit longer, but you can use alias to shorten the command.
$ pipx run youtube-dl url

Step3 install Poetry

Eventually reached the most important part of this post.
The official page is using curl to install Poetry, but this post uses pipx because I need to understand pipx.

I think following the official site is better.

Poetry - Official
https://python-poetry.org/docs/

$ pipx install poetry 

Step4 Create a sample project

$ mkdir hoge && cd hoge
$ pipx run poetry init
or
$ pipx run poetry new hoge

# add package
$ pipx run poetry add ipython
$ pipx run poetry run ipython
Python 3.8.0 (default, Dec 25 2019, 21:53:01) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.10.2 -- An enhanced Interactive Python. Type '?' for help.

In [1]: print('a')                                                              
a

In [2]:      

pyproject.toml

[tool.poetry]
name = "hoge"
version = "0.1.0"
description = ""
authors = ["koji <kk2796@nyu.edu>"]

[tool.poetry.dependencies]
python = "^3.8"
ipython = "^7.10.2"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

export

$ pipx run poetry export -f requirements.txt > requirements.txt

Step5 Allow making venv in a project

The default settings create a virtual environment in a specific place, so this is the same as pyenv + virtualenv that I'd like to avoid. So I need to modify the setting.

$ pipx run poetry config --list
cache-dir = "/home/koji/.cache/pypoetry"
virtualenvs.create = true
virtualenvs.in-project = false <-- chage this
virtualenvs.path = "{cache-dir}/virtualenvs"  # /home/koji/.cache/pypoetry/virtualenvs

$ pipx run poetry config virtualenvs.in-project true
$ pipx run poetry config --list                                          
cache-dir = "/home/koji/.cache/pypoetry"
virtualenvs.create = true
virtualenvs.in-project = true
virtualenvs.path = "{cache-dir}/virtualenvs"  # /home/koji/.cache/pypoetry/virtualenvs

Caution

I just started using Poetry before 2020, so there might be some mistakes.
If you find out the mistake, please let me know!