How to install a package from a private repository on Poetry for CI

Why?

One of the methods to install a package via poetry is to use deploy tokens directly on the package URL, and that used to work on poetry too before version 1.0.3.

What changed? The git URL parsing has changed significant to be more consistent.

Are they planning on fixing this within poetry? No.
They don't want to expose credentials on the pyproject.toml file, which makes sense.

source: This issue on gitlab

How to do this then?

There are multiple ways, using git credential helpers, setting credentials globally for a particular URL.

The one thing that was easy enough to me was to tell git to replace the URL when calling gitlab via HTTPS.

After obtaining your deploy tokens from gitlab, this is how you tell git to use them

deploy_user=YOUR_DEPLOY_TOKEN_USER
deploy_token=YOU_DEPLOY_TOKEN
git config --global url."https://$deploy_user:$deploy_token@gitlab.com".insteadOf https://gitlab.com

Note this is intended to run in a docker container via CI, so it does not matter that much if you rewrite that URL, I do not recommend you to do this on your local machine, for that use SSH keys.

Which takes me to the next section

Poetry install Locally vs Gitlab CI

I do use SSH to clone mu libraries from gitlab, and the deploy token requires cloning via HTTPS, so now what?

I replace the urls on my before_script, hacky but effective.

For example, let's say you have this library on your pyproject.toml file

testrail-api-wrapper = {git = "git@gitlab.com:mundo03/sample_library.git", rev = "main"}

You can rewrite it using sed like this:

export SSH_URL=git@gitlab\.com:mundo03\/sample_library.git
export TOKEN_URL=https:\\/\\/gitlab.com\\/mundo03\/sample_library.git
sed -i "s|$SSH_URL|$TOKEN_URL|g" pyproject.toml

your .gitlab-ci.yml file would look like this, given you want to install you project and run pytest

stages:
  - test

.build:
  before_script:
      - pip install poetry
      - poetry config virtualenvs.in-project false #Ignore venv
      - export SSH_URL=git@gitlab\.com:mundo03\/sample_library.git
      - export TOKEN_URL=https:\\/\\/gitlab.com\\/mundo03\/sample_library.git
      - sed -i "s|$SSH_URL|$TOKEN_URL|g" pyproject.toml
      - git config --global url."https://$deploy_user:$deploy_token@gitlab.com".insteadOf https://gitlab.com
      - poetry update
      - poetry install -v

pytest_311:
  image: python:3.11
  extends: .build
  stage: test
  script:
    - poetry run pytest --cov
  retry: 2
  coverage: '/TOTAL.*\s+(\d+%)$/'

Conclusion

Poetry did me dirty and I had to find another way to install dependencies in CI that come from private git repositories.