In all the CI/CD tools I've used (Jenkins, TeamCity, Bitbucket Pipelines, Gitlab), I would say Buildkite is my favourite so far 👍. Being a fan of the sayings "right tool for the right problem" and "it doesn't matter what tool you use", it still helps that your DevOps tool offer flexibility with a cleaner and minimalist interface unlike the very cluttered Gitlab 🫣.
In one of its flexible feature, it allows the user with a conditional modal before the pipeline proceeds to the next steps. They call it block step 🧱 in Buildkite.
Not all our automated tests are embedded into our build pipelines. We usually enable happy path smoke tests to make our micro-service pipelines run faster. Hence, we allow users to run ad-hoc/on-demand tests based on their preferences.
We created a standalone test pipeline called "tests"
We set the CI variables / steps with default values then call the pipeline.yml in our code repository:
STAGING_BASE_URL: https://staging.mydomain.net
STAGING_PASSWORD: def456
TEST_BASE_URL: https://test.mydomain.net
TEST_PASSWORD_TEST: abc123
SPECS: tests/**/*.spec.ts # will run the whole test suite
BUILDKITE_PARALLEL_RUN: 1 # default to 1 but can be overriden
ENVIRONMENT: test
steps:
- label: ':pipeline: Initiate pipeline'
command: buildkite-agent pipeline upload .buildkite/pipeline.yml
In our code repository, we have two files, the .buildkite/pipeline.yml which triggers the block modal:
steps:
- block: ':playwright: Run tests on demand'
# branches: '!master'
fields:
- select: ':earth_americas: Select environment'
key: 'environment'
default: 'test'
options:
- label: 'test'
value: 'test'
- label: 'staging'
value: 'staging'
- text: ':test_tube: Test spec path / file (e.g. tests/api/**/*.ts)'
default: 'tests/e2e/**/*.ts'
key: 'specs'
- select: ':man-running: Parallel runs'
key: 'parallelism'
default: '3'
options:
- label: '1'
value: '1'
- label: '2'
value: '2'
- label: '3'
value: '3'
- label: ':buildkite: Trigger tests'
commands:
- chmod 777 .buildkite/pipeline-adhoc.sh
- .buildkite/pipeline-adhoc.sh | buildkite-agent pipeline upload
... and the .buildkite/pipeline-adhoc.sh, which captures the entered values in the block modal and also allows parameterising the parallelism option in Buildkite as the integer value (currently defaulted to 1) can only be updated via a script:
#!/bin/bash
set -e
ENVIRONMENT=$(buildkite-agent meta-data get "environment")
SPECS=$(buildkite-agent meta-data get "specs")
PARALLEL_RUN=$(buildkite-agent meta-data get "parallelism")
if [ "${ENVIRONMENT}" = "test" ]; then
PASSWORD=${TEST_PASSWORD}
BASE_URL=${TEST_BASE_URL}
else
PASSWORD=${STAGING_PASSWORD}
BASE_URL=${STAGING_BASE_URL}
fi
cat <<YAML
env:
ENVIRONMENT: "${ENVIRONMENT}"
SPECS: "${SPECS}"
PASSWORD: "${PASSWORD}"
BASE_URL: "${BASE_URL}"
steps:
- label: ':playwright: Run tests'
if: build.env('SPECS') != null # some components might not have spec files specified so need to skip running tests
commands:
- 'npx playwright test'
parallelism: ${PARALLEL_RUN} # this is defaulted to > 1 so need to set PARALLEL_RUN as a CI variable for repo's with only 1 test spec file so CI doesn't fail with 'no spec files' error
retry:
manual:
allowed: false
reason: 'Integration with other platforms means retries should not be performed for tests.'
plugins: # pass details to Test Analytics
- test-collector#v1.0.0:
files: 'test-results/junit-*.xml'
format: 'junit'
artifact_paths:
- 'results/**/*'
YAML
Once all these are set up, the pipeline can be triggered anytime and will stop on the step with block
Clicking the "Run tests on demand" button would take you to the modal where you can choose your own adventure 🤠 then clicking "Continue" button on the modal would trigger the tests based on your selections ✅
Top comments (0)