GitLab
An example pipeline for a Python application
# A basic GitLab pipeline for a Python application.
# Copyright 2021 Viacheslav Kolupaev
# https://viacheslavkolupaev.ru/
1. Global config
Configures pipeline behavior.
Keyword reference for the .gitlab-ci.yml file
1.1. default
1.2. include
1.3. stages
1.4. workflow
workflow:
rules:
# Skip branch pipelines when MR pipelines already exist.
- if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
when: never
# All merge requests: (`feature`|`bugfix`|`infra`) → `main`.
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" &&
$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^(feature|bugfix|infra)\/.*$/ &&
$CI_MERGE_REQUEST_TARGET_BRANCH_NAME == $CI_DEFAULT_BRANCH'
# All merge commits to the `main` branch.
# Fires after feature has been merged into the `main` branch.
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
1.4. variables
variables:
# This magic feature allows you to enable inline display of coverage for codes by tests.
coverage_report_view: "true"
2. Job config
2.1. Base Python image with venv installed
Developers generally do not use the python:latest image due to possible compatibility issues with project
dependencies.
You must specify a specific version of Python. The image value must match what is specified in the Dockerfile at
the root of the project.
.base_python_image_w_venv:
image: "python:3.9.9-slim"
before_script:
- pip install virtualenv
- virtualenv venv
- source venv/bin/activate
2.2 Cache configuration for Python
.python_caching_params:
variables:
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
cache:
key: $CI_COMMIT_REF_SLUG
paths:
- .cache/pip
- venv/
2.3. Jobs
2.3.1. notify-deploy-freeze
Set up a rule to apply feature Deploy freezes for stage Deploy tasks.
Schedule example: 16:00 on Thu — 09:00 on Mon, [UTC 3] MSK.
If you need to manually restart the pipeline for the main branch again, go to the following path:
repo > Pipelines > Run pipeline > main.
notify-deploy-freeze:
image: busybox:latest
stage: ".pre"
environment:
name: staging
variables:
GIT_STRATEGY: none
GIT_CHECKOUT: "false"
script:
- echo "Deploy freeze is active now. Schedule 16:00 on Thu — 09:00 on Mon, [UTC 3] MSK."
- exit 1 # Mark the task as erroneous to prevent the pipeline from continuing.
# To disable artifact passing.
# Documentation: https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html
dependencies: []
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" &&
$CI_DEPLOY_FREEZE != null'
when: always
allow_failure: true # Notify only for detached pipeline.
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH &&
$CI_DEPLOY_FREEZE != null'
when: always
allow_failure: false # Notify and block further work of the pipeline.
2.3.2. unit-test via pytest
unit-test:
extends:
- .base_python_image_w_venv
- .python_caching_params
stage: test
script:
# A separate file with the necessary dependencies has been prepared in the project.
- pip install --requirement requirements/out/unit_test.txt
# The parameters for running `pytest` are specified in the config file.
- pytest
dependencies: []
artifacts:
when: always
paths:
- ./coverage.xml # Publishing a report on the degree of code coverage by tests.
reports:
cobertura: coverage.xml
junit: report.xml # Publishing a report on the execution of specific unit tests.
rules:
# See documentation: https://docs.gitlab.com/ee/topics/gitlab_flow.html#testing-before-merging
# Testing BEFORE merging branches.
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: on_success
allow_failure: false # If unit tests fail, branch merging is prohibited.
# Testing AFTER merging branches.
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
when: on_success
allow_failure: false # If unit tests fail, deploy to staging is prohibited.
2.3.3. type-test via mypy
type-test:
extends:
- .base_python_image_w_venv
- .python_caching_params
stage: test
script:
# A separate file with the necessary dependencies has been prepared in the project.
- pip install --requirement requirements/out/type_test.txt
# The parameters for running `mypy` are specified in the config file.
- mypy
dependencies: []
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: on_success
# Completion of type testing with an error now does not prevent further work of the pipeline.
# In the future, we will set up a ban on merging branches with such errors.
allow_failure: true
2.3.4. lint-test via flake8
lint-test:
extends:
- .base_python_image_w_venv
- .python_caching_params
stage: test
script:
# A separate file with the necessary dependencies has been prepared in the project.
- pip install --requirement requirements/out/lint_test.txt
# The parameters for running `flake8` are specified in the config file.
- flake8
dependencies: []
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: on_success
# Completion of lint testing with an error now does not prevent further work of the pipeline.
# In the future, we will set up a ban on merging branches with such errors.
allow_failure: true
2.3.5. Auto-documenting code via mkdocs and GitLab pages
pages:
# Do not rename.
extends:
- .base_python_image_w_venv
- .python_caching_params
stage: build
script:
# A separate file with the necessary dependencies has been prepared in the project.
- pip install --requirement requirements/out/docs.txt
# The parameters for running `mkdocs` are specified in the config file.
- mkdocs build --site-dir public
artifacts:
paths:
- public
dependencies: []
rules:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH &&
$CI_DEPLOY_FREEZE == null'
when: on_success
allow_failure: false # Do not turn off this job. In the event of a breakdown, it needs to be repaired.
2.3.6. deploy-to-staging
deploy-to-staging:
extends:
- .base_python_image_w_venv
- .python_caching_params
stage: deploy
environment:
name: staging
script:
- echo "Deploy to staging..."
rules:
# Whether or not to deploy to staging before merging branches. Off now.
# - if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_DEPLOY_FREEZE == null'
# when: manual
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH &&
$CI_DEPLOY_FREEZE == null'
Project Settings
General
Merge requests
Merge commit message template
Merge branch `%{source_branch}` into `%{target_branch}`
%{title}
%{issues}
See merge requests %{reference}
Badges
Pipeline status
- Name:
Pipeline status - Link: https://example.gitlab.com/%{project_path}/-/commits/%{default_branch}
- Badge image URL: https://example.gitlab.com/%{project_path}/badges/%{default_branch}/pipeline.svg
Coverage report
- Name:
Coverage report - Link: https://example.gitlab.com/%{project_path}/-/commits/%{default_branch}
- Badge image URL: https://example.gitlab.com/%{project_path}/badges/%{default_branch}/pipeline.svg
Repository
Push rules
Require expression in commit messages
/^((([A-Z][a-z]{2,}|WIP:))( [A-Za-z0-9_`'"]{2,})+(\. |\.?\n{1,2}))+((Fixes|Closes|Relates) )?(jira|JIRA)\-\d+\.?$/gm
Branch name
CI/CD
General pipelines
Test coverage parsing
When to test your code.
Testing before merging. In old workflows, the continuous integration (CI) server commonly ran tests on the main
branch only. Developers had to ensure their code did not break the main branch. When using GitLab flow, developers
create their branches from this main branch, so it is essential that it never breaks. Therefore, each merge
request must be tested before it is accepted.
There is one drawback to testing merge requests: the CI server only tests the feature branch itself, not the
merged result. Ideally, the server could also test the main branch after each change. However, retesting on every
commit to main is computationally expensive and means you are more frequently waiting for test results. Because
feature branches should be short-lived, testing just the branch is an acceptable risk. If new commits in main
cause merge conflicts with the feature branch, merge main back into the branch to make the CI server re-run the
tests. As said before, if you often have feature branches that last for more than a few days, you should make your
issues smaller.