Перейти к содержанию

GitLab

An example pipeline for a Python application

# A basic GitLab pipeline for a Python application.

# Copyright 2021 Viacheslav Kolupaev
# https://viacheslavkolupaev.ru/

1. Global config

Configures pipeline behavior.

Keyword reference for the .gitlab-ci.yml file

1.1. default

Documentation

default:
  tags:
    - specify_the_correct_runner_tag

1.2. include

Documentation

include:
  - project: 'my-group/my-project'
    ref: main
    file: '/templates/.gitlab-ci-template.yml'

1.3. stages

Documentation

stages:
  - ".pre"
  - test
  - build
  - deploy
  - ".post"

1.4. workflow

Documentation

workflow:
  rules:
    # Skip branch pipelines when MR pipelines already exist.
    - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
      when: never

      # All merge requests: (`feature`|`bugfix`|`infra`) → `main`.
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event" &&
           $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ /^(feature|bugfix|infra)\/.*$/ &&
           $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == $CI_DEFAULT_BRANCH'

    # All merge commits to the `main` branch.
    # Fires after feature has been merged into the `main` branch.
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'

1.4. variables

Documentation

variables:
  # This magic feature allows you to enable inline display of coverage for codes by tests.
  coverage_report_view: "true"

2. Job config

2.1. Base Python image with venv installed

Developers generally do not use the python:latest image due to possible compatibility issues with project dependencies.

You must specify a specific version of Python. The image value must match what is specified in the Dockerfile at the root of the project.

.base_python_image_w_venv:
  image: "python:3.9.9-slim"
  before_script:
    - pip install virtualenv
    - virtualenv venv
    - source venv/bin/activate

2.2 Cache configuration for Python

Documentation

.python_caching_params:
  variables:
    PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
  cache:
    key: $CI_COMMIT_REF_SLUG
    paths:
      - .cache/pip
      - venv/

2.3. Jobs

2.3.1. notify-deploy-freeze

Documentation

Set up a rule to apply feature Deploy freezes for stage Deploy tasks.

Schedule example: 16:00 on Thu — 09:00 on Mon, [UTC 3] MSK.

If you need to manually restart the pipeline for the main branch again, go to the following path: repo > Pipelines > Run pipeline > main.

notify-deploy-freeze:

  image: busybox:latest
  stage: ".pre"
  environment:
    name: staging
  variables:
    GIT_STRATEGY: none
    GIT_CHECKOUT: "false"
  script:
    - echo "Deploy freeze is active now. Schedule 16:00 on Thu — 09:00 on Mon, [UTC 3] MSK."
    - exit 1  # Mark the task as erroneous to prevent the pipeline from continuing.
  # To disable artifact passing.
  # Documentation: https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html
  dependencies: []
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event" &&
           $CI_DEPLOY_FREEZE != null'
      when: always
      allow_failure: true  # Notify only for detached pipeline.
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH &&
           $CI_DEPLOY_FREEZE != null'
      when: always
      allow_failure: false  # Notify and block further work of the pipeline.
2.3.2. unit-test via pytest
unit-test:
  extends:
    - .base_python_image_w_venv
    - .python_caching_params
  stage: test
  script:
    # A separate file with the necessary dependencies has been prepared in the project.
    - pip install --requirement requirements/out/unit_test.txt
    # The parameters for running `pytest` are specified in the config file.
    - pytest
  dependencies: []
  artifacts:
    when: always
    paths:
      - ./coverage.xml  # Publishing a report on the degree of code coverage by tests.
    reports:
      cobertura: coverage.xml
      junit: report.xml  # Publishing a report on the execution of specific unit tests.
  rules:
    # See documentation: https://docs.gitlab.com/ee/topics/gitlab_flow.html#testing-before-merging

    # Testing BEFORE merging branches.
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      when: on_success
      allow_failure: false  # If unit tests fail, branch merging is prohibited.

    # Testing AFTER merging branches.
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
      when: on_success
      allow_failure: false  # If unit tests fail, deploy to staging is prohibited.
2.3.3. type-test via mypy
type-test:
  extends:
    - .base_python_image_w_venv
    - .python_caching_params
  stage: test
  script:
    # A separate file with the necessary dependencies has been prepared in the project.
    - pip install --requirement requirements/out/type_test.txt
    # The parameters for running `mypy` are specified in the config file.
    - mypy
  dependencies: []
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      when: on_success
      # Completion of type testing with an error now does not prevent further work of the pipeline.
      # In the future, we will set up a ban on merging branches with such errors.
      allow_failure: true
2.3.4. lint-test via flake8
lint-test:
  extends:
    - .base_python_image_w_venv
    - .python_caching_params
  stage: test
  script:
    # A separate file with the necessary dependencies has been prepared in the project.
    - pip install --requirement requirements/out/lint_test.txt
    # The parameters for running `flake8` are specified in the config file.
    - flake8
  dependencies: []
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      when: on_success
      # Completion of lint testing with an error now does not prevent further work of the pipeline.
      # In the future, we will set up a ban on merging branches with such errors.
      allow_failure: true
2.3.5. Auto-documenting code via mkdocs and GitLab pages

Documentation

pages:
  # Do not rename.
  extends:
    - .base_python_image_w_venv
    - .python_caching_params
  stage: build
  script:
    # A separate file with the necessary dependencies has been prepared in the project.
    - pip install --requirement requirements/out/docs.txt
    # The parameters for running `mkdocs` are specified in the config file.
    - mkdocs build --site-dir public
  artifacts:
    paths:
      - public
  dependencies: []
  rules:
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH &&
           $CI_DEPLOY_FREEZE == null'
      when: on_success
      allow_failure: false  # Do not turn off this job. In the event of a breakdown, it needs to be repaired.
2.3.6. deploy-to-staging
deploy-to-staging:
  extends:
    - .base_python_image_w_venv
    - .python_caching_params
  stage: deploy
  environment:
    name: staging
  script:
    - echo "Deploy to staging..."
  rules:
    # Whether or not to deploy to staging before merging branches. Off now.
    #    - if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_DEPLOY_FREEZE == null'
    #      when: manual
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH &&
           $CI_DEPLOY_FREEZE == null'

Project Settings

General

Merge requests

Merge commit message template
Merge branch `%{source_branch}` into `%{target_branch}`

%{title}

%{issues}

See merge requests %{reference}

Badges

Pipeline status

Coverage report

Repository

Push rules

Require expression in commit messages
/^((([A-Z][a-z]{2,}|WIP:))( [A-Za-z0-9_`'"]{2,})+(\. |\.?\n{1,2}))+((Fixes|Closes|Relates) )?(jira|JIRA)\-\d+\.?$/gm
Branch name
^(feature|bugfix|infra)\/(jira|JIRA)\-\d+$

CI/CD

General pipelines

Test coverage parsing
^TOTAL.+?(\d+\%)$

When to test your code.

Testing before merging. In old workflows, the continuous integration (CI) server commonly ran tests on the main branch only. Developers had to ensure their code did not break the main branch. When using GitLab flow, developers create their branches from this main branch, so it is essential that it never breaks. Therefore, each merge request must be tested before it is accepted.

There is one drawback to testing merge requests: the CI server only tests the feature branch itself, not the merged result. Ideally, the server could also test the main branch after each change. However, retesting on every commit to main is computationally expensive and means you are more frequently waiting for test results. Because feature branches should be short-lived, testing just the branch is an acceptable risk. If new commits in main cause merge conflicts with the feature branch, merge main back into the branch to make the CI server re-run the tests. As said before, if you often have feature branches that last for more than a few days, you should make your issues smaller.