One of the major advantages of GitLab for the LAVA Software Community
Project is the ability to run a wide range of CI operations in a
variety of situations. LAVA Software has always tried to run the unit
tests on every code change prior to merge. GitLab has allowed this to
be extended to static code analysis, packaging checks and building of
custom and official docker images. These tasks are specified as jobs in
GitLab. Jobs are grouped into Stages and Stages are grouped into
Pipelines, Stages and Jobs in GitLab
Pipelines are typically initiated when a change is pushed to a branch.
Note that the CI will start to run as soon as a branch is pushed,
even if a merge request has not yet been created for that branch.
Jobs are grouped into Stages, for example test, build, publish.
Each stage must complete before jobs in the next stage will start.
Stages are ordered by the stages list in the .gitlab-ci.yml file.
All jobs in each stage will attempt to start simultaneously.
There needs to be a balance of the number of stages, the number of jobs
in each stage and the available resources.
Pipelines can also be initiated after a change is merged (i.e. when the
master branch is updated) and when tags are pushed. GitLab allows
control over which jobs are run for which pipelines. Some jobs can be
set to only run on merge requests (skipping master and tags) or
only on master and/or only on tags.
Finally, pipelines can be initiated on a schedule. GitLab calls these
environments. These are designed to deploy tested software to a
public location but can also be used to run other scheduled operations.
Only environments are able to run jobs outside of a docker container
by being configured for the deploy stage.
Control and scripting
GitLab CI is controlled by the .gitlab-ci.yml file in the top
directory of the source code repository. This file defines the stages
and jobs within each CI pipeline. There are likely to be a lot of
changes within this file as your CI develops. It is YAML syntax, so
use comments liberally to describe each part of the file.
Each stage defines a series of jobs. The name of each job needs to
match an executable script in the same path as the job name below
the .gitlab-ci/ directory.
dispatcher-debian-9 in the tests stage runs a script at
Jobs can also use templates to allow re-use of control blocks. Think of
the templates as a YAML dictionary - whatever occurs later will replace
values set earlier. Specifically, a value set in the template can be
modified in a job using that template just by repeating that part of
the control block and setting a new value.
Templates start with a dot in the name and provide a reference which
will be used by jobs based on that template:
Other templates can be based on templates themselves:
This can lead to a job control definition of just a single line:
The .gitlab-ci/test/dispatcher-debian-9.sh then determines how this
job will differ from other jobs using the same template.
Dependencies and Artifacts
GitLab will run most jobs in a temporary docker container which gets
discarded at the end of the job. To store files for use outside the
container, the job needs to define artifacts.
Other jobs can then use these artifacts by expressing a dependency on
the job which builds those artifacts.
For example, a job which builds a set of binary packages from the
source code may define a set of artifacts using a wildcard:
A job which deploys those binary packages to a public apt repository
would define a dependency on the job providing the artifacts:
Some artifacts have special meaning to GitLab and are used to create
reports which are available in every merge request. Check the GitLab
documentation for more details. For example, the LAVA Software
Community Project uses bandit to do static code analysis for security
Note - this is an example of a job which executes the docker
program inside the docker image used by GitLab. The sast job uses the
dind template (DinD stands for Docker in Docker). This is useful
when building docker images or using prebuilt docker images for
Jobs are assigned to runners which are instances of gitlab-runner,
configured to work with particular projects in GitLab using secure
tokens. Runners can be restricted to particular projects and can use