Combine all common yaml parts in manifest-pipeline.yml and add
manifest-pipeline-yocto.yml and manifest-pipeline-ci-test.yml
containing the different variable assignments for each environment.

This change implicitly introduces parent-child build job generation in
the ci-test pipeline, like it is done in the yocto pipeline already.
The ci-test build jobs have been moved to build-jobs-ci-test.jinja2
accordingly.

Furthermore rename GITLAB_CI_CURRENT_REV to GITLAB_CI_REVISION and
remove the run conditions from all generated build jobs, since these are
already present in the upstream trigger job.

The repos including these files have to be updated with the new file
and variable names.

The job generation script implicitly passes the OS environment to the
template, so that the template has access to all GitLab CI variables.
Hence there is no need to explicitly pass any of them as command line
arguments.

This change makes the "generate-build-jobs" job more generic, so that
it can be shared with the ci-test pipeline in the future.

This job was accidentally disabled in 2e6c1b4f, because it was included
in the generated child pipeline. This pipeline is not created on the
master, though, so the condition for the job was never met.

Move the job to the parent pipeline again, so that it gets executed on
the master like before.

Was accidentally removed in previous commit.

The yamllint job was included in the pipeline implicitly via the
inclusion of common.yml. This is not very transparent. Make the common
definition a base job and extend it explicitly in each pipeline now.

Add infrastructure stage comment header into both yaml files.

Leave all common yaml parts in manifest-integration.yml and add
manifest-integration-yocto.yml and manifest-integration-ci-test.yml
containing the different variable assignments for each environment.

The repos including these files have to be updated with the new names.

This is the first step for reducing code duplicates.

BCS 746-000807

Costum manifest may override this variable

If the MASTER_BRANCH_MANIFEST variable is set in the manifest's
.gitlab-ci.yml file, it may not find its way to the generated job,
if it is not explicitly set during generation.
That is done with this patch.

* Move all build, deploy and test stubs to the manifest-build.yml
* Create a new job to generate and trigger the build jobs dynamically
* Add the base jinja2 file for the build jobs.
* Add initial docs for the manifest pipeline
* Remove unused files

BCS 746-000808

According to the documentation [1] we should get a 406 error from the
API if we try to merge a merge request which has a merge conflict. We
are experiencing a different behaviour, though, where we are getting a
405 error in this case. There is an open GitLab issue on this topic [2].

In our error handling we assumed that the reason for a 405 error is most
likely a required pipeline which is still running. This assumtion,
however, leads to wrong behaviour in cases where no pipeline is started
at all.

Fixing this now by explicitly checking for merge conflicts at the very
beginning of the error handling sequence.

[1]: https://docs.gitlab.com/ee/api/merge_requests.html#merge-a-merge-request
[2]: https://gitlab.com/gitlab-org/gitlab/-/issues/364102

This makes merging a project MR possible even if the manifest has
received new commits since the MR pipeline build has run. The developer
is responsible for deciding whether it is safe to merge it without
having a build based on the latest manifest.

BCS 746-000794

Our server installation uses port 4444 for ssh access. The lava tests
didn't took this into account.

Prior to this change the integration stop when one of the projects had
already the latest ref of gitlab-ci in to submodule.

Prior to this change the integration stop when one of the
projects had already the latest ref of gitlab-ci in to submodule.

Prior to this change the integration stop when one of the projects had
already the latest ref of gitlab-ci in to submodule.

The script can also be run in the normal python container, so this
slightly reduces the complexity.

The image and distro for the buildfng job can now be set via
global/external variables.

BCS 746-000636

The integration branch was reused even if there where new commits on the
target branch, which could lead into reverting some changes.
This adds a check, if the existing integration branch is directly based
on the target branch and deletes it if not.

BCS 746-000785

The pipeline is not setup to build from different manifest and so on,
and the dual_espresso manifest will probably go to a seperate repo
to keep it private.

BCS 746-000518