This is the first step for reducing code duplicates.

BCS 746-000807

Costum manifest may override this variable

If the MASTER_BRANCH_MANIFEST variable is set in the manifest's
.gitlab-ci.yml file, it may not find its way to the generated job,
if it is not explicitly set during generation.
That is done with this patch.

* Move all build, deploy and test stubs to the manifest-build.yml
* Create a new job to generate and trigger the build jobs dynamically
* Add the base jinja2 file for the build jobs.
* Add initial docs for the manifest pipeline
* Remove unused files

BCS 746-000808

According to the documentation [1] we should get a 406 error from the
API if we try to merge a merge request which has a merge conflict. We
are experiencing a different behaviour, though, where we are getting a
405 error in this case. There is an open GitLab issue on this topic [2].

In our error handling we assumed that the reason for a 405 error is most
likely a required pipeline which is still running. This assumtion,
however, leads to wrong behaviour in cases where no pipeline is started
at all.

Fixing this now by explicitly checking for merge conflicts at the very
beginning of the error handling sequence.

[1]: https://docs.gitlab.com/ee/api/merge_requests.html#merge-a-merge-request
[2]: https://gitlab.com/gitlab-org/gitlab/-/issues/364102

This makes merging a project MR possible even if the manifest has
received new commits since the MR pipeline build has run. The developer
is responsible for deciding whether it is safe to merge it without
having a build based on the latest manifest.

BCS 746-000794

Our server installation uses port 4444 for ssh access. The lava tests
didn't took this into account.

Prior to this change the integration stop when one of the projects had
already the latest ref of gitlab-ci in to submodule.

Prior to this change the integration stop when one of the
projects had already the latest ref of gitlab-ci in to submodule.

Prior to this change the integration stop when one of the projects had
already the latest ref of gitlab-ci in to submodule.

The script can also be run in the normal python container, so this
slightly reduces the complexity.

The image and distro for the buildfng job can now be set via
global/external variables.

BCS 746-000636

The integration branch was reused even if there where new commits on the
target branch, which could lead into reverting some changes.
This adds a check, if the existing integration branch is directly based
on the target branch and deletes it if not.

BCS 746-000785

The pipeline is not setup to build from different manifest and so on,
and the dual_espresso manifest will probably go to a seperate repo
to keep it private.

BCS 746-000518

BCS 746-000637

* Split up build and package steps in separate files
* Add additional comments
* Add log output

BCS 746-000637

Rename variable JENKINSGUF_SSH_PRIVATE_KEY / SSH_PRIVATE_KEY to
GITLAB_PRIVATE_KEY on this occasion, because it contains a private key
that was generated exclusively for this use case. The according public
key has been added as a deploy key in GitLab to all repositories that
this repository needs access to.

Add more detailed documentation concerning this configurationdirectly in
the gitlab-ci files.

Tim Jaacks authored 2 years ago and Jonas Höppner committed 2 years ago

BCS DevOps-000035