Explicitly triggering a pipeline in the target repository does not work
easily from this place because we would have to distinguish between the
manifest (which uses branch pipelines) and the sub-projects (which use
merge request pipelines). This separation should not be hard-coded here,
so we move the decision whether or not to run a pipeline to the
.gitlab-ci.yml file of the target. Reverting these two commits:

63279799
    deploy_gitlab_ci: trigger pipeline only if it has not run before

dfc6204a
    deploy_gitlab_ci: trigger pipeline on integration branch

The function used to rely on the given merge request object without
checking its current state. If the given object was not in one of the
"unchecked" states, it returned immediately. The state, however, could
have changed between its last query and the call of this function, so
it now checks the state first before deciding anything.

Fix pylint errors introduced with new pylint version:
* unspecified-encoding: explicitly set encoding when opening a file
* assigning-non-slot: use "set_reference()" function instead of assigning
  a value to the "reference" attribute
* unused-variable: remove "e" variable on exceptions when not referenced

A bug in python-gitlab > 2.7.0 [1] brought this up: In the past we
relied on the merge request's setting, whether the source branch should
be deleted or not after a merge. Since we're using the API's merge
function in a fully automated task, however, the branch should always be
deleted. Add the according argument explicitly.

[1]: https://github.com/python-gitlab/python-gitlab/issues/1750

* Combine two return statements into one.
* Switch if statement order to remove else branch.

The outdated YAML 1.1 specification allowed values like 'yes' or 'no'
for boolean fields [1]. The newer spec 1.2 allows only 'true' and
'false' [2], so we update them to be conform with it.

[1]: https://yaml.org/type/bool.html
[2]: https://yaml.org/spec/1.2.2/#10212-boolean

Remove the trigger source parameter again because it did not serve the
use case. Add ref parameter instead.

BCS DevOps-000023 DevOps-000024

alphaplan_fwr.py: Creates alphaplan software packages for the build
release artifacts. According to the release, all relevant artifacts are
collected in a list structure and the corresponding aphaplan
information are added for each artifact (attributes, md5sum, path, etc.).
The list is attached to major software package structure. This structure
is written in a json file for documentation. Afterwards the structure is
send to an AlphaPlan webservice that creates the FWR articles.
alphaplan_keys.py: Generates a dictionary which contains all AlphaPlan
specific information for the artifacts. Flag classes are used to define
unique keys for the dictionary accesses.

Previously the default branch was allways used to generate the tests
from, now it is possible to specify the branch as cmdline option.
Additionally, the revision of the test repository is added to the
call of the test generator, so the tests run at a fix rev.

BCS 746-000547

Uses submit_all instead of submit.
Some code cleanup also.

The testsuite is generated per platform and submited to the lava server.
The resulting jobs are monitored and the results are downloaded in the
results/ subfolder.

The paths have been hardcoded inside the script before. To
allow reuse in different situations, the hardcoded paths are removed
and options are added instead.

To upload the sdk the local deployment on build host is needed as it is
too big for the uploads.

Move copy functionality to extra function. It also generates md5sum.txt
from the source file's md5sums.
Copy locally (host system) and as gitlab artifact.

Add option to copy doc files ( all .md in given folder).

BCS 746-000262

Unfortunately, GitLab still interpreted three of the four dashes as
start and end of YAML syntax, so we go back to two dashes.

The former "---" was still rendered as YAML syntax by GitLab in some
cases.

Restore the "---" separator which was removed in a previous commit. The
description simply must not start with this separator. If we include it
only in between the blocks, all instances are rendered as a horizontal
line, which makes the MR description more readable. Include a helper
function for this purpose, which extracts the message body from a commit
message.

If integration branch already existed and had to be replaced, the new
commit was accidentally pushed on top of the already existing commit
instead of replacing it. Fixed this by checking out the master branch
again before creating the commit.

GitLab renders "---" as a horizontal line or as a frame. We do not want
this, so replace it with "--".

Sometimes the job log of canceled jobs still is truncated, even with the
larger sleep. Thus we can reduce it again. Usually the job log is
complete after a retry.

If an upstream pipeline has been canceled, this is reflected in the job
result now.

The jobs in the deploy stage have to be triggered manually in GitLab.
There is one deploy job for each project which uses the gitlab-ci
scripts as a submodule, so that the deployment can be performed step by
step.

If executed within MR context, an integration MR is created and left
open. The user can extend this integration MR, e.g. if CI scripts have
been renamed, changed command line arguments or other changes requiring
updates of the correspronding .gitlab-ci.yml file. Subsequent runs of
this job will re-create the integration branch, so manual changes are
lost in this case.

If executed on the master branch (i.e. after the source MR has been
merged), the job does exactly the same, plus the integration MR will be
automatically merged. If this fails, the job will fail as well.

We still had cases where a merge request was rebased multiple times after an
initial (actually required) rebase:

https://gitlab.com/garz-fricke/yocto/infrastructure/ci-test/minimal-bar/-/jobs/1345964713

I assume that GitLab simply does not update the MR that fast and so we attempt
to do a rebase, while the MR still reports it is merge-ready.

Adding a wait loop now where the MR's sha is compared to the one of the new
commit in order to make sure that the MR has been updated before we attempt
another rebase.

We had one case where the merge() function returned successfully, but the MR
was not merged (needed a rebase actually):

https://gitlab.com/garz-fricke/yocto/infrastructure/ci-test/minimal-bar/-/jobs/1345574290

This seems to be a known issue:

https://gitlab.com/gitlab-org/gitlab-foss/-/issues/59577
https://gitlab.com/gitlab-org/release-tools/-/issues/277#note_154733089

It seems like in these rare cases, the error is reflected in the merge_error
field. Add evaluation of this to the error handling.

There are rare cases where a merge request does not have a pipeline, even
though this might only occur under testing circumstances, e.g.:

https://gitlab.com/garz-fricke/yocto/infrastructure/ci-test/minimal-foo/-/merge_requests/58

The retrigger_mr_pipeline_job script throws an exception in these cases:

https://gitlab.com/garz-fricke/yocto/infrastructure/ci-test/minimal-manifest/-/jobs/1345385316

Fix this by checking for existing pipeline before querying it.

We had some cases where the merge job added rebases of a commit where rebasing
was not necessary at all, e.g.:
https://gitlab.com/garz-fricke/yocto/layers/meta-guf-machine/-/jobs/1270293595

This is obviously a known issue in GitLab concerning the HTTP return value of
the merge endpoint:
https://gitlab.com/gitlab-org/gitlab/-/issues/196962

Suggested workaround is to manually check merge_status and wait until it is
ready before attempting to merge the MR.

* Add a draw.io diagram that illustrates the flow of the CI pipelines
across the manifest and layer repos
* Add a png export of the diagram

* Add a draw.io diagram that illustrates the flow of the CI pipelines
across the manifest and layer repos