Previously the default branch was allways used to generate the tests
from, now it is possible to specify the branch as cmdline option.
Additionally, the revision of the test repository is added to the
call of the test generator, so the tests run at a fix rev.

BCS 746-000547

Uses submit_all instead of submit.
Some code cleanup also.

The testsuite is generated per platform and submited to the lava server.
The resulting jobs are monitored and the results are downloaded in the
results/ subfolder.

The paths have been hardcoded inside the script before. To
allow reuse in different situations, the hardcoded paths are removed
and options are added instead.

To upload the sdk the local deployment on build host is needed as it is
too big for the uploads.

Move copy functionality to extra function. It also generates md5sum.txt
from the source file's md5sums.
Copy locally (host system) and as gitlab artifact.

Add option to copy doc files ( all .md in given folder).

BCS 746-000262

Unfortunately, GitLab still interpreted three of the four dashes as
start and end of YAML syntax, so we go back to two dashes.

The former "---" was still rendered as YAML syntax by GitLab in some
cases.

Restore the "---" separator which was removed in a previous commit. The
description simply must not start with this separator. If we include it
only in between the blocks, all instances are rendered as a horizontal
line, which makes the MR description more readable. Include a helper
function for this purpose, which extracts the message body from a commit
message.

If integration branch already existed and had to be replaced, the new
commit was accidentally pushed on top of the already existing commit
instead of replacing it. Fixed this by checking out the master branch
again before creating the commit.

GitLab renders "---" as a horizontal line or as a frame. We do not want
this, so replace it with "--".

Sometimes the job log of canceled jobs still is truncated, even with the
larger sleep. Thus we can reduce it again. Usually the job log is
complete after a retry.

If an upstream pipeline has been canceled, this is reflected in the job
result now.

The jobs in the deploy stage have to be triggered manually in GitLab.
There is one deploy job for each project which uses the gitlab-ci
scripts as a submodule, so that the deployment can be performed step by
step.

If executed within MR context, an integration MR is created and left
open. The user can extend this integration MR, e.g. if CI scripts have
been renamed, changed command line arguments or other changes requiring
updates of the correspronding .gitlab-ci.yml file. Subsequent runs of
this job will re-create the integration branch, so manual changes are
lost in this case.

If executed on the master branch (i.e. after the source MR has been
merged), the job does exactly the same, plus the integration MR will be
automatically merged. If this fails, the job will fail as well.

We still had cases where a merge request was rebased multiple times after an
initial (actually required) rebase:

https://gitlab.com/garz-fricke/yocto/infrastructure/ci-test/minimal-bar/-/jobs/1345964713

I assume that GitLab simply does not update the MR that fast and so we attempt
to do a rebase, while the MR still reports it is merge-ready.

Adding a wait loop now where the MR's sha is compared to the one of the new
commit in order to make sure that the MR has been updated before we attempt
another rebase.

We had one case where the merge() function returned successfully, but the MR
was not merged (needed a rebase actually):

https://gitlab.com/garz-fricke/yocto/infrastructure/ci-test/minimal-bar/-/jobs/1345574290

This seems to be a known issue:

https://gitlab.com/gitlab-org/gitlab-foss/-/issues/59577
https://gitlab.com/gitlab-org/release-tools/-/issues/277#note_154733089

It seems like in these rare cases, the error is reflected in the merge_error
field. Add evaluation of this to the error handling.

There are rare cases where a merge request does not have a pipeline, even
though this might only occur under testing circumstances, e.g.:

https://gitlab.com/garz-fricke/yocto/infrastructure/ci-test/minimal-foo/-/merge_requests/58

The retrigger_mr_pipeline_job script throws an exception in these cases:

https://gitlab.com/garz-fricke/yocto/infrastructure/ci-test/minimal-manifest/-/jobs/1345385316

Fix this by checking for existing pipeline before querying it.

We had some cases where the merge job added rebases of a commit where rebasing
was not necessary at all, e.g.:
https://gitlab.com/garz-fricke/yocto/layers/meta-guf-machine/-/jobs/1270293595

This is obviously a known issue in GitLab concerning the HTTP return value of
the merge endpoint:
https://gitlab.com/gitlab-org/gitlab/-/issues/196962

Suggested workaround is to manually check merge_status and wait until it is
ready before attempting to merge the MR.

* Add a draw.io diagram that illustrates the flow of the CI pipelines
across the manifest and layer repos
* Add a png export of the diagram

* Add a draw.io diagram that illustrates the flow of the CI pipelines
across the manifest and layer repos

The sdk is split into 512M parts to be handled as gitlab artefact.
All sdk steps are optional for now.

BCS 746-000262

When a MR is merged via WebUI using the 'squash commits' option the
squashed commit is assigned to the 'squash_commit_sha' property of the
MR object. We have to test for this property too, when looking which MR
a commit belongs to in case the pipeline runs on a squashed commit.
Otherwise parent MR cannot be found.

Don't rename the release artifacts, just copy to a versioned folder.
The list of files to copy is taken from variables set in yocto
DISTRO_ARTIFACTS and DISTRO_IMAGES.
Images are linked, all other files are copied.

BCS 746-000273

Don't rename the release artifacts, just copy to a versioned folder.
The list of files to copy is taken from variables set in yocto
DISTRO_ARTIFACTS and DISTRO_IMAGES.
Images are linked, all other files are copied.

BCS 746-000273