The images from the build job can be directly installed from gitlab.
To achive this some changes in the complete pipeline have been needed.

1. The variables used in the build job, like CI_PARAM_IMAGE, ... and
   related variables like BUILDPATH are only valid in the build job now.
2. The build job writes every variable needed in a follow up job into
   build.env. This also includes the url to the fng-install.sh of the
   final image.
3. The build.env file is used as dotenv artifact, as well as normal
   file artifact.
   The dotenv make the written variables automatically available in
   follow up jobs, that are using the aritfacts, like the deploy job.
   The normal file artifact is available via artifact download.
   (I did't found a way to download the dotenv file instead)
4. Some scripts have been added:
   - Find a job inside the pipeline by name, as the id is not known in
     advance.
   - Download all artifacts or one file of the artifacts from a given
     job
   - Download one file of the latest job by name
5. The scripts are used to download the build.env into the test job
   (where not artifacts are needed anymore)
6. The script is sourced and all variables are available inside the
   script.

Additionally this adds a fake build job to the ci-test pipeline, that
copies an image from srv73 and stores it as artifact in a way that a
test-job can run on it, like in the normal yocto pipeline.

This shares YAML code for the following procedures:
- Build environment
- Source code checkout via repo
- SSH key and known hosts setup
- LAVA test submission
- Docker check if build folder is empty

The variables CI_PARAM_IMAGE, CI_PARAM_DISTRO, CI_MACHINE have been
defined outside the job in the '.gitlab-ci.yml' file. This has a lower
precedence then variables defined inside the job, but it seems that
these variables are used in the scope of 'Trigger variables' when
available in the trigger job. These override the value set in the
template. As main results this made FNGSystem jobs build the normal
yocto.

According to the docs a manual job always has allow_failure set to
true, but we see a lot of jobs staying in an unfinished state since
the removal of the allow_failure: true keyword for the sdk and test
jobs. This should fix the broken behaviour.

Some gitlab update(?) seem to have changed the behaviour in
variable passing. In the scripts the variables are available
but in the artifacts path does not resolve them any more.
May be it has something to do with the introduction of the
trigger:forward keyword.

Previous commit did not have the effect due to wrong extend order.

This way they can be reused instead of redefining them in each job.

Combine all common yaml parts in manifest-pipeline.yml and add
manifest-pipeline-yocto.yml and manifest-pipeline-ci-test.yml
containing the different variable assignments for each environment.

This change implicitly introduces parent-child build job generation in
the ci-test pipeline, like it is done in the yocto pipeline already.
The ci-test build jobs have been moved to build-jobs-ci-test.jinja2
accordingly.

Furthermore rename GITLAB_CI_CURRENT_REV to GITLAB_CI_REVISION and
remove the run conditions from all generated build jobs, since these are
already present in the upstream trigger job.

The repos including these files have to be updated with the new file
and variable names.

The job generation script implicitly passes the OS environment to the
template, so that the template has access to all GitLab CI variables.
Hence there is no need to explicitly pass any of them as command line
arguments.

This change makes the "generate-build-jobs" job more generic, so that
it can be shared with the ci-test pipeline in the future.

This job was accidentally disabled in 2e6c1b4f, because it was included
in the generated child pipeline. This pipeline is not created on the
master, though, so the condition for the job was never met.

Move the job to the parent pipeline again, so that it gets executed on
the master like before.

Was accidentally removed in previous commit.

The yamllint job was included in the pipeline implicitly via the
inclusion of common.yml. This is not very transparent. Make the common
definition a base job and extend it explicitly in each pipeline now.

Add infrastructure stage comment header into both yaml files.

Leave all common yaml parts in manifest-integration.yml and add
manifest-integration-yocto.yml and manifest-integration-ci-test.yml
containing the different variable assignments for each environment.

The repos including these files have to be updated with the new names.

This is the first step for reducing code duplicates.

BCS 746-000807

Costum manifest may override this variable

If the MASTER_BRANCH_MANIFEST variable is set in the manifest's
.gitlab-ci.yml file, it may not find its way to the generated job,
if it is not explicitly set during generation.
That is done with this patch.

* Move all build, deploy and test stubs to the manifest-build.yml
* Create a new job to generate and trigger the build jobs dynamically
* Add the base jinja2 file for the build jobs.
* Add initial docs for the manifest pipeline
* Remove unused files

BCS 746-000808

According to the documentation [1] we should get a 406 error from the
API if we try to merge a merge request which has a merge conflict. We
are experiencing a different behaviour, though, where we are getting a
405 error in this case. There is an open GitLab issue on this topic [2].

In our error handling we assumed that the reason for a 405 error is most
likely a required pipeline which is still running. This assumtion,
however, leads to wrong behaviour in cases where no pipeline is started
at all.

Fixing this now by explicitly checking for merge conflicts at the very
beginning of the error handling sequence.

[1]: https://docs.gitlab.com/ee/api/merge_requests.html#merge-a-merge-request
[2]: https://gitlab.com/gitlab-org/gitlab/-/issues/364102

This makes merging a project MR possible even if the manifest has
received new commits since the MR pipeline build has run. The developer
is responsible for deciding whether it is safe to merge it without
having a build based on the latest manifest.

BCS 746-000794

Our server installation uses port 4444 for ssh access. The lava tests
didn't took this into account.

Prior to this change the integration stop when one of the projects had
already the latest ref of gitlab-ci in to submodule.

Prior to this change the integration stop when one of the
projects had already the latest ref of gitlab-ci in to submodule.

Prior to this change the integration stop when one of the projects had
already the latest ref of gitlab-ci in to submodule.