MANIFEST_PROJECT, MASTER_BRANCH_MANIFEST, MASTER_BRANCH_PROJECT

Branch can be implicitly determined via the repo.

Using new function raise_if_error() of GitPython 3.1.25:
https://gitpython.readthedocs.io/en/stable/tutorial.html#handling-remotes

This makes it possible to integrate a project branch into different
branches of the same manifest project.

Evaluate the INTEGRATION variable to determine target project and branch
for integration instead of using the hard-coded variables. For now this
works only for a single integration per target project because the
integration branch name does not include the target branch and hence
would be identical when trying to integrate into multiple branches of
the same project.

We have created tags in our poky repo which are needed by repo, e.g.
https://git.seco.com/seco-ne/3rd-party/yocto/poky/-/tags/seconorth%2Fkirkstone-4.0.0

Do not explicitly exclude tags in order to fetch them.

Especially when downloading the SDK the deploy takes longer than 10min.

When the dump_install_command script is executed in the SDK build step
an error occurs because the fng-install script/vars are not available.

Adds scripts for mark down generation.
Adds scripts for lava test result queries.
Adds script to generate a mark down report from this data
Call script from submit_test to generate a report directly
after the testrun.

The images from the build job can be directly installed from gitlab.
To achive this some changes in the complete pipeline have been needed.

1. The variables used in the build job, like CI_PARAM_IMAGE, ... and
   related variables like BUILDPATH are only valid in the build job now.
2. The build job writes every variable needed in a follow up job into
   build.env. This also includes the url to the fng-install.sh of the
   final image.
3. The build.env file is used as dotenv artifact, as well as normal
   file artifact.
   The dotenv make the written variables automatically available in
   follow up jobs, that are using the aritfacts, like the deploy job.
   The normal file artifact is available via artifact download.
   (I did't found a way to download the dotenv file instead)
4. Some scripts have been added:
   - Find a job inside the pipeline by name, as the id is not known in
     advance.
   - Download all artifacts or one file of the artifacts from a given
     job
   - Download one file of the latest job by name
5. The scripts are used to download the build.env into the test job
   (where not artifacts are needed anymore)
6. The script is sourced and all variables are available inside the
   script.

Additionally this adds a fake build job to the ci-test pipeline, that
copies an image from srv73 and stores it as artifact in a way that a
test-job can run on it, like in the normal yocto pipeline.

This shares YAML code for the following procedures:
- Build environment
- Source code checkout via repo
- SSH key and known hosts setup
- LAVA test submission
- Docker check if build folder is empty

The variables CI_PARAM_IMAGE, CI_PARAM_DISTRO, CI_MACHINE have been
defined outside the job in the '.gitlab-ci.yml' file. This has a lower
precedence then variables defined inside the job, but it seems that
these variables are used in the scope of 'Trigger variables' when
available in the trigger job. These override the value set in the
template. As main results this made FNGSystem jobs build the normal
yocto.

According to the docs a manual job always has allow_failure set to
true, but we see a lot of jobs staying in an unfinished state since
the removal of the allow_failure: true keyword for the sdk and test
jobs. This should fix the broken behaviour.

Some gitlab update(?) seem to have changed the behaviour in
variable passing. In the scripts the variables are available
but in the artifacts path does not resolve them any more.
May be it has something to do with the introduction of the
trigger:forward keyword.

Previous commit did not have the effect due to wrong extend order.

This way they can be reused instead of redefining them in each job.

Combine all common yaml parts in manifest-pipeline.yml and add
manifest-pipeline-yocto.yml and manifest-pipeline-ci-test.yml
containing the different variable assignments for each environment.

This change implicitly introduces parent-child build job generation in
the ci-test pipeline, like it is done in the yocto pipeline already.
The ci-test build jobs have been moved to build-jobs-ci-test.jinja2
accordingly.

Furthermore rename GITLAB_CI_CURRENT_REV to GITLAB_CI_REVISION and
remove the run conditions from all generated build jobs, since these are
already present in the upstream trigger job.

The repos including these files have to be updated with the new file
and variable names.

The job generation script implicitly passes the OS environment to the
template, so that the template has access to all GitLab CI variables.
Hence there is no need to explicitly pass any of them as command line
arguments.

This change makes the "generate-build-jobs" job more generic, so that
it can be shared with the ci-test pipeline in the future.

This job was accidentally disabled in 2e6c1b4f, because it was included
in the generated child pipeline. This pipeline is not created on the
master, though, so the condition for the job was never met.

Move the job to the parent pipeline again, so that it gets executed on
the master like before.

Was accidentally removed in previous commit.

The yamllint job was included in the pipeline implicitly via the
inclusion of common.yml. This is not very transparent. Make the common
definition a base job and extend it explicitly in each pipeline now.

Add infrastructure stage comment header into both yaml files.

Leave all common yaml parts in manifest-integration.yml and add
manifest-integration-yocto.yml and manifest-integration-ci-test.yml
containing the different variable assignments for each environment.

The repos including these files have to be updated with the new names.

This is the first step for reducing code duplicates.

BCS 746-000807

Costum manifest may override this variable

If the MASTER_BRANCH_MANIFEST variable is set in the manifest's
.gitlab-ci.yml file, it may not find its way to the generated job,
if it is not explicitly set during generation.
That is done with this patch.

* Move all build, deploy and test stubs to the manifest-build.yml
* Create a new job to generate and trigger the build jobs dynamically
* Add the base jinja2 file for the build jobs.
* Add initial docs for the manifest pipeline
* Remove unused files

BCS 746-000808