From ab708157d653ffeadfaec237d01ad84d255e8e0a Mon Sep 17 00:00:00 2001
From: gitbot <gitbot@garz-fricke.com>
Date: Tue, 23 Aug 2022 17:21:44 +0000
Subject: [PATCH] Integrate gitlab-ci/multiple-integration-pipelines and 19
more
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/051871d93c7c6183ea611100d37adb56ad332a27
Retrigger MRs from all integrating projects
Instead of getting the projects from the manifest file, search them via
their INTEGRATION variable.
Improve log messages on this occasion: print URL instead of just IDs.
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/e90153660198bb968299a6ba0551d17caa73760f
Deploy gitlab-ci to multiple branches
Add different integration jobs for different manifest branches. The
merge stage only has one job for each manifest, though, because
otherwise we cannot guarantee a consistent state for all branches.
Extend the deploy script for this purpose, so that it can deploy to
multiple manifest branches at once.
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/8840fc7e2dbeccb91fccc15ece507cdf24d858b5
Fix temporary dir path
We were passing the complete TemporaryDirectory object to the repo clone
function instead of just the path string, resulting in the repo being
cloned into a local dir "<TemporaryDirectory '/tmp/tmphwakypf8'>".
Fix this to actually use the generated temp dir.
This change makes it necessary to keep the TemporaryDirectory object
reference until we don't need the directory anymore, otherwise it will
be removed immediately.
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/b50c9ace0cad7ab2beef070535c4e68cf5da9c06
Print less important output in grey color
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/1e3136cd92a943b65831571f36504fb59b287695
deploy_gitlab_ci: remove hard-coded integration targets
Get targets dynamically using the INTEGRATION variable instead.
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/02683cc1dc898a138634b11d03b7d6e65155148f
Recurse to child pipelines on retrigger
This makes it possible to retrigger jobs that are not part of the
given pipeline but in any nested child pipeline of it.
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/0efe0f897615011f2a4fa1e1b97c41ad75ed2573
Ignore __pycache__ in scripts directory
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/db4d10516d543f051fda321b2359ac835a4a5ab5
Refactoring: rename arguments for consistency
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/59ce0ad8b62eeb76fca30e4ff9a547df88227830
Remove hard coded project and branch variables
MANIFEST_PROJECT, MASTER_BRANCH_MANIFEST, MASTER_BRANCH_PROJECT
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/68bd12000f40a93533f839f01f44dd0659517599
Add rules for integration pipeline
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/02f902123c8a19591318f8dbe257c2ca21f564c6
Improve log messages
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/cedc7c1d87b474d4ab45949ed7f5a28583dcfc10
commit_and_push: do not pass branch
Branch can be implicitly determined via the repo.
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/f462d140919fb0f508501a9d94d4b70df0f741b4
Throw and catch exception if git push fails
Using new function raise_if_error() of GitPython 3.1.25:
https://gitpython.readthedocs.io/en/stable/tutorial.html#handling-remotes
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/892c12fe363d66e63757ec4214d27c87dd82fde9
Create separate integration branches for each target branch
This makes it possible to integrate a project branch into different
branches of the same manifest project.
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/77f767c1ab15ffa4fa5a39d9a563ec063d1f8af3
Generate integration pipeline based on CI variable
Evaluate the INTEGRATION variable to determine target project and branch
for integration instead of using the hard-coded variables. For now this
works only for a single integration per target project because the
integration branch name does not include the target branch and hence
would be identical when trying to integrate into multiple branches of
the same project.
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/5c2defce01299c88e288deb065f5bb3c0522c45a
CI: prepare to work also with renamed machine names lime seco-mx6
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/ef08f4efda5cd346d8750db962bb6dfd066dc3f7
deploy: Increase deploy timeout to 30min
Especially when downloading the SDK the deploy takes longer than 10min.
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/968aaadd87c2046c419c4ad6ec97d4c6281a7ece
build-jobs: Remove dump_install_command from buildsdk job
When the dump_install_command script is executed in the SDK build step
an error occurs because the fng-install script/vars are not available.
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/52bae4a14bf18d84ac5007db6ecd2ac4756142a1
Lava Testreporting: Add script to generate markdown report for given jobids
Adds scripts for mark down generation.
Adds scripts for lava test result queries.
Adds script to generate a mark down report from this data
Call script from submit_test to generate a report directly
after the testrun.
--
Commit: https://git.seco.com/seco-ne/yocto/infrastructure/gitlab-ci/-/commit/ab09db0b5ed305ecae20a578bf84e5acba3eb93b
CI: Lava Test: Allow to install test image directly from gitlab
The images from the build job can be directly installed from gitlab.
To achive this some changes in the complete pipeline have been needed.
1. The variables used in the build job, like CI_PARAM_IMAGE, ... and
related variables like BUILDPATH are only valid in the build job now.
2. The build job writes every variable needed in a follow up job into
build.env. This also includes the url to the fng-install.sh of the
final image.
3. The build.env file is used as dotenv artifact, as well as normal
file artifact.
The dotenv make the written variables automatically available in
follow up jobs, that are using the aritfacts, like the deploy job.
The normal file artifact is available via artifact download.
(I did't found a way to download the dotenv file instead)
4. Some scripts have been added:
- Find a job inside the pipeline by name, as the id is not known in
advance.
- Download all artifacts or one file of the artifacts from a given
job
- Download one file of the latest job by name
5. The scripts are used to download the build.env into the test job
(where not artifacts are needed anymore)
6. The script is sourced and all variables are available inside the
script.
Additionally this adds a fake build job to the ci-test pipeline, that
copies an image from srv73 and stores it as artifact in a way that a
test-job can run on it, like in the normal yocto pipeline.
---
.gitlab-ci | 2 +-
.gitlab-ci.yml | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/.gitlab-ci b/.gitlab-ci
index 075aeccb..051871d9 160000
--- a/.gitlab-ci
+++ b/.gitlab-ci
@@ -1 +1 @@
-Subproject commit 075aeccbcc116de5ff05de53460f5db3fd40802d
+Subproject commit 051871d93c7c6183ea611100d37adb56ad332a27
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 9ba07d15..04101394 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -4,7 +4,7 @@
# ---------------------------------------------------------------------------------------
include:
- project: '${CI_PROJECT_ROOT_NAMESPACE}/yocto/infrastructure/gitlab-ci'
- ref: 075aeccbcc116de5ff05de53460f5db3fd40802d
+ ref: 051871d93c7c6183ea611100d37adb56ad332a27
file: 'manifest-integration-yocto.yml'
variables:
--
GitLab