Integrate gitlab-ci/CI-deploy-yocto and 19 more

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/04cd24f23863acb3ff54535ec166c0c7734abe4e

CI: Remove debug, enable yocto manifest deploy

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/89bed7e6841bb0bba9a8dcc221060b8f772f59c2

CI: Add function to merge the gitlab-ci MRs

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/b9cb9a29a80973721956fb6c153c5bc9bfb51dde

CI: Add function to merge the gitlab-ci MRs

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/25d7cabdb130955c0754c3742a5be2e4b747c491

CI: Harmonize foobar and yocto CI files

Also set yocto deploy job to manual to test merge.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/2cdf882e1e25e6c9e072d3b4e4160a3f3a1b6c54

CI: Enable analyze stage again in .gitlab-ci.yml again

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/26bd3edfe3ba62745cb5484422b366cccd609aab

CI: Add common.yml containing common code for manifest and integration.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/8bfa443a1550db2b464166f0a69db6f01bcb6c0b

CI: Use only one stage for integration to allow parallel execution

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/bb392f5037b495a3c9f7dfa3ea7b340cf0163754

CI: Rewrite check job python file to use gitlab api (no clone anymore)

For projects like the kernel the git checkout needs a log of time and
space. Directly using the gitlab api only loads the infomation needed.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/e60e50b1ff159331a257fb02fa77b2f38952c526

CI: Limit clone depth to 1 for speed

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/56b4cacb6f2a62b0dc325275585fe6673d453ba3

CI: Adapt manifests-integration.yml and manifest.yml for yocto projects

Integrates the changes done in the foobar- yml files to the yocto
projects.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/7cc7fa7701ec2713d3af7e8e21563163e600b4d7

CI: Add all yocto projects to the job list

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/b097a25350e1b4192c0b84aa6e7bb5a77477e3b6

CI: Generate .gitlab-ci.yml with include commands in deploy step

This change should be reverted after once used.
It is needed to initially add the include .gitlab-ci.yml in
all subprojects.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/4f194e2141fc72023dcf0ace4651475839392f5b

CI: Check: integration branch may have multiple commits

As all projects are commited in the same branch the 'up-to-date' check
may not only check if the first parent commit points to the
master/dunfell branch. Now it is needed to loop through the history
until the integration branch's commit is found.

On fail a message is displayed which merge request needs to be
retriggered manually. This can now also be the 'parent'-MR that
triggered the complete chain.

The check job is used pipeline again.

The retrigger job also looks in the .gitlab-ci project for check jobs to
retrigger.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/873ae97fa52ff62e515cf7681a383ed9a9ef608b

CI: Disable foobar manifest build for integration branch

BCS 746-000646

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/e081a8d8d4d4b6096f2b64fe450fa0d50de159b2

CI: Create integration branch optionally also when no submodule change is detected

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/4452c5cbe26b80b4559c95543c6258cbd8382f25

CI: Implement that a merge-request may also be specified with project!number

Before only the ID was used but the CI_OPEN_MERGE_REQUESTS variable has
the complete path.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/e86d320a42e60f253a534a66e8cee4afe93b30e2

CI: Multideploy: Detect exiting integration branch

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/0ee24df09955c771ab2d06591ab98873a1bc115a

CI: Deploy: create integration branches and trigger pipelines in subprojects

Allow deploy_gitlab_ci to change multiple projects at once.
Use it to create integration branches and merge requests in all
projects.

Add a python file to generate a job yml from jinja2 template.
Add a template for the jobs to trigger. These execute the actual
integration in all 'subprojects'.
Create the yml file in the deploy job and trigger it in a new trigger
job.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/6b6d57180c086b313399d268605e39f903d96358

CI: Add common gitlab-ci file for all yocto repositories.

Create a merged version of the gitlab-ci files from the layers and the
repositoried referenced by the SRCREV file to be included in the
projects, to replace the project-individual implementation.

BCS 746-000646

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/369e59b3f333487c032a21a69eb4f1445f821c52

CI: Update include file's ref in gitlab-ci.yml when deploying changes

Add update gitlab-ci file with function to adapt the include ref to a
given revision.
Add a 'pre-commit-hook' to the update-submodule function.
Adapt deploy_gitlab_ci to use these to update the include statement in
the base project .gitlab-ci.yml to use the same ref as the submodule
is set to.

BCS 746-000646

.gitlab-ci @ 04cd24f2

-Subproject commit 947da6d74caa0963406ecc430a436a22794761e7
+Subproject commit 04cd24f23863acb3ff54535ec166c0c7734abe4e

.gitlab-ci.yml

+---
 # ---------------------------------------------------------------------------------------
-# Global
+# Include the default CI steps from the gitlab-ci repo
 # ---------------------------------------------------------------------------------------
-image: "${CI_IMAGES}/python/3.9:99e363bc5feaa27ff18dbe7731a76ff04d7d0deb"
-
-stages:
-  - integrate
-  - merge
-  - build
-  - check
-
-variables:
-  # CI_IMAGES_BASEPATH: Environment variable configure in gitlab
-  CI_IMAGES: ${CI_IMAGES_BASEPATH}/ci-images
-  # Include git submodules
-  GIT_SUBMODULE_STRATEGY: recursive
-  # FIXME: due to a missing feature in GitLab we cannot use this variable
-  # in the build stage further down this file. If it ever changes, it has
-  # to be changed there too.
-  # (https://gitlab.com/gitlab-org/gitlab/-/issues/249583)
-  MANIFEST_PROJECT: ${CI_PROJECT_ROOT_NAMESPACE}/yocto/manifest
-  MASTER_BRANCH: dunfell
-
-workflow:
-  rules:
-    # Do not run pipelines on forked projects
-    - if: $CI_PROJECT_PATH == "SECO-Northern-Europe/yocto/config"
-        && $CI_MERGE_REQUEST_SOURCE_PROJECT_ID == $CI_MERGE_REQUEST_PROJECT_ID
-
-# ---------------------------------------------------------------------------------------
-# Stage: integrate
-# ---------------------------------------------------------------------------------------
-integrate:
-  stage: integrate
-  rules:
-    # We have to make sure that the pipeline runs for the current manifest
-    # master at the time a merge request is created. Otherwise we cannot
-    # guarantee a green master after merging.
-    - if: $CI_MERGE_REQUEST_IID
-  tags:
-    - infrastructure
-  timeout: 2m
-  cache:
-    policy: push
-  script:
-    - cd ${CI_PROJECT_DIR}
-    - .gitlab-ci/integrate_into_manifest.py
-        --gitlab-url=${CI_SERVER_URL}
-        --token=${GITBOT_TOKEN}
-        --manifest-project=${MANIFEST_PROJECT}
-        --integration-base=${MASTER_BRANCH}
-        --project=${CI_PROJECT_PATH}
-        --merge-request=${CI_MERGE_REQUEST_IID}
-        --save-revision-to=manifest_revision
-  artifacts:
-    paths:
-      - manifest_revision
-
-# ---------------------------------------------------------------------------------------
-# Stage: merge
-# ---------------------------------------------------------------------------------------
-merge:
-  stage: merge
-  rules:
-    - if: $CI_COMMIT_BRANCH == $MASTER_BRANCH
-  tags:
-    - infrastructure
-  timeout: 3m
-  script:
-    - cd ${CI_PROJECT_DIR}
-    - .gitlab-ci/merge_into_manifest.py
-        --gitlab-url=${CI_SERVER_URL}
-        --token=${GITBOT_TOKEN}
-        --manifest-project=${MANIFEST_PROJECT}
-        --master-branch=${MASTER_BRANCH}
-        --project=${CI_PROJECT_PATH}
-        --commit=${CI_COMMIT_SHA}
-        --save-revision-to=manifest_revision
-  artifacts:
-    paths:
-      - manifest_revision
-
-# ---------------------------------------------------------------------------------------
-# Stage: build
-# ---------------------------------------------------------------------------------------
-build:
-  stage: build
-  rules:
-    - if: $CI_MERGE_REQUEST_IID
-  trigger:
-    project: SECO-Northern-Europe/yocto/manifest
-    branch: "integrate/${CI_PROJECT_NAME}/${CI_COMMIT_REF_NAME}"
-    strategy: depend
-
-# ---------------------------------------------------------------------------------------
-# Stage: check
-# ---------------------------------------------------------------------------------------
-check:
-  stage: check
-  rules:
-    - if: $CI_MERGE_REQUEST_IID
-  needs: ["integrate"]
-  tags:
-    - infrastructure
-  timeout: 2m
-  script:
-    - cd ${CI_PROJECT_DIR}
-    - .gitlab-ci/check_if_integration_branch_is_up_to_date.py
-        --gitlab-url=${CI_SERVER_URL}
-        --token=${GITBOT_TOKEN}
-        --manifest-project=${MANIFEST_PROJECT}
-        --integration-base=${MASTER_BRANCH}
-        --project=${CI_PROJECT_PATH}
-        --merge-request=${CI_MERGE_REQUEST_IID}
+include:
+  - project: '${CI_PROJECT_ROOT_NAMESPACE}/yocto/infrastructure/gitlab-ci'
+    ref: 04cd24f23863acb3ff54535ec166c0c7734abe4e
+    file: 'manifest-integration.yml'