Skip to content
Snippets Groups Projects
Commit 88c50591 authored by GitBot's avatar GitBot
Browse files

Integrate gitlab-ci/CI-deploy-yocto and 19 more 

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/04cd24f23863acb3ff54535ec166c0c7734abe4e

CI: Remove debug, enable yocto manifest deploy

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/89bed7e6841bb0bba9a8dcc221060b8f772f59c2

CI: Add function to merge the gitlab-ci MRs

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/b9cb9a29a80973721956fb6c153c5bc9bfb51dde

CI: Add function to merge the gitlab-ci MRs

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/25d7cabdb130955c0754c3742a5be2e4b747c491

CI: Harmonize foobar and yocto CI files

Also set yocto deploy job to manual to test merge.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/2cdf882e1e25e6c9e072d3b4e4160a3f3a1b6c54

CI: Enable analyze stage again in .gitlab-ci.yml again

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/26bd3edfe3ba62745cb5484422b366cccd609aab

CI: Add common.yml containing common code for manifest and integration.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/8bfa443a1550db2b464166f0a69db6f01bcb6c0b

CI: Use only one stage for integration to allow parallel execution

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/bb392f5037b495a3c9f7dfa3ea7b340cf0163754

CI: Rewrite check job python file to use gitlab api (no clone anymore)

For projects like the kernel the git checkout needs a log of time and
space. Directly using the gitlab api only loads the infomation needed.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/e60e50b1ff159331a257fb02fa77b2f38952c526

CI: Limit clone depth to 1 for speed

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/56b4cacb6f2a62b0dc325275585fe6673d453ba3

CI: Adapt manifests-integration.yml and manifest.yml for yocto projects

Integrates the changes done in the foobar- yml files to the yocto
projects.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/7cc7fa7701ec2713d3af7e8e21563163e600b4d7

CI: Add all yocto projects to the job list

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/b097a25350e1b4192c0b84aa6e7bb5a77477e3b6

CI: Generate .gitlab-ci.yml with include commands in deploy step

This change should be reverted after once used.
It is needed to initially add the include .gitlab-ci.yml in
all subprojects.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/4f194e2141fc72023dcf0ace4651475839392f5b

CI: Check: integration branch may have multiple commits

As all projects are commited in the same branch the 'up-to-date' check
may not only check if the first parent commit points to the
master/dunfell branch. Now it is needed to loop through the history
until the integration branch's commit is found.

On fail a message is displayed which merge request needs to be
retriggered manually. This can now also be the 'parent'-MR that
triggered the complete chain.

The check job is used pipeline again.

The retrigger job also looks in the .gitlab-ci project for check jobs to
retrigger.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/873ae97fa52ff62e515cf7681a383ed9a9ef608b

CI: Disable foobar manifest build for integration branch

BCS 746-000646

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/e081a8d8d4d4b6096f2b64fe450fa0d50de159b2

CI: Create integration branch optionally also when no submodule change is detected

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/4452c5cbe26b80b4559c95543c6258cbd8382f25

CI: Implement that a merge-request may also be specified with project!number

Before only the ID was used but the CI_OPEN_MERGE_REQUESTS variable has
the complete path.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/e86d320a42e60f253a534a66e8cee4afe93b30e2

CI: Multideploy: Detect exiting integration branch

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/0ee24df09955c771ab2d06591ab98873a1bc115a

CI: Deploy: create integration branches and trigger pipelines in subprojects

Allow deploy_gitlab_ci to change multiple projects at once.
Use it to create integration branches and merge requests in all
projects.

Add a python file to generate a job yml from jinja2 template.
Add a template for the jobs to trigger. These execute the actual
integration in all 'subprojects'.
Create the yml file in the deploy job and trigger it in a new trigger
job.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/6b6d57180c086b313399d268605e39f903d96358

CI: Add common gitlab-ci file for all yocto repositories.

Create a merged version of the gitlab-ci files from the layers and the
repositoried referenced by the SRCREV file to be included in the
projects, to replace the project-individual implementation.

BCS 746-000646

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/369e59b3f333487c032a21a69eb4f1445f821c52

CI: Update include file's ref in gitlab-ci.yml when deploying changes

Add update gitlab-ci file with function to adapt the include ref to a
given revision.
Add a 'pre-commit-hook' to the update-submodule function.
Adapt deploy_gitlab_ci to use these to update the include statement in
the base project .gitlab-ci.yml to use the same ref as the submodule
is set to.

BCS 746-000646
parent 50a4b049
No related branches found
No related tags found
1 merge request!485Integrate gitlab-ci/CI-deploy-yocto and 19 more
Hide whitespace changes
Inline Side-by-side
.gitlab-ci @ 04cd24f2
Compare 47a0776a...04cd24f2
Subproject commit 47a0776a2ef5615117d425e9b29adde9f3b6d625 Subproject commit 04cd24f23863acb3ff54535ec166c0c7734abe4e
.gitlab-ci.yml
+ 5
− 513
View file @ 88c50591
# --------------------------------------------------------------------------------------
# Global
# --------------------------------------------------------------------------------------
--- ---
include:
- local: .gitlab-ci-build.yml
stages:
- retrigger
- prepare
- build
- test
- deploy
- uploadftp
variables:
# CI_IMAGES_BASEPATH: Environment variable configure in gitlab
CI_IMAGES_PATH: ${CI_IMAGES_BASEPATH}/ci-images
CI_IMAGES_REV: 44965ccdd847f1e077670f49d546047f8ad0110c
CI_IMAGE_PYTHON: "${CI_IMAGES_PATH}/python/3.9:${CI_IMAGES_REV}"
CI_IMAGE_YOCTO: "${CI_IMAGES_PATH}/yocto-build/ubuntu-20.04:${CI_IMAGES_REV}"
# The master branch is hardcoded here, because it cannot be determined
# automatically. Has to be modified for new branches, e.g. new Yocto
# versions or fix releases.
MASTER_BRANCH: dunfell
# Default image and distro
CI_PARAM_IMAGE: guf-image
CI_PARAM_DISTRO: guf-wayland
workflow:
rules:
# Explicitly allow externally triggered pipelines in every case
- if: $CI_PIPELINE_SOURCE == "pipeline" || $CI_PIPELINE_SOURCE == "api"
# Do not run pipelines for merge requests
- if: $CI_MERGE_REQUEST_IID
when: never
# Do not run pipelines on forked projects
# (use id instead of name because of rename)
- if: $CI_PROJECT_ID != "17852514"
when: never
# Do not run pipelines on integration branches, except from gitlab-ci repo
- if: $CI_COMMIT_REF_NAME =~ /^integrate\/.*/ &&
$CI_COMMIT_REF_NAME !~ /^integrate\/gitlab-ci\/.*/
when: never
# In all other cases, run the pipeline automatically
- when: always
# --------------------------------------------------------------------------------------
# Common infrastructure settings
# --------------------------------------------------------------------------------------
.infrastructure:
tags:
- infrastructure
timeout: 10m
image: "${CI_IMAGE_PYTHON}"
needs: []
variables:
# Include git submodules
GIT_SUBMODULE_STRATEGY: recursive
# --------------------------------------------------------------------------------------
# Stage: retrigger
# --------------------------------------------------------------------------------------
retrigger:
extends: .infrastructure
stage: retrigger
rules:
- if: $CI_COMMIT_REF_NAME == $MASTER_BRANCH && $CI_PIPELINE_SOURCE != "api"
script:
- PROJECTS=$(
.gitlab-ci/get_manifest_projects.py
--manifest=default.xml
--remote=seco-north
--concat-namespaces
)
- echo -e "Projects:\n${PROJECTS}"
- for PROJECT in ${PROJECTS}; do
.gitlab-ci/retrigger_mr_pipeline_jobs.py
--gitlab-url=${CI_SERVER_URL}
--token=${GITBOT_TOKEN}
--project=${PROJECT}
--state=opened
--target-branch=${MASTER_BRANCH}
--job=check
;
done
# --------------------------------------------------------------------------------------
# Stage: prepare
# --------------------------------------------------------------------------------------
yamllint:
extends: .infrastructure
stage: prepare
rules:
- if: $CI_COMMIT_REF_NAME != $MASTER_BRANCH || $CI_PIPELINE_SOURCE == "api"
script:
- yamllint .*.yml
changelog:
stage: prepare
rules:
- if: $CI_COMMIT_REF_NAME != $MASTER_BRANCH || $CI_PIPELINE_SOURCE == "api"
tags:
- infrastructure
variables:
IMAGE_PATH: ${CI_IMAGES_BASEPATH}/changelog-generator
IMAGE_REVISION: 2fd56690a5dabde6eda101182e8359f1c3609664
image:
name: "${IMAGE_PATH}:${IMAGE_REVISION}"
# set entrypoint to noop to be able to run from script
entrypoint: [""]
script: "changelog_generator.py --branch ${MASTER_BRANCH} > changelog.md"
artifacts:
expire_in: 4 weeks
paths:
- "changelog.md"
# --------------------------------------------------------------------------------------
# Stage: build
# --------------------------------------------------------------------------------------
.buildbase:
tags:
- builds
timeout: 8h
interruptible: true
image:
name: "${CI_IMAGE_YOCTO}"
# Override entrypoint so we can pass --id to set the UID and GID for the
# user that is created in the container. This is a feature of the
# crops/poky images. See poky-entry.py for details.
entrypoint:
- "/usr/bin/distro-entry.sh"
- "/usr/bin/dumb-init"
- "--"
- "/usr/bin/poky-entry.py"
- "--id=118:998"
artifacts:
expire_in: 4 weeks
.buildimage:
extends:
- .buildbase
- .build
rules:
- if: $CI_COMMIT_REF_NAME != $MASTER_BRANCH || $CI_PIPELINE_SOURCE == "api"
needs: []
.buildfng:
extends:
- .buildimage
variables:
CI_PARAM_IMAGE: fngsystem-image
CI_PARAM_DISTRO: guf-fngsystem
build:merge_request:
extends: .infrastructure
stage: build
timeout: 1h
rules:
- if: $CI_COMMIT_REF_NAME == $MASTER_BRANCH && $CI_PIPELINE_SOURCE != "api"
script:
- cd ${CI_PROJECT_DIR}
# Get pipeline for merge request
- MR_PIPELINE=$(.gitlab-ci/get_pipelines.py
--gitlab-url=${CI_SERVER_URL}
--token=${GITBOT_TOKEN}
--project=${CI_PROJECT_PATH}
--commit=${CI_COMMIT_SHA}
--ref=^${MASTER_BRANCH} || true | head -1)
# If pipeline exists, mirror its result
- if [ ! -z "${MR_PIPELINE}" ]; then
.gitlab-ci/mirror_pipeline_result.py
--gitlab-url=${CI_SERVER_URL}
--token=${GITBOT_TOKEN}
--project=${CI_PROJECT_PATH}
--pipeline=${MR_PIPELINE}
# If no pipeline found, trigger a new one on the master
- else
.gitlab-ci/trigger_pipeline.py
--gitlab-url=${CI_SERVER_URL}
--token=${GITBOT_TOKEN}
--project=${CI_PROJECT_PATH}
--ref=${MASTER_BRANCH}
- fi
.buildsdk:
extends:
- .buildimage
- .package
stage: build
rules:
- if: $CI_COMMIT_REF_NAME != $MASTER_BRANCH || $CI_PIPELINE_SOURCE == "api"
when: manual
allow_failure: true
variables:
BITBAKE_TASK: "populate_sdk"
artifacts:
paths:
- "${BUILDPATH}/${SDKPATH}/*.manifest"
- "${BUILDPATH}/${SDKPATH}/*.json"
reports:
dotenv: package.env
# ---------------------------------------------------------------------------------------
# Stage: test
# --------------------------------------------------------------------------------------- # ---------------------------------------------------------------------------------------
.test: # Include the default CI steps from the gitlab-ci repo
extends:
- .infrastructure
- .prepare_test
timeout: 1h
rules:
- if: $CI_COMMIT_REF_NAME != $MASTER_BRANCH || $CI_PIPELINE_SOURCE == "api"
when: manual
allow_failure: true
variables:
# Include git submodules
GIT_SUBMODULE_STRATEGY: recursive
CI_PARAM_TEST_SUITE: '{platform}.jinja2'
CI_PARAM_EXTRA: --nop
artifacts:
when: always
paths:
- "results/**"
reports:
junit: results/results-*.xml
after_script:
- rm -r "${DEPLOYPATH_TEST}"
script:
- |-
# Submit tests to lava server
RELEASE=$(ls ${DEPLOYPATH_TEST}/)
INSTALLSCRIPT_ABS="$DEPLOYPATH_TEST/$RELEASE/$CI_PARAM_MACHINE/fng-install.sh"
FNG_INSTALL_URL="${ARTIFACTS_HOST_URL}/${INSTALLSCRIPT_ABS#/*/}"
.gitlab-ci/submit_test.py \
--fng-install "${FNG_INSTALL_URL}" \
--name \
"Gitlab build test ${CI_PARAM_MACHINE} ${RELEASE} ${CI_PIPELINE_ID}" \
--results-path "results" \
--test-repo ${TESTS_GIT_URL} \
--test-repo-branch ${MASTER_BRANCH} \
--test-plan ${CI_PARAM_TEST_SUITE} \
${CI_PARAM_EXTRA} \
${CI_PARAM_PLATFORMS}
# --------------------------------------------------------------------------------------
# Stage: deploy
# --------------------------------------------------------------------------------------
.deployimage:
extends:
- .infrastructure
- .package
rules:
- if: $CI_COMMIT_REF_NAME != $MASTER_BRANCH || $CI_PIPELINE_SOURCE == "api"
when: manual
allow_failure: true
script:
- echo
artifacts:
paths:
- release/**/**/*
reports:
dotenv: package.env
# --------------------------------------------------------------------------------------
# Stage: uploadftp
# --------------------------------------------------------------------------------------
.uploadftp:
variables:
CI_PARAM_PACKAGE_FTP: "true"
extends:
- .infrastructure
- .package
rules:
- if: $CI_COMMIT_TAG
when: manual
allow_failure: true
script:
- echo
.uploadsdkftp:
variables:
ARTIFACTS_SDK_PATH: "$LOCALDIR/$BUILD_MACHINE/sdk"
# --------------------------------------------------------------------------------------- # ---------------------------------------------------------------------------------------
# Actual jobs include:
# --------------------------------------------------------------------------------------- - project: '${CI_PROJECT_ROOT_NAMESPACE}/yocto/infrastructure/gitlab-ci'
ref: 04cd24f23863acb3ff54535ec166c0c7734abe4e
build:imx6guf: file: 'manifest.yml'
extends: .buildimage
stage: build
variables:
CI_PARAM_MACHINE: imx6guf
build:imx6ullguf:
extends: .buildimage
stage: build
variables:
CI_PARAM_MACHINE: imx6ullguf
build:imx8mguf:
extends: .buildimage
stage: build
variables:
CI_PARAM_MACHINE: imx8mguf
build:imx8mpguf:
extends: .buildimage
stage: build
variables:
CI_PARAM_MACHINE: imx8mpguf
build:imx6guf:fngsystem:
extends: .buildfng
stage: build
variables:
CI_PARAM_MACHINE: imx6guf
build:imx6ullguf:fngsystem:
extends: .buildfng
stage: build
variables:
CI_PARAM_MACHINE: imx6ullguf
build:imx8mguf:fngsystem:
extends: .buildfng
stage: build
variables:
CI_PARAM_MACHINE: imx8mguf
build:imx8mpguf:fngsystem:
extends: .buildfng
stage: build
variables:
CI_PARAM_MACHINE: imx8mpguf
# -------------------------------------------------------------------------------------
buildsdk:imx6guf:
extends: .buildsdk
stage: build
variables:
CI_PARAM_MACHINE: imx6guf
buildsdk:imx6ullguf:
extends: .buildsdk
stage: build
variables:
CI_PARAM_MACHINE: imx6ullguf
buildsdk:imx8mguf:
extends: .buildsdk
stage: build
variables:
CI_PARAM_MACHINE: imx8mguf
buildsdk:imx8mpguf:
extends: .buildsdk
stage: build
variables:
CI_PARAM_MACHINE: imx8mpguf
# -------------------------------------------------------------------------------------
deployimage:imx6guf:
extends: .deployimage
stage: deploy
needs: ["build:imx6guf", "changelog"]
deployimage:imx6ullguf:
extends: .deployimage
stage: deploy
needs: ["build:imx6ullguf", "changelog"]
deployimage:imx8mguf:
extends: .deployimage
stage: deploy
needs: ["build:imx8mguf", "changelog"]
deployimage:imx8mpguf:
extends: .deployimage
stage: deploy
needs: ["build:imx8mpguf", "changelog"]
deployimage:imx6guf:fngsystem:
extends: .deployimage
stage: deploy
needs: ["build:imx6guf:fngsystem", "changelog"]
deployimage:imx6ullguf:fngsystem:
extends: .deployimage
stage: deploy
needs: ["build:imx6ullguf:fngsystem", "changelog"]
deployimage:imx8mguf:fngsystem:
extends: .deployimage
stage: deploy
needs: ["build:imx8mguf:fngsystem", "changelog"]
deployimage:imx8mpguf:fngsystem:
extends: .deployimage
stage: deploy
needs: ["build:imx8mpguf:fngsystem", "changelog"]
# -------------------------------------------------------------------------------------
uploadftp:imx6guf:
extends:
- .uploadftp
- .uploadsdkftp
stage: uploadftp
needs: ["build:imx6guf", "buildsdk:imx6guf", "changelog"]
uploadftp:imx6ullguf:
extends:
- .uploadftp
- .uploadsdkftp
stage: uploadftp
needs: ["build:imx6ullguf", "buildsdk:imx6ullguf", "changelog"]
uploadftp:imx8mguf:
extends:
- .uploadftp
- .uploadsdkftp
stage: uploadftp
needs: ["build:imx8mguf", "buildsdk:imx8mguf", "changelog"]
uploadftp:imx8mpguf:
extends:
- .uploadftp
- .uploadsdkftp
stage: uploadftp
needs: ["build:imx8mpguf", "buildsdk:imx8mpguf", "changelog"]
uploadftp:imx6guf:fngsystem:
extends: .uploadftp
stage: uploadftp
needs: ["build:imx6guf:fngsystem", "changelog"]
uploadftp:imx6ullguf:fngsystem:
extends: .uploadftp
stage: uploadftp
needs: ["build:imx6ullguf:fngsystem", "changelog"]
uploadftp:imx8mguf:fngsystem:
extends: .uploadftp
stage: uploadftp
needs: ["build:imx8mguf:fngsystem", "changelog"]
uploadftp:imx8mpguf:fngsystem:
extends: .uploadftp
stage: uploadftp
needs: ["build:imx8mpguf:fngsystem", "changelog"]
# -------------------------------------------------------------------------------------
platformtest:imx6guf:
extends: .test
stage: test
needs:
- job: build:imx6guf
variables:
CI_PARAM_MACHINE: imx6guf
CI_PARAM_PLATFORMS: santaro santoka santino santino-lt
platformtest:imx6ullguf:
extends: .test
stage: test
needs:
- job: build:imx6ullguf
variables:
CI_PARAM_MACHINE: imx6ullguf
CI_PARAM_PLATFORMS: nallino
platformtest:imx8mguf:
extends: .test
stage: test
needs:
- job: build:imx6ullguf
variables:
CI_PARAM_MACHINE: imx8mguf
CI_PARAM_PLATFORMS: tanaro
smoketest:imx6guf:
extends: .test
stage: test
needs:
- job: build:imx6guf
variables:
CI_PARAM_MACHINE: imx6guf
CI_PARAM_PLATFORMS: imx6guf
CI_PARAM_TEST_SUITE: boot.jinja2
CI_PARAM_EXTRA: --all-devices
smoketest:imx6ullguf:
extends: .test
stage: test
needs:
- job: build:imx6ullguf
variables:
CI_PARAM_MACHINE: imx6ullguf
CI_PARAM_PLATFORMS: imx6ullguf
CI_PARAM_TEST_SUITE: boot.jinja2
CI_PARAM_EXTRA: --all-devices
smoketest:imx8mguf:
extends: .test
stage: test
needs:
- job: build:imx8mguf
variables:
CI_PARAM_MACHINE: imx8mguf
CI_PARAM_PLATFORMS: imx8mguf
CI_PARAM_TEST_SUITE: boot.jinja2
CI_PARAM_EXTRA: --all-devices
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment