GitBot authored 2 years ago and Tim Jaacks committed 2 years ago

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@686323c6

Remove yaml redundancies in manifest-integration

Leave all common yaml parts in manifest-integration.yml and add
manifest-integration-yocto.yml and manifest-integration-ci-test.yml
containing the different variable assignments for each environment.

The repos including these files have to be updated with the new names.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@4c751096

Harmonize manifest-integration yaml files for yocto and ci-test

This is the first step for reducing code duplicates.

BCS 746-000807

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c877f9c9

retrigger: Allow to specify the manifest's remote identifier in a variable

Costum manifest may override this variable

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@d9660a80

build-jobs.jinja2: Specify master branch variable again

If the MASTER_BRANCH_MANIFEST variable is set in the manifest's
.gitlab-ci.yml file, it may not find its way to the generated job,
if it is not explicitly set during generation.
That is done with this patch.

GitBot authored 2 years ago and Tim Jaacks committed 2 years ago

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@2e6c1b4f

Build jobs: Use parent-child jobs to generate needed build jobs

* Move all build, deploy and test stubs to the manifest-build.yml
* Create a new job to generate and trigger the build jobs dynamically
* Add the base jinja2 file for the build jobs.
* Add initial docs for the manifest pipeline
* Remove unused files

GitBot authored 2 years ago and Tim Jaacks committed 2 years ago

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@1bc683d5

Move python scripts to subfolder

BCS 746-000808

GitBot authored 2 years ago and Tim Jaacks committed 2 years ago

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@e6980d48

Update meta-seconorth-dual-espresso path

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@83002e07

accept_merge_request: explicitly check if MR has merge conflicts

According to the documentation [1] we should get a 406 error from the
API if we try to merge a merge request which has a merge conflict. We
are experiencing a different behaviour, though, where we are getting a
405 error in this case. There is an open GitLab issue on this topic [2].

In our error handling we assumed that the reason for a 405 error is most
likely a required pipeline which is still running. This assumtion,
however, leads to wrong behaviour in cases where no pipeline is started
at all.

Fixing this now by explicitly checking for merge conflicts at the very
beginning of the error handling sequence.

[1]: https://docs.gitlab.com/ee/api/merge_requests.html#merge-a-merge-request
[2]: https://gitlab.com/gitlab-org/gitlab/-/issues/364102

GitBot authored 2 years ago and Tim Jaacks committed 2 years ago

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@0885629d

Document "skip build" label

GitBot authored 2 years ago and Tim Jaacks committed 2 years ago

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@5e514076

Add possibility to skip MR pipeline check via label

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@ee151673

Add possibility to skip MR pipeline build via label

GitBot authored 2 years ago and Tim Jaacks committed 2 years ago

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@ad9d367e

Allow check job to fail

This makes merging a project MR possible even if the manifest has
received new commits since the MR pipeline build has run. The developer
is responsible for deciding whether it is safe to merge it without
having a build based on the latest manifest.

BCS 746-000794

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@81c2ad85

merge_into_manifest: fix error message if MR already exists

GitBot authored 2 years ago and Tim Jaacks committed 2 years ago

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@6fbbf338

CI: Add fake lava test to ci-test, reduce fetch depth in update submodule

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@d820110c

CI: Add variable containing the port for ssh access of the gitlab instance

Our server installation uses port 4444 for ssh access. The lava tests
didn't took this into account.

GitBot authored 2 years ago and Tim Jaacks committed 2 years ago

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a06956de

Rename layers to seconorth

GitBot authored 2 years ago and Tim Jaacks committed 2 years ago

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@67b0195a

CI: Increase infrastructure timeout to 10min

GitBot authored 2 years ago and Tim Jaacks committed 2 years ago

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@2d2210b6

CI: deploy-gitlab-ci: Don't stop integration when one project is already updated

Prior to this change the integration stop when one of the projects had
already the latest ref of gitlab-ci in to submodule.

GitBot authored 2 years ago and Tim Jaacks committed 2 years ago

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@6f09c66c

CI: deploy-gitlab-ci: Don't stop integration when one project is already updated

Prior to this change the integration stop when one of the
projects had already the latest ref of gitlab-ci in to submodule.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@6c39166f

CI: deploy-gitlab-ci: Don't stop integration when one project is already updated

Prior to this change the integration stop when one of the projects had
already the latest ref of gitlab-ci in to submodule.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@aaac8d1c

changelog_generator: Use logging instead of print, fix black

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@79afb233

CI: Move changelog_generator to gitlab-ci project

The script can also be run in the normal python container, so this
slightly reduces the complexity.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@383a5aca

Added qt-multi-screen-compositor to project list

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@3e88ad8e

buildfng: Moved the image and distro variables to global

The image and distro for the buildfng job can now be set via
global/external variables.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@751fc360

Improve CI documentation

BCS 746-000636

GitBot authored 2 years ago and Tim Jaacks committed 2 years ago

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@bcb6076b

CI: Add check if integration branch is on top of target branch before reusing it

The integration branch was reused even if there where new commits on the
target branch, which could lead into reverting some changes.
This adds a check, if the existing integration branch is directly based
on the target branch and deletes it if not.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@3b5e9be0

CI: Pull new changelog_generator image, add token to the call.

BCS 746-000785

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@968886c0

Update README.md: Add description on the gitlab-ci CI and how to deploy

GitBot authored 2 years ago and Tim Jaacks committed 2 years ago

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@f2b7ec90

CI: Remove build for dual espresso again

The pipeline is not setup to build from different manifest and so on,
and the dual_espresso manifest will probably go to a seperate repo
to keep it private.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@b81d573a

CI: Remove '--verbose' from deploy step

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c1ebf70c

manifest: Added manifest file selection to ci-test

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@1285aad7

manifest: Added manifest file name to integration script

BCS 746-000518

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@dc6d7858

build: Added DualEspresso

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@0941262a

layers: Added nogplv3 and dual-espresso layer to CI/CD

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@6c3a8ff7

CI:package_release: Remove "GUF-" prefix from gitlab artifacts dir

BCS 746-000637

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@d809d630

manifest-CI: Refactor manifest pipeline files

* Split up build and package steps in separate files
* Add additional comments
* Add log output

BCS 746-000637

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c917791b

Fix remote name on retrigger stage

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@1ba52853

Move repositories to git.seco.com

Rename variable JENKINSGUF_SSH_PRIVATE_KEY / SSH_PRIVATE_KEY to
GITLAB_PRIVATE_KEY on this occasion, because it contains a private key
that was generated exclusively for this use case. The according public
key has been added as a deploy key in GitLab to all repositories that
this repository needs access to.

Add more detailed documentation concerning this configurationdirectly in
the gitlab-ci files.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/3e948f5db0c926a1226746fd597b5a53163a0796

manifest: check if docker build folder is empty

BCS DevOps-000035

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/ab3189781f0b0e97c23c7719a5fdbccbf0c4db39

CI: integrate_into_manifest: Fix integration for recipes in SRCREV file

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/976aaec9d9c6ef10defd4b3a7329d366452bd522

CI: deploy_gitlab_ci: Fix merge, integration branch was used after delete

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/65c6f273b63e4d1beca064f6edb3073321fd5cea

CI: Rename foobar to ci-test, like the job names

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/60c311abd4930bf490f13ab13bf2534e160bf842

CI: Reuse existing integration branch preserving manual changes

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/3318487d518476135ce978fe3fa3a113ca29295f

CI: integrate_into_manifest: Fix bug introduced when splitting functions

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/4c74c580f08a1432d7bd89cf2967abbb7e071bf4

CI: disable 'check' job for gitlab-ci integration as integrate is already disabled

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/5d1fb5c4a3613690375c76cb97204eddf9dcd4ac

CI: Enable MR pipeline for integration of gitlab-ci again

Our settings refuse to merge if no pipeline has been run.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/69d604a4fe09fefe779eb1599da7ff29d86c7c0c

CI: Fix merge pipeline

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/31d6d7dad54ab322345a6f44b3ff913341500f1c

CI: deploy_gitlab_ci: Rework the deployment of gitlab-ci into the child projects

The deploy_gitlab_ci now creates the integration commit and branch in
each passed subproject and create an integration commit in the manifest
containing all these new revisions.
A build is then triggered on this commit to test the functionality.

Split the update_submodule functions to reuse them in different ways.
Remove some previously used files.

BCS 746-000740

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/0a7d845ea64bf85363f1ce58229223bb0f9ac0d1

mainfest-CI:package_release: Increase timeout for ftp upload

Added additional log output for sdk upload.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/5cb383e6e87bb5860dcfd9a3d2b4726e8202dd42

alphaplan_fwr:manifest-CI: Cleanup

Remove alphaplan_fwr debug code and run black and pylint.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/adddfca255410c5b211631cef79a604c17b3ddaf

package_release: Move metadata generation function to a separate file

To improve the readability of the package_release.py script.

BCS 746-000734

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/5a60fd6a844e2f161aee2aa1121974b7a300b922

package_release: Improve md5sums.txt file creation

Now md5sums.txt can be assembled from different copy processes.
The build md5sums file is not overwritten by the license copy process
anymore.

BCS 746-000637 746-000734

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/68f1959f3948d0b1ed18a6d9229eb4efa5730545

package_release:manifest-CI: split alphaplan FWR generation and package_release

The alphaplan FWR generation is now separated to make the manifest gitlab-ci
file more easy to understand and reduce the complexity of the
package_release file.

BCS 746-000637

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/5c3ac7a26ba751f31a2832f06778f5135b07f021

CI: merge_into_manifest: Fix detection of gitlab-ci integration branch

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/3dbeefbdf9c50c2ad55e8e39ebfe3b7eb983e93f

CI: merge_gitlab_ci: Remove deletion extra deletion of integration branch.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/227bccb6f9def1cde91174afef585c8ffdc12e1b

CI: Debug messages

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/98c8fdf5daee69b0ed21c357c9bc5502faa6b7d7

CI: merge gitlab-ci: manifest is merged at last

The merge job of a subproject include in the merge request merged by
gitlab ci, should not merge the integration branch in the manifest by
themself. This is now disabled by checking the source branch name.

This allow to merge the manifests commit at last.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/e8c2f2214402955c1dc3fd2b43e5e2a642545d03

CI: Enable analyze stage again

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/c7f8bf0db4e4eecc12b5710fb0f9c8bfceba4e2f

CI: Integration job is now optional, remove check job

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/55e34b981856d953a0dc0b73c1f3bda608cdbbf9

CI: Rename gitlab-ci jobs

Tim suggested to use the same name scheme for all jobs and use the group
names yocto and gitlab-ci as base.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/3f293145ca987e3ea3bfb7b1530485167d011a77

CI: merge_gitlab_ci: Fix for merge.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/2aeb899ae3a3ad32ccac3b15f7025d42de5d5dac

CI: merge_gitlab_ci: Fix some post merge issues

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/7381d5983371f0bedd89d6c40db6c600932909e6

CI: merge_into_manifest: Fix already merged detection

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/5bb84a04cd6de815350f7dbcf59dd04c2b94a6f7

CI: Merge_into_manifest: Fixes for merge detection

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/285ce1951cc2ce2f6a0c390a6ece5042d930e95a

CI: Merge_into_manifest: Fixes for merge detection

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/ce59c40987d7a247d6e0eae713fc806237ace207

CI: deploy_gitlab_ci: Move one-time code to own function as reference

Initially the moved code was needed to create the files including the
new implementation.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/b86fe23805678b5183fe3411dda3a42d697cbe38

CI: merge_into_manifest: Check if a commit is already merged

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/daa4cbeed2fad173a436d3d5c484b43dbddaa25a

CI: Move the loop to check if a revision is related to another to common

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/3e491ba0735c42e431609468a1b06a212e5905c4

CI: Add new file to query the revision for a project from the manifest.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/633e40df8078a6ffcb7b937ac0ecec66e78d485d

CI: Remove foobar projects from the yocto project list

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/7ce0fd7430aa162965ea7fa411806b8379ebf220

CI: merge_gitlab_ci: Improve error handling.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/04cd24f23863acb3ff54535ec166c0c7734abe4e

CI: Remove debug, enable yocto manifest deploy

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/89bed7e6841bb0bba9a8dcc221060b8f772f59c2

CI: Add function to merge the gitlab-ci MRs

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/b9cb9a29a80973721956fb6c153c5bc9bfb51dde

CI: Add function to merge the gitlab-ci MRs

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/25d7cabdb130955c0754c3742a5be2e4b747c491

CI: Harmonize foobar and yocto CI files

Also set yocto deploy job to manual to test merge.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/2cdf882e1e25e6c9e072d3b4e4160a3f3a1b6c54

CI: Enable analyze stage again in .gitlab-ci.yml again

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/26bd3edfe3ba62745cb5484422b366cccd609aab

CI: Add common.yml containing common code for manifest and integration.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/8bfa443a1550db2b464166f0a69db6f01bcb6c0b

CI: Use only one stage for integration to allow parallel execution

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/bb392f5037b495a3c9f7dfa3ea7b340cf0163754

CI: Rewrite check job python file to use gitlab api (no clone anymore)

For projects like the kernel the git checkout needs a log of time and
space. Directly using the gitlab api only loads the infomation needed.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/e60e50b1ff159331a257fb02fa77b2f38952c526

CI: Limit clone depth to 1 for speed

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/56b4cacb6f2a62b0dc325275585fe6673d453ba3

CI: Adapt manifests-integration.yml and manifest.yml for yocto projects

Integrates the changes done in the foobar- yml files to the yocto
projects.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/7cc7fa7701ec2713d3af7e8e21563163e600b4d7

CI: Add all yocto projects to the job list

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/b097a25350e1b4192c0b84aa6e7bb5a77477e3b6

CI: Generate .gitlab-ci.yml with include commands in deploy step

This change should be reverted after once used.
It is needed to initially add the include .gitlab-ci.yml in
all subprojects.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/4f194e2141fc72023dcf0ace4651475839392f5b

CI: Check: integration branch may have multiple commits

As all projects are commited in the same branch the 'up-to-date' check
may not only check if the first parent commit points to the
master/dunfell branch. Now it is needed to loop through the history
until the integration branch's commit is found.

On fail a message is displayed which merge request needs to be
retriggered manually. This can now also be the 'parent'-MR that
triggered the complete chain.

The check job is used pipeline again.

The retrigger job also looks in the .gitlab-ci project for check jobs to
retrigger.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/873ae97fa52ff62e515cf7681a383ed9a9ef608b

CI: Disable foobar manifest build for integration branch

BCS 746-000646

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/e081a8d8d4d4b6096f2b64fe450fa0d50de159b2

CI: Create integration branch optionally also when no submodule change is detected

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/4452c5cbe26b80b4559c95543c6258cbd8382f25

CI: Implement that a merge-request may also be specified with project!number

Before only the ID was used but the CI_OPEN_MERGE_REQUESTS variable has
the complete path.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/e86d320a42e60f253a534a66e8cee4afe93b30e2

CI: Multideploy: Detect exiting integration branch

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/0ee24df09955c771ab2d06591ab98873a1bc115a

CI: Deploy: create integration branches and trigger pipelines in subprojects

Allow deploy_gitlab_ci to change multiple projects at once.
Use it to create integration branches and merge requests in all
projects.

Add a python file to generate a job yml from jinja2 template.
Add a template for the jobs to trigger. These execute the actual
integration in all 'subprojects'.
Create the yml file in the deploy job and trigger it in a new trigger
job.

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/6b6d57180c086b313399d268605e39f903d96358

CI: Add common gitlab-ci file for all yocto repositories.

Create a merged version of the gitlab-ci files from the layers and the
repositoried referenced by the SRCREV file to be included in the
projects, to replace the project-individual implementation.

BCS 746-000646

--

Commit: https://gitlab.com/SECO-Northern-Europe/yocto/infrastructure/gitlab-ci/-/commit/369e59b3f333487c032a21a69eb4f1445f821c52

CI: Update include file's ref in gitlab-ci.yml when deploying changes

Add update gitlab-ci file with function to adapt the include ref to a
given revision.
Add a 'pre-commit-hook' to the update-submodule function.
Adapt deploy_gitlab_ci to use these to update the include statement in
the base project .gitlab-ci.yml to use the same ref as the submodule
is set to.

BCS 746-000646

BCS 746-000629