--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c227e053

Yocto build: use common artifact pathspec for image and SDK builds

This is the first step on the way to a common pipeline for all build
configurations.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@fa2766e6

README: review documentation for adding new project

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c027f42b

README: use consistent heading style

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@f8bf107b

Sourcery: keep min-max-identity check disabled

IMO code is easier to read without the proposed change:
https://docs.sourcery.ai/Reference/Python/Default-Rules/min-max-identity/

Move the disabled check to the top of the list and add a comment that
all the checks below need to be verified.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@01484573

Refactoring: replace if-expression with or

See for reference:
https://docs.sourcery.ai/Reference/Python/Default-Rules/or-if-exp-identity/

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@ccdac3d5

Sourcery: enable more passing checks

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@d5908ee5

Refactoring: remove needless str() from print()

See for reference:
https://docs.sourcery.ai/Reference/Python/Default-Rules/remove-str-from-print/

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@479861fb

Sourcery: enable passing checks

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@7c87d4d1

Sourcery: skip use-assigned-variable

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@26591c0b

Refactoring: use datetime.now() instead of datetime.today()

now() is officially preferred over today() according to the docs.
See for reference:

https://docs.sourcery.ai/Reference/Python/Default-Rules/use-datetime-now-not-today/
https://docs.python.org/3/library/datetime.html#datetime.datetime.now

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@414ebe6d

Add python code analysis via sourcery

Disable all checks which do not pass currently. These should be enabled
in the future along with a code refactoring to make them pass.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@df817ecf

README: Add howto to create a new project.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@31e5a427

CI: package: Increase timeout to 90

We had timeouts during the upload of the artifacts
so I increase it to 90 minutes.

Prior versions of fngsystem used the overlays textfile, now we have an
overlays folder. This may disturb the copy overation so we delete it
in case it exists.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@23680f48

Fix check for branch protection state

Since the changes of 939ade2b9f98b3e1569332270a7ab16ce7d70b4a we observe
that pipelines are executed on non-protected branches, which is not what
we want.

GitLab's documentation was a bit unclear on the according CI variables
CI_COMMIT_REF_PROTECTED and CI_MERGE_REQUEST_TARGET_BRANCH_PROTECTED, as
stated in this stack overflow answer:
https://stackoverflow.com/a/59023344/3018229

We need to explicitly check for the variable values to be "true" instead
of just checking whether they are set.

GitBot authored 1 year ago and Tim Jaacks committed 1 year ago

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@939ade2b

Depend intgration pipeline execution on branch protection state

Previously we had the master branch hard-coded in a variable in order to
determine whether to run the integration pipeline (on non-master branch)
or the merge pipeline (on master branch). This variable had to be
adapted everytime we copy a branch, which can potentially be forgotten
and lead to problems.

Remove the hard-coded branch and use the branch protection state for
this decision instead in order to make this more flexible.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@dbc49ad0

Simplify yamllint job rules

The rules for the yamllint jobs have been defined in the .yamllint class
before. They were overridden in the manifest pipeline's yamllint job,
though, which has its own rules, so we don't need them to be inherited
in this case. And in the manifest integration pipeline the rules can be
much easier, so define them there instead of at the class-level.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@cd8de4ff

.gitlab-ci: add analyze stage for limiting YAML script blocks

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@e943a55e

Use project path instead of name in integration branch name

Usually project name and path are identical after project creation. It
is possible, though, to change the name of a project without changing
its path.
When triggering a pipeline on the integration branch, we are using the
CI_PROJECT_NAME variable to reference the project. Unexpectedly, this
variable contains the *path*, not the *name* of the project (see
https://gitlab.com/gitlab-org/gitlab/-/issues/20030).
Hence we have to use the project path in all other places where the
integration branch name is constructed in our Python code.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@bd7025b8

Manifest: run automatic pipeline on tags

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@068749c7

Manifest build: add support for SSH_CONFIG variable

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@d67898a2

Manifest: skip build if "skip build" label is set

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@bffc6097

Manifest: switch from branch pipelines to merge request pipelines

This will make the manifest project support the "skip build" label.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@5ba1ac68

Revert "Skip build if "skip build" label is set"

This reverts commit 4db257d36052c74eabed7ef43a05d98d3998ffec.

It does not work on the manifest, because we do not run merge request
pipelines there, so CI_MERGE_REQUEST_LABELS is not set at all.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@4db257d3

Skip build if "skip build" label is set

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@9d430fe8

Manifest integration: do not cancel previous pipelines on master

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@8b08a083

Fix error handling on merge

GitLab API has changed:
https://gitlab.com/gitlab-org/gitlab/-/merge_requests/94164/diffs#diff-content-c201dae00b72426693ddcb40edf6005e7702e935

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a59a93a0

Package: Add needs to changelog, to package changelog also

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@8edbf2a6

Changelog generator: fix typo

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@e6ae06dd

Changelog generator: pass projects as arguments

Instead of hard-coding the project list in the changelog generator
script, make them passable via arguments. This way we can have different
changelogs in different envrionments. Hence we're adding the changelog
job in the ci-test environment as well.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@ed36b465

mirror_mr_pipeline: include pipeline-not-found message in output

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@5ee69ab9

mirror_mr_pipeline: fix error when no pipeline found

This case should trigger a new pipeline instead of reporting an error.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@e69d303e

Changelog generator: remove debug output

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@0264fe09

Changelog generator: get all merge requests

This fixes older releases not appearing in the changelog.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c657f31f

mirror_mr_pipeline: improve debug output

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@473f12b1

mirror_mr_pipeline: add main call

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@88708b64

Convert CRLF to LF

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@169b2fa7

Improve mirroring MR pipeline

Refactor three scripts into one in order to support triggering a new
pipeline in the case of a skipped MR pipeline. This was not possible to
implement with the previous three-script design. The single scripts
were not used in other places anyway.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@897ba059

Manifest integration: remove dynamic job generation

The dynamic job was only necessary because GitLab did not expand
variables in the "trigger:project" keyword:
https://gitlab.com/gitlab-org/gitlab/-/issues/347469

This has been fixed in GitLab 15.3, so we don't need it anymore:
https://gitlab.com/gitlab-org/gitlab/-/merge_requests/92346

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@ea1f3bad

cancel_pipelines: include downstream pipelines

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c8f97623

Add job to cancel outdated pipelines in merge requests

Everytime we push a new commit to a branch, a new pipeline is created.
Often the pipeline for the previous commit has not finished yet, and
GitLab does not automatically cancel it. It consumes valuable build
time, even if we don't need the results anymore.
Adding a MR pipeline job to search for previous pipelines on the same
branch and cancel them explicitly.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@bab6f811

Package: move SDK files to subfolder "sdk"

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@8bcaa426

scripts: Add delete_job_artifacts script to delete all artifacts from untagged jobs

For a given project all artifacts may be deleted. This was needed due to
space limitations on the gitlab server.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@28906544

Reduce artifact expiration of build jobs to 1 week

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@d404f257

Alphaplan: allow failure on import

Otherwise the parent pipeline is stuck in "running" state until the
manual action has been performed. See for reference:
https://gitlab.com/gitlab-org/gitlab/-/issues/361574

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@dd2a6d7d

Fix error message

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@04ac29ad

CI-Test: fix deploy-files job

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@22c519f1

Move FTP jobs to FTP runner

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@84d0616b

Add new deploy pipeline to Yocto

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@f0436868

Move reusable parts from CI-Test to common

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@3d3b4d09

Yocto: fix SDK package

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@37360288

Yocto: fix SDK build

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@2b6fc739

Package: add comment

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@95d6e368

alphaplan_fwr_import: add error handling

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@277d4af7

Package & deploy: do not allow missing dotenv file

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@3c68ccab

Alphaplan: add script for importing alphaplan data

The script has mostly been copied from alphaplan_fwr.py, but with
different input data.

Adding a job to the CI-test environment for testing this.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@3b3238a1

Alphaplan: add script for generating alphaplan data

This script has mostly been copied from alphaplan_fwr.py, but with
different input data (artifacts from the SoftwareStore deploy job) and
without actually importing the data to the Alphaplan database.

Adding a job to the CI-test environment for testing this.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@387b8184

Deploy: save variables in deploy.env

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@902c9276

Deploy: save rsync file list for later stages

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@15622149

CI-test: use caching between the pipeline stages

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a96a923e

CI-test: add deploy stages for SoftwareStore and FTP

Rename all stages to uppercase on this occasion, since we're adding
stages with names containing multiple words.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@056f1af1

Add package class and stage

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a1972724

Add script to query job logs filtered by status, tag, and log pattern

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@d501fed4

Update isort config

Tobias Kahlki authored 2 years ago and Jonas Höppner committed 2 years ago

The handling of custom devicetrees and devicetree overlays has changed.
Applied the changes made to the fng-install script to the fngsystem-
self-update script as well.

(cherry picked from commit af4918cb)

BCS 746-001157

(cherry picked from commit 1c52dfec)

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@9982711f

Move deploy jobs to deploy runner

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@e48b4c0b

package_release: Do not fail if a file does not exist

The build for the imx8mpguf hit the limit.

(cherry picked from commit 597c7a49)

(cherry picked from commit 401a744f)

Gitlab uses textfiles to present links when downloaded.
Handle these during file download.

(cherry picked from commit 70706773)

Tobias Kahlki authored 2 years ago and Jonas Höppner committed 2 years ago

The "target" parameter was colliding with the TFTP parameter. Removed the
shortcut (-t) for the target parameter to resolve the collision.
Also, some filesystem operations weren't respecting the TARGETDEVICE
parameter.

(cherry picked from commit 9d32efa8)

Also cleanup the output a little.
Make sure that the prepare_rtc script only affects Tanaro devices

(cherry picked from commit 3b84b29d)

In image-minimal some commands fail otherwise.

We have a multi-image system setup that can't easily be created with the
WIC tool. Because of this, we don't support WIC images at the moment.
Remove the layout file for now.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a5e27f6e

FTP upload: Add flag to hide FTP upload stage

This is a workaround to prevent customer releases being uploaded to our
public FTP area. It should be removed as soon as we support uploading to
different FTP target folders.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@543a2170

manifest_package: add release suffix

Customer specific manifest projects can now set the RELEASE_SUFFIX
variable, which then gets appended to the deployment folder names.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c763c43f

alphaplan_fwr: change variable "release_name_local" to "release_name"

Otherwise it can be assumed that it might be correlated with
outputdir_local, which is not the case.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@2d8d26a5

package_release: more refactoring

- Make output-dir an argument that can be passed multiple times instead
  of hard-coding two dirs with certain meanings.
- Remove LOCALDIR variable from package.env, because it is not used
  anywhere (only occurence was job class ".uploadsdkftp" which wasn't
  used anywhere either).

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@dc3698c8

package_release: refactoring

Improve readability of package_release.py and its submodule
generate_release_metadata.py.

- Add describing comments throughout the file.
- Remove duplicate code from functions. Simplify them, so that they do
  what their names say (and nothing more).
- Rename variables and function arguments to make their purpose clearer.
- Harmonize variable names with their corresponding command line arguments.
- Add pydoc for functions.
- Add type declarations to function arguments.
- Generate MD5 and metadata files in the source directories so that they
  can be copied along with the rest of the files.

The Medium Quality Sound (MQS) interface on the Nallino was patched to reduce
unwanted noise during audio playback. The service triggers some initialization
routines in the audio stack by playing a silent sine wave.

Original commit by Sam Crossman <sam.crossman@seco.com>

BCS 746-001052

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c831d1da

get_integration_sources: check if CI is enabled

Trying to query CI variables on a project which does not have the CI
feature enabled leads to an HTTP 403 (forbidden) error.
Check whether CI is enabled using the "jobs_enabled" flag.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@5e3dfaea

CI:deploy: fix fngsystem deployment path

The match to detect fngsystem still contained 'guf'. Changed to 'seco'.

BCS 746-001043

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@f8d6de86

Make check error message more verbose

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@21e7edeb

Improve ci-test build

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@76bd81d8

Fix check job rules

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a6721243

Move check job to parent pipeline

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@593784e1

Refactor check_if_integration_branch_is_up_to_date

- Use "source" and "target" terminology in the script, just like in the
  GitLab CI variables.
- Spefify data types for function arguments.
- Rename function arguments to better reflect their data types.
- Move determination of integration branch into main function, so that
  the successfull status message can be more verbose.
- Remove parent_merge_request feature and checking for gitlab-ci
  integrations, because the check job does not run on gitlab-ci
  integration branches at all.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@45b49ee3

Remove build stage on master

We don't need to explicitly trigger a build after merging changes in
gitlab-ci. A master pipeline on the target projects runs automatically.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a0f7e84f

Rename pipeline stages to reflect YAML filenames