--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@3eeebfdc

Changelog: Include all related projects into the changelog

In build-pipeline.yml, enable the new parameter by default.

YT-256

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@03af7519

Changelog-generator: Add parameter to include all integrated projects

Before this, the changelog was generated on a given list of projects,
all using the same branch.
The new parameter now goes the the specified projects from the
parameters, and detects all integrated projects into this.
So normally you would specify one or two manifest projects, everything
else is automatically added to the list.

This also handles different branch names. As for example the kernel
branches are different, the integration branch is used instead of the
branch from the command line.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@37b396e8

YT-247: Automate Initiation of Build Stages in Scheduled Pipelines

  Enable automatic initiation of the build stage for the SDK
  and the test stage for pipelines within scheduled pipelines.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@e3a76466

scripts: Add simple python script to convert license.manifest to md table

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@f6b301dc

YT-227 Fix submit LAVA task for smoke tests

  Smoke tests use a single type of test (boot.jinja2).
  However, some parameters need to be dynamic, such as device type.
  To address this, the `test-plan-args-var` introduced
  what allows environment variables with extra parameters to be used
  in the LAVA job generation script.
  The pipeline has been adjusted accordingly.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@be2a04ab

Enable verbose mode for submit_test.py

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@cf0071f2

YT-234: Fix ci-test pipelines (was broken with YT-65 MR)

 The YT-65 MR refactored the repo_checkout job,
 which is also used in CI-test pipelines.
 Added the necessary changes to the CI-test code.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@667964a6

Fix typos

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@bd7e9366

YT-65 We perform a repository checkout in the manifest project for each build job.
  Each pipeline builds code for multiple MACHINES.
  This results in multiple downloads from each repository in the manifest.
  Occasionally, certain repositories may fail to fetch data.
  To avoid fetching the same data repeatedly for each build job in the pipeline,
  the `repo_checkout` stage is added.
  In this stage, the repo tool prepares the repository structure once
  and stores it in the cache.
  Subsequent build jobs use the pre-fetched data from the cache.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@de29fa3e

    YT-230: Fix Manifest Pipeline after "Improve Release" Changes

    Fix typo introduced in previous commit

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@417b79ef

YT-230: Fix Manifest Pipeline after "Improve Release" Changes

The recent changes introduced new rules in manifest-pipeline-yocto.yml,
which have overwritten the general rules defined in build-pipeline.yml.
Use !reference to combine both sets of rules effectively.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a7e07f19

YT-180 Generation of the firmware package failed due to a typo
  in the code that checks of existance of the variable.
  What caused  TypeError: unhashable type: 'list'

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@7bd78810

PBB-32 Fix suffix addition in FNGo pipeline for custom projects
 due to an issue in GitLab where escaped variables are not correctly
 passed to child pipelines, a workaround is required.
 As a result, we use raw variables.
 The DEFERRED_RELEASE_VERSION variable was introduced for this purpose.
 However, for custom builds, we have an additional place where we modify
 the DEFERRED_RELEASE_VERSION variable to add a suffix.
 Therefore, extra interim variables need to be added to handle the workaround
 and ensure we get the raw variable in the end.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@83f4f610

Take group path from group instead of options

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@93eabeb3

YT-157 Adjust GitLab Backup Script to avoid shared projects
 Ensure backups are only processed for direct member projects of a group.
 If a project is shared with a group, backing it up without the appropriate
 permissions can cause the backup to fail.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@77a91f92

YT-93 Improve release workflow

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@edbfded2

YT-164 Sort artifacts on confluence release page for FNG system

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@6ab45a4c

YT-190 Enable credential.useHttpPath for git credential manages in CI pipeline

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@516090fa

YT-167 Implement parsing full URL and multiple tokens

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@8abfbca7

YT-176 Skip projects with no permissions
  If a project have no permissions,
  it probably won't be processed in seco-ne CI pipelines.

  Exclude venv directory from version control

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@01d6973b

Scripts: Add script to delete all integration/ branches

It happens that the integration branches are left over from provious
runs of the pipeline, when MRs are closed without merging, for example.

Also a bug in the gitlab-ci makes the pipeline fail, when an integration
branch should be reused. The new scripts just deletes all integration
branch in the given project tree without further checks.

Needs to be run manually.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c05acbe1

Add scripts to setup a virtual python environment

Use:
```
cd script
source setup_venv.sh
```
to change to the venv.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@58554035

scripts:release_info: Deep copy dict in merge_variables

Without the deep copy, the merge operation manipulates the dict it is
storing the new values in.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@bd4caeae

report_image_diff: rework variables extraction

The current solution makes too much assumptions about the pipeline and needs
to reimplement some of the gitlab yaml interpretion.

The new version of the script extracts MACHINE and DISTRO from the jobs themselves:
- MACHINE is part of the build job name ("build-$MACHINE");
- DISTRO is printed by bitbake and can be found in build log.

Typos

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@cbf168eb

build:credentials: Fix error when storing credentials in cache

The credentials helper wasn't storing the credentials in the cache.
Seemingly, the echo added whitespaces at the beginning of the lines
which can't be interpreted by the git credential approve command.
Also the USERNAME was renamed to TOKEN_USER because the variable is
already filled with the system users name.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@d09e3a96

CI: generate_firmware: Add releasename as name to json file

To use the generated json file in the json-fwr tooling, it helps to fill
the name.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c2cd3b43

pipeline:confluence: Add parent for Flash-N-Go release page

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@887db0df

pipelines: Add wic.img files to supported artefacts

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@54f17a69

Update CI_IMAGES_REVISION

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@4fd7f8a8

report_image_diff: get rid of hardcode

Parse machines, pipeline names and distros from gitlab yaml file instead of
harcoding these values in the script.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@05d6a8db

fullbuildpipeline.py: fix timeout

The problem was found in pipeline:
https://git.seco.com/seco-ne/yocto/manifest/-/jobs/954246

Fix missing update of upstream pipeline status inside while loop.

The timeout in the script was not in sync with the default timeout for build jobs.
Set timeout in __get_build_pipelines() function to 8 hours.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@3077d252

handle_artifacts: add "try" block for keep_/delete_artifacts

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@b0be529b

handle_artifacts: fix case when pipeline was canceled

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@1a4a5ab1

Refactoring: move common code to separate files

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@bf6de920

convert_md2html: new font style for main text

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@dc43f865

convert_md2html: move content to left

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@f0088d25

convert_md2html: fix long commands

Several blocks of code are too long and don't fit on the page.
Words are not wrapped to the next line.
To fix this, adjust the span slacc in css.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@caa6efa2

convert_md2html: restore several css rules

- fix h1 color
- fix missing gorizontal line above h1
- increased h2 font size
- fix missing borders for table
- restore link style

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@28a1bc5e

convert_md2html: Update the style for the generated html file

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@6656c465

CI: allow failure in new jobs 'handle_artifacts' and 'report-image-diff'

It seems that these jobs are not yet completly errorsafe and we could
ignore the failure without changing the build artifacts.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@de9851be

ci: add a job to handle artifacts after merge

In the latest commit in the manifest it often happens, that the pipeline
does not contain the artifacts itself, but just a link to the pipeline of the MR.

Added a python script with corresponding GitLab CI job to launch this script.
The script searches for the latest pipelines for default branch in manifest repo,
and then decides either to keep or delete artifacts for particular build.

The build artifacts are kept for:
- the latest commit
- the last successful build
- the tagged commits (e.g. "fngsystem/47.0", "kirkstone/20.0")

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@1299a9d6

CI: add "report-image-diff" job

Add a python script and a GitLab CI job to run it. The job is triggered
when the build is complete.

The script retrieves build artifacts for the default/main branch
(currently "kirkstone") in the manifest repo and build artifacts
for the integration branch of merge request.

After that, it compares those builds and creates a summary of the overall
changes. In particular:
- size of compressed artifacts (artifacts.zip)
- size of built image
- difference between manifest files

The summary is reported to stdout and as a comment to the merge request.
Each run of the pipeline means removing the old summary comment and
creating a new one when the build is finished.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@dcc8d92b

build:hardware: Add Trizeps VIII Plus, fix MX93 and sort entries

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@add2bb17

Revert "CI: remove 'setup_ssh' as ssh access is disabled"

The SSH setup is required for some private projects that pull in
external repos via SSH.

This reverts commit 4d5bca8fb8aaebf8794ffa88a6d0aef779c9eedf.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@19ff9d08

package_release: Fix missing variable for os.walk

Even though the variable isn't used in the package_release script, it
seems to be required by the os.walk function. Underscore satisfies the
linter preventing it from causeing unused variable errors.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@6e6f124b

package_release: Add a way to copy the whole directory

Item should end with "/" to be treated as a directory

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@ab19eb61

CI: collect_release_information: jump to next pipeline if no deploy jobs found

See for error:
https://git.seco.com/seco-ne/yocto/infrastructure/ci-test/minimal-manifest/-/jobs/828261

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@ae764fbc

CI: Confluence page: Fix not created mx6 section

The function that merges the variables of the different jobs, i.e:

`MACHINE="mx6"`

`MACHINE="mx6ull"`

used python `in` to check if a value already exists.
But `"mx6" in "mx6ull"` is true, so mx6 was dropped.
This rewrites it, storing the variables in a set (which is unique
automatically) instead of space seperated strings.
Space seperation is added before printing the variables to be used on
on the shell.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@094983ae

git_config: Add "*" as safedir to omnit 'dubious ownership' error

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@44000de1

Check bitbake recipes with linter

The job checks bitbake recipes added/updated in MR using the
oelint-adv tool (https://github.com/priv-kweihmann/oelint-adv).

.oelint.json is a rulefile, created with the command
"oelint-adv --print-rulefile". It can be used during development.
It contains default rules for now, but will probably be customized later.

To get the list of changed files in MR, we adjusted the GIT_DEPTH variable.
The new value is 50, which should be fine for getting MR changes,
and still small enough to keep checkout fast for large projects.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@9d9574d1

convert_md2html: fix text duplication

Fix the problem introduced by commit 4f06516 "convert_md2html: fix id
fields for headings".

The text converted from markdown was written twice to the HTML document.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@4f06516a



convert_md2html: fix id fields for headings

Converter adds "user-content-" to "id" fields for headings.
As a result links from TOC do not work.
Fix this for now by removing "user-content-" from the resulting html.

Signed-off-by: Mikhail Vanyulin <mikhail.vanyulin@rtsoft.de>

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@97898e9e



convert_md2html: use gitlab api

Update convert_md2html.py to use gitlab api with gitlab flavor.

Signed-off-by: Mikhail Vanyulin <mikhail.vanyulin@rtsoft.de>

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@4595835d

build:machines: Add Genio510 to the build job template

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@d8b5ab7b

scripts:firmware: Fix check for U-Boot file types

On i.MX6 devices, the U-Boot file types are not set at all. Extent check for
the existance of the uboot_file_types var.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@3146ed1e

CI: increase timeout of package-documentation to 5m

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@4d5bca8f

CI: remove 'setup_ssh' as ssh access is disabled

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@8353d920

CI: lava testing: replace ssh with https url and CI_JOB_TOKEN

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@e207c443

gitlab-ci: Disable sourcery

The sourcery plugin fails again due to a missing login. Disable it for
now.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a3d1e77c

manifest:integration: Fix issue in trigger/include

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@b6e66192

pipelines:build: Add check to JSON generation

Without the check, we have a race condition when building from sources
outside of the Yocto group.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@42bfe4f0

build:prepare: Add setup script for Git Credentials

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@24653ff8

build:machines: Add i.MX93 to the build job template

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@7aae72ea

Re-enable sourcery

With the free open source plan, sourcery works only when called from the
repository's root directory. Re-enable it by calling it from there.

Also apply sourcery's suggestions to file generate_firmware_package.py,
which did not exist before disabling sourcery.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@51e0cf67

build:stages: Add Firmware Package JSON stage

The old Alphaplan stage is deprecated and was removed in a prior commit.
Add new stage to generate the Firmware Package JSON files that do replace
the old Alphaplan articles.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@1830cb02

And another improvement for the build-up-to-date job

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@f7f33de5

Improve the output message in the build-up-to-date job a bit more

Use explicit line breaks to make the message look better.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@68aaab94

Improve the output message in the build-up-to-date job

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@26b9eb1d

Rename "check" job to "build-up-to-date"

This is a more meaningful name and might give a hint what the job does.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@07f0c898

Make build runner tag configurable

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@0d70d719

Disable sourcery check

Sourcery changed its license model to only be available for open source
projects with the free plan. According to the documentation, sourcery
should detect automatically whether it is running on an open source
repo. This does not work in our case, though. Disable it until we find a
way to correctly make it work.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@9be2c52b

pipeline:build: Fix indentation of here-doc

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@245e6537

pipeline:build: Make generation of install script optional

The Genio 700 platform doesn't have an fng_install.sh. Currently, the
pipeline fails because of the exit code in the script check.
Make the generation of the install command optional and only show a
warning if the script is missing.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@8fedce8a

pipeline:build: Add MediaTek Genio 700 SoC

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@6f773de5

Remove Alphaplan stage

We do not use Alphaplan anymore. Remove everything Alphaplan related
stuff from the pipeline.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@56bd1900

Confluence: add helper script to get page content

This can be helpful if something is configurable in the GUI editor and
we can't figure out to do the same thing via the API.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c3b794c6

Confluence: enforce fixed page width on creation

On page creation via the API, the page always seems to be created with
full width, even if "full_width=False" is set. Update the page right
after creating it in order to correctly set it to fixed width.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@82053a2f

Confluence: set width for download table columns

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@96f5745c

Confluence: group download files by type

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@3697b4ea

cancel_pipelines: reverse order of cancellation

Cancel parent pipeline first before cancelling its children in order not
to have failed parent pipelines (trigger jobs are failed if their
downstream pipeline is cancelled).

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@5f4ca2fb

cancel_pipelines: list actually cancelled (child-)pipelines

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c80cba84

Cancel all pipelines on newer commit

We already canceled running pipelines before. Gitlab, however, might
have already canceled previous pipelines automatically, but without
canceling their child pipelines.
Remove the "running" status filter when cancelling pipelines, so that
also already cancelled pipelines will be included.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@84eb6e22

Revert "Cancel all pipelines on newer commit"

This reverts commit f7582b74e7c565d55bb8a0315fe75c4ab4385fa2.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@f7582b74

Cancel all pipelines on newer commit

We already canceled running pipelines before. Gitlab, however, might
have already canceled previous pipelines automatically, but without
canceling their child pipelines.
Extend the mechanism now to cancel all pipelines up to and including
"running" as well as "canceled" states.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@0ec55c07

Fix list() call in update_submodule

Here we don't just iterate over the results, so we have to use
"all=True" instead of "as_list=False". See previous commit for
explaination.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@4b3a517e

Make sure to get all items on Gitlab list() calls

The list() call on Gitlab classes uses pagination per default. If the
number of items exceeds a certain limit, we only get a part of the
results. We can pass "all=True" to the calls in order to disable
pagination and get all results.
If we're looping over the items, though, the recommended way of getting
them is to use a generator object, which works calling list() with
"as_list=False". See documentation for reference:

https://python-gitlab.readthedocs.io/en/v2.10.1/api-usage.html#pagination

Note: the "as_list" argument is deprecated in newer versions of the
python-gitlab library, in favor of the newer "iterator=True":
https://python-gitlab.readthedocs.io/en/v3.6.0/changelog.html#feature
We are using v2.10.1 of the library, though, so this commit also updates
calls using the new way, which is not working with our version.