--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c831d1da

get_integration_sources: check if CI is enabled

Trying to query CI variables on a project which does not have the CI
feature enabled leads to an HTTP 403 (forbidden) error.
Check whether CI is enabled using the "jobs_enabled" flag.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@5e3dfaea

CI:deploy: fix fngsystem deployment path

The match to detect fngsystem still contained 'guf'. Changed to 'seco'.

BCS 746-001043

The USDHC busses had been rearranged, because we have a different pin mapping
on the MC3 (as compared to the Trizeps modules and the DE).
We now have a boot bus detection, which sets the correct bus for the boot
scripts. This makes the rearrangement obsolete.
Removing the rearrangement also fixes an error in the save function for the
environment, which mixes the boot bus with the device tree definitions.

BCS 746-000814

The boot bus detection sets an environment variable that contains the USDHC
bus, from which U-Boot was started. The variable is used in the search for the
boot scripts and prevents U-Boot from mixing up the eMMC and SD-Card.

BCS 746-000814

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@f8d6de86

Make check error message more verbose

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@21e7edeb

Improve ci-test build

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@76bd81d8

Fix check job rules

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a6721243

Move check job to parent pipeline

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@593784e1

Refactor check_if_integration_branch_is_up_to_date

- Use "source" and "target" terminology in the script, just like in the
  GitLab CI variables.
- Spefify data types for function arguments.
- Rename function arguments to better reflect their data types.
- Move determination of integration branch into main function, so that
  the successfull status message can be more verbose.
- Remove parent_merge_request feature and checking for gitlab-ci
  integrations, because the check job does not run on gitlab-ci
  integration branches at all.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@45b49ee3

Remove build stage on master

We don't need to explicitly trigger a build after merging changes in
gitlab-ci. A master pipeline on the target projects runs automatically.

Marc-Oliver Westerburg authored 2 years ago and Jonas Höppner committed 2 years ago

This patch adds basic MC3 support to U-Boot.
Note: This currently only partially integrates with
Trizeps VIII Plus support and in several places still breaks
the Trizeps support.

This still needs quite a bit of clean-up, esp. regarding:
 * eMMC configuration
 * pun-muxing
 * separate DTs

Tobias Kahlki authored 2 years ago and Jonas Höppner committed 2 years ago

The pull-up of the CMD line of the eMMC isn't placed on the hardware.
Therefore, the internal pull-up of the pad is required, for the CMD line
to function properly.
Also increased the clock rate to 400 MHz (see NXP EVK) and assigned the
correct regulators (the eMMC uses fixed power rails for 1.8 V and 3.3 V).

Cherry-pick from linux-imx-kuk c8c74463 and 482dc0dc

Felix Gerking authored 2 years ago and Jonas Höppner committed 2 years ago

Boots fngsystem (kernel 5.15) via uuu

The modifications are taken from Marcs diff (US 746-000812). I just had
to reenable the FASTBOOT functionalities in
configs/trizeps8plus_defconfig. Keep in mind that the correct bl31
binary is required.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a0f7e84f

Rename pipeline stages to reflect YAML filenames

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@d25cdebf

Really fix potential error message in retrigger job

TypeError: 'NoneType' object is not subscriptable

Example:
https://git.seco.com/seco-ne/yocto/manifest/-/jobs/129201

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@3eef782e

deploy_gitlab_ci: integrate only correct projects into the manifest

Previously we integrated all projects into the manifest, even those that
belonged to a different manifest branch. That led to all manifests
having the same project revisions, even if they should be different due
to different intgrated project branches.

Save a list of integration sources for each manifest branch separately
now, so that we can check which project belongs to which manifest branch
later.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@2b851ac6

Fix potential error message in retrigger job

TypeError: 'NoneType' object is not subscriptable
Example:
https://git.seco.com/seco-ne/yocto/infrastructure/ci-test/minimal-manifest/-/jobs/127610

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c3fd0ecc

Check if variable is defined before checking it value

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c1be0f4b

Fix jinja2 "or" operator

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@8da534ec

Run integration pipeline for target branch only

When we're integrating a single branch into a single manifest branch,
there is no need to run the intgeration pipeline for all configured
integrations, but only for those that match the target branch of the
current merge request.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@8c94b725

ci-test: add job to check for correct foo branch

This should catch an error where during a gitlab-ci merge process the
wrong foo branch is integrated into the manifest, which happened in
Yocto.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@163c7e1e

Add yml extension to jinja2 templates

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@59523aa0

check job: disable verbose output

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@28bf0065

retrigger: add job states to log message

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@fd742017

deploy_gitlab_ci: fix target branch in integration merge request

We need to pass the correct target branch to the merge request creation
instead of using the project's default branch.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@38379173

Run yamllint job on gitlab-ci integrations

Otherwise no pipeline is created at all in these merge requests, which
prevents the MR from being merged.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@dce4f8a3

changelog_generator: Fix machine project ID

We have deleted the original project due to git corruption. Replace with
the new project's ID.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a7aa7c26

deploy_gitlab_ci: fix project argument passing

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@8a38f194

Put all manifest intgeration jobs into a single stage

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a09564c1

deploy_gitlab_ci: explicitly pass gitlab-ci project and branch

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@172e7625

Improve error messages

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@f0f5a370

Improve naming of functions and variables

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c8b484ca

Retrigger MRs from all integrating projects

Instead of getting the projects from the manifest file, search them via
their INTEGRATION variable.

Improve log messages on this occasion: print URL instead of just IDs.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@5e75bfc1

Deploy gitlab-ci to multiple branches

Add different integration jobs for different manifest branches. The
merge stage only has one job for each manifest, though, because
otherwise we cannot guarantee a consistent state for all branches.

Extend the deploy script for this purpose, so that it can deploy to
multiple manifest branches at once.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@771dcff1

Fix temporary dir path

We were passing the complete TemporaryDirectory object to the repo clone
function instead of just the path string, resulting in the repo being
cloned into a local dir "<TemporaryDirectory '/tmp/tmphwakypf8'>".

Fix this to actually use the generated temp dir.

This change makes it necessary to keep the TemporaryDirectory object
reference until we don't need the directory anymore, otherwise it will
be removed immediately.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@dce219c8

Print less important output in grey color

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@5e36261d

deploy_gitlab_ci: remove hard-coded integration targets

Get targets dynamically using the INTEGRATION variable instead.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@94cd6781

Recurse to child pipelines on retrigger

This makes it possible to retrigger jobs that are not part of the
given pipeline but in any nested child pipeline of it.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@fb1219a7

Ignore __pycache__ in scripts directory

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@0c29c995

Refactoring: rename arguments for consistency

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@d461090b

Remove hard coded project and branch variables

MANIFEST_PROJECT, MASTER_BRANCH_MANIFEST, MASTER_BRANCH_PROJECT

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a4aae414

Add rules for integration pipeline

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@bfa38b24

Improve log messages

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@1c9670d9

commit_and_push: do not pass branch

Branch can be implicitly determined via the repo.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@91dc115a

repo: do not exclude tags

We have created tags in our poky repo which are needed by repo, e.g.
https://git.seco.com/seco-ne/3rd-party/yocto/poky/-/tags/seconorth%2Fkirkstone-4.0.0

Do not explicitly exclude tags in order to fetch them.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@5c2defce

CI: prepare to work also with renamed machine names lime seco-mx6

Sven Hillger authored 2 years ago and Felix Gerking committed 2 years ago

(cherry picked from commit 7548225e)

Added the support for old and raw initramfs images to U-Boot.
Also added the driver for Realtek phys to U-Boot.


(cherry picked from commit 47242cec)

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@ef08f4ef

deploy: Increase deploy timeout to 30min

Especially when downloading the SDK the deploy takes longer than 10min.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@968aaadd

build-jobs: Remove dump_install_command from buildsdk job

When the dump_install_command script is executed in the SDK build step
an error occurs because the fng-install script/vars are not available.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@52bae4a1

Lava Testreporting: Add script to generate markdown report for given jobids

Adds scripts for mark down generation.
Adds scripts for lava test result queries.
Adds script to generate a mark down report from this data
Call script from submit_test to generate a report directly
after the testrun.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@ab09db0b

CI: Lava Test: Allow to install test image directly from gitlab

The images from the build job can be directly installed from gitlab.
To achive this some changes in the complete pipeline have been needed.

1. The variables used in the build job, like CI_PARAM_IMAGE, ... and
   related variables like BUILDPATH are only valid in the build job now.
2. The build job writes every variable needed in a follow up job into
   build.env. This also includes the url to the fng-install.sh of the
   final image.
3. The build.env file is used as dotenv artifact, as well as normal
   file artifact.
   The dotenv make the written variables automatically available in
   follow up jobs, that are using the aritfacts, like the deploy job.
   The normal file artifact is available via artifact download.
   (I did't found a way to download the dotenv file instead)
4. Some scripts have been added:
   - Find a job inside the pipeline by name, as the id is not known in
     advance.
   - Download all artifacts or one file of the artifacts from a given
     job
   - Download one file of the latest job by name
5. The scripts are used to download the build.env into the test job
   (where not artifacts are needed anymore)
6. The script is sourced and all variables are available inside the
   script.

Additionally this adds a fake build job to the ci-test pipeline, that
copies an image from srv73 and stores it as artifact in a way that a
test-job can run on it, like in the normal yocto pipeline.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@075aeccb

Refactoring: share build and test job basics for ci-test and Yocto

This shares YAML code for the following procedures:
- Build environment
- Source code checkout via repo
- SSH key and known hosts setup
- LAVA test submission
- Docker check if build folder is empty

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@6508e800

CI: Add hardcoded artifact's path generation also for fng system

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c503a6d4

CI: pipeline yocto: Change scope of CI_PARAM_IMAGE, ... to fix fngsystem build

The variables CI_PARAM_IMAGE, CI_PARAM_DISTRO, CI_MACHINE have been
defined outside the job in the '.gitlab-ci.yml' file. This has a lower
precedence then variables defined inside the job, but it seems that
these variables are used in the scope of 'Trigger variables' when
available in the trigger job. These override the value set in the
template. As main results this made FNGSystem jobs build the normal
yocto.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@ea3aa816

CI: Readd allow_failure:true to fix not finished pipelines

According to the docs a manual job always has allow_failure set to
true, but we see a lot of jobs staying in an unfinished state since
the removal of the allow_failure: true keyword for the sdk and test
jobs. This should fix the broken behaviour.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@dc0503b6

CI: Hardcode the artifacts path in the generated job description

Some gitlab update(?) seem to have changed the behaviour in
variable passing. In the scripts the variables are available
but in the artifacts path does not resolve them any more.
May be it has something to do with the introduction of the
trigger:forward keyword.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@f39e65ab

Manifest: really include yamllint in the full pipeline

Previous commit did not have the effect due to wrong extend order.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@0dd147c8

SDK: Remove package step from build sdk, add sdk to artifacts, create deploy_sdk

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@ecab2962

Manifest: include yamllint in full pipeline

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a4071423

Manifest pipeline: run full build pipeline when triggered via web UI

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@8862b213

Manifest refactoring: move job rules into classes

This way they can be reused instead of redefining them in each job.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@d604b330

Fix wrong remote name in ci-test pipeline

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@e810c2ac

Fix wrong manifest project ID for ci-test pipeline

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@9b8cf51c

Share manifest pipeline for ci-test and yocto

Combine all common yaml parts in manifest-pipeline.yml and add
manifest-pipeline-yocto.yml and manifest-pipeline-ci-test.yml
containing the different variable assignments for each environment.

This change implicitly introduces parent-child build job generation in
the ci-test pipeline, like it is done in the yocto pipeline already.
The ci-test build jobs have been moved to build-jobs-ci-test.jinja2
accordingly.

Furthermore rename GITLAB_CI_CURRENT_REV to GITLAB_CI_REVISION and
remove the run conditions from all generated build jobs, since these are
already present in the upstream trigger job.

The repos including these files have to be updated with the new file
and variable names.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c1b939fe

Use CI variables in build template directly

The job generation script implicitly passes the OS environment to the
template, so that the template has access to all GitLab CI variables.
Hence there is no need to explicitly pass any of them as command line
arguments.

This change makes the "generate-build-jobs" job more generic, so that
it can be shared with the ci-test pipeline in the future.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@775c09c1

Add gitlab_backup script to download all project exports of a gitlab group

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@7f68f307

Yocto manifest build: re-enable build:merge_request job

This job was accidentally disabled in 2e6c1b4, because it was included
in the generated child pipeline. This pipeline is not created on the
master, though, so the condition for the job was never met.

Move the job to the parent pipeline again, so that it gets executed on
the master like before.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@970c7fd7

Re-add yamllint stage to ci test manifest pipeline

Was accidentally removed in previous commit.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@2b2bc220

Define yamllint job explicitly in each pipeline

The yamllint job was included in the pipeline implicitly via the
inclusion of common.yml. This is not very transparent. Make the common
definition a base job and extend it explicitly in each pipeline now.

Add infrastructure stage comment header into both yaml files.

GitBot authored 2 years ago and Tim Jaacks committed 2 years ago

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@686323c6

Remove yaml redundancies in manifest-integration

Leave all common yaml parts in manifest-integration.yml and add
manifest-integration-yocto.yml and manifest-integration-ci-test.yml
containing the different variable assignments for each environment.

The repos including these files have to be updated with the new names.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@4c751096

Harmonize manifest-integration yaml files for yocto and ci-test

This is the first step for reducing code duplicates.

BCS 746-000807

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c877f9c9

retrigger: Allow to specify the manifest's remote identifier in a variable

Costum manifest may override this variable

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@d9660a80

build-jobs.jinja2: Specify master branch variable again

If the MASTER_BRANCH_MANIFEST variable is set in the manifest's
.gitlab-ci.yml file, it may not find its way to the generated job,
if it is not explicitly set during generation.
That is done with this patch.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@2e6c1b4f

Build jobs: Use parent-child jobs to generate needed build jobs

* Move all build, deploy and test stubs to the manifest-build.yml
* Create a new job to generate and trigger the build jobs dynamically
* Add the base jinja2 file for the build jobs.
* Add initial docs for the manifest pipeline
* Remove unused files

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@1bc683d5

Move python scripts to subfolder

BCS 746-000808

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@e6980d48

Update meta-seconorth-dual-espresso path

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@83002e07

accept_merge_request: explicitly check if MR has merge conflicts

According to the documentation [1] we should get a 406 error from the
API if we try to merge a merge request which has a merge conflict. We
are experiencing a different behaviour, though, where we are getting a
405 error in this case. There is an open GitLab issue on this topic [2].

In our error handling we assumed that the reason for a 405 error is most
likely a required pipeline which is still running. This assumtion,
however, leads to wrong behaviour in cases where no pipeline is started
at all.

Fixing this now by explicitly checking for merge conflicts at the very
beginning of the error handling sequence.

[1]: https://docs.gitlab.com/ee/api/merge_requests.html#merge-a-merge-request
[2]: https://gitlab.com/gitlab-org/gitlab/-/issues/364102

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@0885629d

Document "skip build" label