diff --git a/README.md b/README.md
index 06f4b0c87fdbc05dbf5e51a893314f92a7ce7866..061ed773c561c06ee12bd163428b985cbc6bad42 100644
--- a/README.md
+++ b/README.md
@@ -2,21 +2,21 @@
# gitlab-ci Repository
<!----------------------------------------------------------------------------->
-This repository contains **GitLab CI scripts** for Yocto infrastructure tasks,
+This repository contains **GitLab CI scripts** for Edgehog OS infrastructure tasks,
e.g. automatic integration of project changes into the manifest. It is meant to
-be included into all relevant Yocto repositories as a [git submodule][1].
+be included into all relevant Edgehog repositories as a [git submodule][1].
[1]: https://git-scm.com/book/en/v2/Git-Tools-Submodules
<!----------------------------------------------------------------------------->
-## The EdgeBot token
+## The GitBot token
<!----------------------------------------------------------------------------->
Most scripts inside this repository need a [group GitLab access token][2] of
a user with access to all relevant repositories (manifest and all contained
projects) in order to work correctly.
-We have created the **EdgeBot** group access token for this task. The token has
+We have created the **GitBot** group access token for this task. The token has
maintainer permissions in the [`edgehog` group][3] and has all of the token
permissions enabled (read/write API access, read/write registry access, etc.).
@@ -33,12 +33,21 @@ The group access token value is provided via the CI environment variable
## Automatic manifest integration
<!----------------------------------------------------------------------------->
-See this chapter for information on how the automatic integration process works:
+See this chapter for information on how the automatic manifest integration process works:
â–¶ [Automatic manifest integration][5]
[5]: docs/automatic-manifest-integration.md
+<!----------------------------------------------------------------------------->
+## Automatic layers integration
+<!----------------------------------------------------------------------------->
+
+See this chapter for information on how the automatic layers integration process works:
+
+â–¶ [Automatic layers integration][8]
+
+[8]: docs/automatic-layers-integration.md
<!----------------------------------------------------------------------------->
## Deployment
diff --git a/docs/automatic-layers-integration.md b/docs/automatic-layers-integration.md
new file mode 100644
index 0000000000000000000000000000000000000000..d3b72e6175a29ae58088eca64afb9f4057c943df
--- /dev/null
+++ b/docs/automatic-layers-integration.md
@@ -0,0 +1,143 @@
+<!----------------------------------------------------------------------------->
+# Automatic layers integration
+<!----------------------------------------------------------------------------->
+
+**Contents:**
+- [Automatic layers integration](#automatic-layers-integration)
+ - [Source repositories used by BitBake](#source-repositories-used-by-bitbake)
+ - [Integrating BSP changes](#integrating-bsp-changes)
+ - [Merging project changes](#merging-project-changes)
+ - [Adding a new project to SRCREV.conf](#adding-a-new-project-to-srcrevconf)
+ - [Source code](#source-code)
+ - [Repository settings](#repository-settings)
+
+
+## Source repositories used by BitBake
+
+Some bitbake recipes, more precisely those regarding the Board Support Packages (BSPs) references, are set to
+`SRCREV = "${AUTOREV}"`. To enable reproducible builds, the revision for these recipes
+is written to the `SRCREV.conf` file contained in the `conf` folder of the
+corresponding meta-layer. For instance, the [meta-seco-rk][4]
+meta-layer recipes contain the reference to the Rockhip Board Support Package (BSP)
+repositories and the corresponding [conf/SRCREV.conf][5] file looks something like this:
+
+```ini
+SRCREV:pn-u-boot-seco-rk = "7d205eec8c6d3e6d3e08390b3c276f9fafb67489"
+SRCREV:pn-linux-seco-rk = "e6962a7c84438f36487ca8519f2b93b45efd59a6"
+SRCREV:pn-linux-seco-px30-rk = "645936464a1d6641c2bab5686045d4fce281fdd3"
+```
+
+Each project is listed with its bitbake **recipe name** after `SRCREV:pn-` and
+the fixed git **revision**.
+
+[1]: https://git.seco.com/edgehog/seco-manifest
+[2]: https://git.seco.com/edgehog/seco-manifest/-/blob/kirkstone/distro/seco-distro.xml
+[3]: https://gerrit.googlesource.com/git-repo
+[4]: https://git.seco.com/edgehog/layers/seco/meta-seco-rk
+[5]: https://git.seco.com/edgehog/layers/seco/meta-seco-rk/-/blob/kirkstone/conf/SRCREV.conf
+
+<!----------------------------------------------------------------------------->
+## Integrating BSP changes
+<!----------------------------------------------------------------------------->
+
+Everytime a developer pushes a change to one of the projects referenced in the
+recipes, the corresponding `SRCREV.conf` file needs to get a new revision.
+In order for this to happen, a similar process to that described in the
+[Automatic manifest integration][6] document occurs. This process has been automated
+as part of our **CI pipeline** and runs automatically as soon as a
+**merge request is created** in one of the recipe repositories (i.e. currently
+those in the [bsp group][12]):
+
+
+
+[6]: automatic-manifest-integration.md
+
+The pipeline running in the recipe repositories is described in the `layers-integration.yml` [file][11].
+
+[11]:https://git.seco.com/edgehog/infrastructure/gitlab-ci/-/blob/master/layers-integration.yml
+[12]: https://git.seco.com/edgehog/bsp
+
+The pipeline consists of three jobs:
+
+1. **Integrate-into-layer**
+
+ Create an integration branch on the meta-layer and update the project revision
+ on it.
+
+2. **Create-merge-request**
+
+ Create a Merge Requeste (MR) in the meta-layer from the newly created integration branch into the main one.
+ The creation of this MR automatically triggers all the CI jobs described in detail in the
+ [Automatic manifest integration][6] document.
+
+1. **Check**
+
+ Check if the integration branch is up to date with the current meta-layer
+ master branch (see [Automatic manifest integration][6] for why this is necessary).
+
+See the following diagram for a visualization of this process:
+
+
+
+The above workflow makes project changes **buildable** and **testable** in the
+full manifest environment **before merging them**. Actually, the user could even
+[prevent the project merge requests from being merged unless the pipeline has run
+successfully][7].
+
+[6]: https://docs.gitlab.com/ee/ci/pipelines/multi_project_pipelines.html
+[7]: https://docs.gitlab.com/ee/user/project/merge_requests/merge_when_pipeline_succeeds.html
+[8]: #merging-project-changes
+
+
+<!----------------------------------------------------------------------------->
+## Merging project changes
+<!----------------------------------------------------------------------------->
+
+As soon as a pushed project change is ready to be merged, a developer can merge it.
+This triggers a **pipeline** with only a **single job**:
+
+
+
+1. **Merge**
+
+ Merge the MR on the integration branch on the meta-layer created in the previous
+ step. This trigger a cascade process according to which a MR is opened and closed
+ in the manifest (see [Automatic manifest integration][6]).
+
+<!----------------------------------------------------------------------------->
+## Adding a new project to SRCREV.conf
+<!----------------------------------------------------------------------------->
+
+This workflow adds a gitlab pipeline to a project which automatically updates
+the git revision in the `SRCREV.conf` file on project changes.
+
+#### Source code
+
+1. Add the gitlab-ci repo as submodule to the project by using the correct
+ relative path:\
+ `git submodule add ../../../infrastructure/gitlab-ci .gitlab-ci`
+
+2. Add an approriate `.gitlab-ci.yml` file. For example, copy it from the
+ repository [linux-seco-rk][9]. Modify the following variables in the file:
+ * `MASTER_BRANCH`: the main branch of the project.
+ * `BB_RECIPE_NAME`: the name of the bitbake recipe that references to the project.
+ * `LAYER_PROJECT_PATH`: the path to the meta-layer repository that contains the recipe.
+
+3. Create a corresponding entry in the `SRCREV.conf` file of the meta-layer repo:\
+ `SRCREV_pn-<bitbake-recipe> = "<current-commit-hash>"`
+
+4. Add the project into the gitlab-ci project's `.gitlab-ci.yml`, so changes to
+ the CI are deployed into the new project.
+
+#### Repository settings
+
+1. Verify that the merge strategy is set to *Fast-forward merge*
+ (General -> Merge requests -> Merge method)
+
+2. Enable the CI/CD functionality for the repository
+ (General -> Visibility, project features, permissions -> CI/CD)
+
+3. Check that the default branch is protected and that *Maintainers* are
+ allowed to merge (Repository -> Protected branches).
+
+[9]: https://git.seco.com/edgehog/bsp/rockchip/linux-seco-rk
\ No newline at end of file
diff --git a/docs/automatic-manifest-integration.md b/docs/automatic-manifest-integration.md
index 1dee3fd68fa961454b5a01d211901b9e21ea98cb..bc90b66e588da4e0df8fd10093a7231a67a73a7d 100644
--- a/docs/automatic-manifest-integration.md
+++ b/docs/automatic-manifest-integration.md
@@ -4,48 +4,34 @@
**Contents:**
- [Automatic manifest integration](#automatic-manifest-integration)
- - [Defined source code state](#defined-source-code-state)
- - [Manifest](#manifest)
- - [External source repositories used by BitBake (Currently not used)](#external-source-repositories-used-by-bitbake-currently-not-used)
+ - [Manifest](#manifest)
- [Integrating project changes](#integrating-project-changes)
- [Skipping the build](#skipping-the-build)
- [Merging project changes](#merging-project-changes)
- - [Adding a new project to SRCREV.conf](#adding-a-new-project-to-srcrevconf)
+ - [Adding a new project to Manifest](#adding-a-new-project-to-manifest)
- [Source code](#source-code)
- [Repository settings](#repository-settings)
-<!----------------------------------------------------------------------------->
-## Defined source code state
-<!----------------------------------------------------------------------------->
-
-### Manifest
+## Manifest
-Our [Yocto manifest][1] contains a list of projects which are used to build our
-Yocto distribution. A simplified version of a manifest file [`default.xml`][2]
+The [Edgehog manifest][1] contains a list of projects which are used to build
+Edgehos OS distribution. A simplified version of a manifest file [`seco-distro.xml`][2]
(here containing only two projects) would look something like this:
```xml
<manifest>
- <remote
- name="seco-ne"
- alias="origin"
- fetch="ssh://git@git.seco.com:4444/seco-ne/yocto"
- />
-
- <project
- name="layers/meta-seconorth-machine"
- revision="7aede82170ff7f92de0dcfeca89ed79849869214"
- remote="seco-ne"
- path="sources/meta-seconorth-machine"
- />
- <project
- name="layers/meta-seconorth-distro"
- revision="c402d855fc8d47e3742ada3b0a3fda4b9649e414"
- remote="seco-ne"
- path="sources/meta-seconorth-distro"
- />
+
+ <default sync-j="8"/>
+
+ <remote name="seco-repos" alias="origin" fetch="https://git.seco.com/edgehog" />
+ <remote name="seco-layers" alias="origin" fetch="https://git.seco.com/edgehog/layers/seco" />
+
+ <project name="meta-seco-edgehog-things" revision="901e92783ffb6c5a36fee99efd57cb0955b53a18" path="layers/meta-seco-edgehog-things" remote="seco-layers" />
+ <project name="meta-seco-edgehog-embedded" revision="8fe5d96c85605645260baa6e78b198a3b69f607e" path="layers/meta-seco-edgehog-embedded" remote="seco-layers" />
+
</manifest>
+
```
Each project is listed with a **name**, a local **path** where the [repo][3]
@@ -53,38 +39,28 @@ tool will clone the source code to, and a fixed git **revision**. We do not use
a branch name here in order to have a defined, taggable manifest state at all
times.
-### External source repositories used by BitBake (Currently not used)
-
-Some bitbake recipes are set to `SRCREV = "${AUTOREV}"`. To enable reproducible
-builds, the revision for these recipes is written to the `SRCREV.conf` file.
-This file is also part of the manifest repo and looks something like this:
-
-```ini
-SRCREV_pn-libmdb = "1442894df13d9b290cfb5d97183eb9a96c8e4eba"
-SRCREV_pn-egalaxi2c = "d2dfb014e8c2ed36801bd020894d8306f02ff146"
-SRCREV_pn-xconfig = "0fc1ea45b55e729d551bb7d40dd25fbde02ee1b6"
-```
-
-Each project is listed with its bitbake **recipe name** after `SRCREV_PN-` and
-the fixed git **revision**.
-
[1]: https://git.seco.com/edgehog/seco-manifest
[2]: https://git.seco.com/edgehog/seco-manifest/-/blob/kirkstone/distro/seco-distro.xml
[3]: https://gerrit.googlesource.com/git-repo
-
+[4]: https://git.seco.com/edgehog/layers/seco/meta-seco-rk
+[5]: https://git.seco.com/edgehog/layers/seco/meta-seco-rk/-/blob/kirkstone/conf/SRCREV.conf
<!----------------------------------------------------------------------------->
## Integrating project changes
<!----------------------------------------------------------------------------->
Everytime a developer pushes a change to one of the projects contained in the
-manifest or in the `SRCREV.conf` file, the project gets a new git revision. In
+manifest, the project gets a new git revision. In
order to have this change integrated into the manifest, the according `revision`
attribute has to be updated. This process has been automated as part of our **CI
pipeline** and runs automatically as soon as a **merge request is created**:
+The pipeline running in the meta-layers is described in the `manifest-integration.yml` [file][11].
+
+[11]:https://git.seco.com/edgehog/infrastructure/gitlab-ci/-/blob/master/manifest-integration.yml
+
The pipeline consists of three jobs:
1. **Integrate**
@@ -95,14 +71,14 @@ The pipeline consists of three jobs:
2. **Build**
Trigger a build on the integration branch, which contains the change.
- This uses GitLab's [multi-project pipeline][4] functionality, which adds a
+ This uses GitLab's [multi-project pipeline][6] functionality, which adds a
separate *Downstream* pipeline running on the manifest repository (seen on
the right).
3. **Check**
Check if the integration branch is up to date with the current manifest
- master (see ["Retrigger" job below][6] for why this is necessary).
+ master (see ["Retrigger" job below][8] for why this is necessary).
See the following diagram for a visualization of this process:
@@ -111,11 +87,11 @@ See the following diagram for a visualization of this process:
The above workflow makes project changes **buildable** and **testable** in the
full manifest environment **before merging them**. Actually, the user could even
[prevent the project merge requests from being merged unless the pipeline has run
-successfully][5].
+successfully][7].
-[4]: https://docs.gitlab.com/ee/ci/pipelines/multi_project_pipelines.html
-[5]: https://docs.gitlab.com/ee/user/project/merge_requests/merge_when_pipeline_succeeds.html
-[6]: #merging-project-changes
+[6]: https://docs.gitlab.com/ee/ci/pipelines/multi_project_pipelines.html
+[7]: https://docs.gitlab.com/ee/user/project/merge_requests/merge_when_pipeline_succeeds.html
+[8]: #merging-project-changes
#### Skipping the build
@@ -170,15 +146,11 @@ See the following diagram for a visualization of this process:
<!----------------------------------------------------------------------------->
-## Adding a new project to SRCREV.conf
+## Adding a new project to Manifest
<!----------------------------------------------------------------------------->
-**NOTE**: Currently the `SRCREV.conf` file is not used but is created in the
-manifest repository for automation purposes. In any case, the steps below shall
-be kept in mind to properly configure CI/CD on the yocto projects/manifest.
-
This workflow adds a gitlab pipeline to a project which automatically updates
-the git revision in the `SRCREV.conf` file on project changes.
+the git revision in the manifest file on project changes.
#### Source code
@@ -187,11 +159,11 @@ the git revision in the `SRCREV.conf` file on project changes.
`git submodule add ../../../infrastructure/gitlab-ci .gitlab-ci`
2. Add an approriate `.gitlab-ci.yml` file. For example, copy it from the
- repository [meta-seco-edgehog-things][8]. Modify the following variables in the file:
- * `BB_RECIPE_NAME`: Set the name of the bitbake recipe
+ repository [meta-seco-edgehog-things][9]. Modify the following variables in the file:
+ * `MASTER_BRANCH`: the main branch of the project.
+ * `MANIFEST_FILE`: the manifest file where the project is present (e.g. `distro/seco-distro.xml` for [meta-seco-edgehog-things][9])
-3. Create a corresponding entry in the `SRCREV.conf` file of the manifest repo:\
- `SRCREV_pn-<bitbake-recipe> = "<current-commit-hash>"`
+3. Create a corresponding entry in the manifest file of the manifest repo.
4. Add the project into the gitlab-ci project's `.gitlab-ci.yml`, so changes to
the CI are deployed into the new project.
@@ -204,7 +176,7 @@ the git revision in the `SRCREV.conf` file on project changes.
2. Enable the CI/CD functionality for the repository
(General -> Visibility, project features, permissions -> CI/CD)
-3. Check that the default branch is protected and that *Maintainers +
- Developers* are allowed to merge (Repository -> Protected branches)
+3. Check that the default branch is protected and that *Maintainers*
+ are allowed to merge (Repository -> Protected branches)
-[8]: https://git.seco.com/edgehog/layers/seco/meta-seco-edgehog-things
+[9]: https://git.seco.com/edgehog/layers/seco/meta-seco-edgehog-things
diff --git a/docs/create-recipe-mr.drawio b/docs/create-recipe-mr.drawio
new file mode 100644
index 0000000000000000000000000000000000000000..bd77700d89a73093f6c3a1a53b89933b01167687
--- /dev/null
+++ b/docs/create-recipe-mr.drawio
@@ -0,0 +1 @@
+<mxfile host="app.diagrams.net" modified="2022-12-13T12:09:38.820Z" agent="5.0 (X11)" etag="YEs0r12hDfQW9MvCnzrW" version="20.6.2" type="device"><diagram id="fUzyECxa7j4ng1UTdmFO" name="Page-1">7Vxtc6JIEP41qbv9oAUMIH6MJu7uXVKVS273dj+OMCoXBG8YTdxffzMIOMyMigi+JIlVu9BAg91PP93TM3gF+tPXzxjOJveRh4IrQ/Ner8DNlWEAzbDof0yyXEl0o2usJGPse6lsLXjyf6FUqKXSue+huHAiiaKA+LOi0I3CELmkIIMYRy/F00ZRULzrDI6RJHhyYSBL//E9MllJO4a2ln9B/niS3tnS0gNTmJ2bCuIJ9KIXTgRur0AfRxFZbU1f+yhgxsvMsrpusOFo/lwYhaTMBfeP33tfv7XixR0Jge4E17evP1pde6VmAYN5+oXTpyXLzAI4moceYlq0K9B7mfgEPc2gy46+UKdT2YRMA7qn082RHwT9KIhwci0Y9NmHyscYej590hsfUy/5UUiPIxgTdkkUEu6S1R+VxwRHz4g70nPYhykLYBynz5ObNdl5RsSdpDsQuymc9Fxb5kKtbVGZbMLUqguECXrlRKlJP6Noighe0lPSo61OBtQU4JkBX9ZgsZ30lAkHlK7tpCBNATrOda+dSDdSP+7hU13hUhtOmaPCYTxLvre4rylEN2iBgmiG8BZE6HsjwoPIGbkq99qug4ajmtxid4tuMYHkFt1WuMXUGvKKsTvQUOhdM8aie2EUUmHPg/EktzNn0yKYqR16xSClhsPLHznOk92f6bXJzs0rf+bNMosfKeSSzzafEIjHiOyGI/IKPCt7jvOMpXBMJsMogMRfFNlZ5a30Dg+RT5+YA4YQr10bFHXE0Ry7KL2M51NRkyFAzOgImlamkTQl8Mm/eHVEAVWcB4xRPX9BN8dsk94EtqbLAC5ZHK8OD3F2NJPQ+3PXXGK802g+r3i36ox3Jn+AhCAcJhJDA42yAMREeLKNPjqXuNY7xWjME8DecS1pEjHScFxnBFUSOi4rhny3CJhNQOBhoO2AQXXKB93zwoYpcH5ekB2ODVFT09iQy4iHeTyhkiGGoTtZVXIJp/scyf83Z6OMHvS81ghBMsdoLeRygL8xA1AaJkWEwcAfMzJyKTpoagE9RtY+HTddpwemvuexy3sYxf4vOExUMVzNmHESc1m9K+uG6ZqTKBZL9SxTpATEp5VdnLRH3gC2VXSo5ciJQ4VNo6nEoXfrDv9/59PZU3qxKtGgV5/k7EC3f3LyNTewnZ3UsAqkC6IGMaA7ol9LU4NIMpKmhqnBMBW4kerBPqbxT6Nfu39MuiG4fuqoWGS+WYoxNQEYWR3KQdzQjkkxoATFHNL1GTkucpU1/9CxTGsrg+xhWN3ZkIwLhpXt2ljN31W1XgS7xhM4Y5vzaXDtkoiH9R0couCBwjTtjw0jQqKpAvckEuwfzUngh9TQWQ9Uk51y2xtc395wrTgB6ls6dHU4y7aKzrIUA7SGYuDpT31mzf6+64Xwh4H/uAv/uv+q6McMfTKEz6hFv75PXXSSQbBkUIXZtzCNYONjDoKVRla1KGrkmYE1GAx6KrM6xhDY9gFgr9cR+RCS84Spovy81KzdFarq4D26wnQ6JV2hg4ZcIbeG3KQqa01xiapos4E3EX75Okrl8yIqmnBJPmfGz5kcMxc0PQl2KbFBH6hcbBjZILV2V3R2u+KYbVOt2Dhtazqo0DxNznhA2Kc2YgHIRZIwDt7Zb9tcy5xsCA3sbgFFOhB0lB5Ci5ocQVF9I2ilGZ06sXf8KbrNbfvdIDsdeIT+S9dyqoJH0KSbzlHRU3vb7pCufS20ojsnhYY4U6vrtUFD1NQwNBQzOu+9a39Y4WLZZpE1ulLd0ljTXu3hEp2fo3TUDrMrEDpqucF2dNQ6WmOGlfs0F0+r57UARgqKyrQqaWqaVku0l85jpqwMbvips3NMx1ISrVypSYld1NQ0bj5mys4zrdvCTJnelfsRjc2UqaGy3zKuS0g/xqm7BUUfA61qt0AHOzQ1TSOqtuFmGsm4Y1PQK6hkOEwno1ohnFIdg7255b0yCQDFZT2GbUhMoquW5Tc2Fabv19m89GLlxCwjcYNZvSe5Q1PTLFOiK3mBI8u8BXOEtRpqu8r9um8zb0XVEiXnRKzmWgrXhR+z6Rl603AzxXNU/PTYf7z93najcMSSdkT/+XJ7TZ9Ci0Y115diDpCe5ffkwUP0wle4IxxNmd2H0QJ9Uj3O5tr1Iwtt5A5LBj1QoN5uCvVZiF1KPXs2ZapZvUw1d2hqOIHka7XeQOFxunpCdGJt9YSkqWk4GG+inhAdkg/+TlVPGPs1Iy9i9jgP5Laj6Vwwt7S2BsytEc12qq942Dm8SDt4597sEIchlmW1Ne7PqJpTBL1ivdA0h8gNVHonNMZJ+byzunt7K9gMS2afo65gM1R9yh0t7aTqr9CV4lw9KL5evH9/6qP3ndcFVrFjZWXtRT6hHTejlVgW+TFWUJR0VubKg8cKkqaGiR3sNzysr4hJ6wt+oWW9lUzDKzDNsyozqi+ikwoLR9tWsDQNR5mB3Alyn99ljWEr3pg6ao0B5BmM/sobmj+qVkocPsG1tcnqx+yxYEz9kpY77hwnDmC/zxVTN//Gzlg1XkV9v8PgBS7ZcYLnKFMw8nHMLkevyJ2zFfmfPkoZuZQRkGsr2p6ql/06jUF3v6XdH6VM/q4yqPrrSGIpI2lqOndczM8onDEcJCeKP3FVue0paWoYDmaJyvYC2555VJ2q7Zl1rz/C7BDWFSbHncqsKy6tlDQ1HWYNv0p5ojAzTz27YB713bSqswQWF5La1pDk5y+MwgyG1u52csGGMurA0XzJxUtlfujstNkZ6G1hmVvlV6Tylzb3e/WSQgEuudPScchezEJ317+CvDp9/VvS4PZ/</diagram></mxfile>
\ No newline at end of file
diff --git a/docs/create-recipe-mr.jpg b/docs/create-recipe-mr.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..c5538ac9346b64bfaf0ec628fdd7e4b2b82b5f3d
Binary files /dev/null and b/docs/create-recipe-mr.jpg differ
diff --git a/docs/manifest-pipeline.md b/docs/manifest-pipeline.md
index 47a7f4a5887c416736bf98b91400dab17a97405c..49e2560d8f69325fbd842fefdd7dddee134177b9 100644
--- a/docs/manifest-pipeline.md
+++ b/docs/manifest-pipeline.md
@@ -4,7 +4,7 @@
The pipeline running in the manifest repository is the one that does the real
work, like building images, generating docs and deploying image to download
-servers and triggering tests in LAVA (TO DO). The pipeline running in the manifest
+servers and triggering tests in LAVA. The pipeline running in the manifest
repository is described in the `manifest-pipeline-yocto.yml` [file][1].
[1]:https://git.seco.com/edgehog/infrastructure/gitlab-ci/-/blob/master/manifest-pipeline-yocto.yml
@@ -19,8 +19,8 @@ whenever a merge request is opened on a project or on the repository containing
CI/CD scripts. Eventually the asynchronous build can be skipped in case the
`skip build` label is specified when opening the merge request.
-The `deploy` and `notify` stage job are currently triggered only on scheduled
-pipeline.
+The `deploy` and `notify` stage jobs are currently triggered only on scheduled
+pipelines.
<!----------------------------------------------------------------------------->
## Build jobs
@@ -32,7 +32,7 @@ files in the `boards` [folder][2]. To include another board within the compiled
one it will then be necessary to include at the beginning of the
`manifest-pipeline-yocto.yml` file the corresponding board configuration file.
-The build jobs either compile the Egdehog image and the bundle.
+The build jobs either compile the Egdehog image and the RAUC bundle for the update.
[2]:https://git.seco.com/edgehog/infrastructure/gitlab-ci/-/tree/master/boards
@@ -53,6 +53,18 @@ defined at group level][3]:
[3]:https://git.seco.com/groups/edgehog/-/settings/ci_cd
[4]:https://learn.microsoft.com/it-it/azure/cognitive-services/translator/document-translation/create-sas-tokens?tabs=Containers
+<!----------------------------------------------------------------------------->
+## Changelog
+<!----------------------------------------------------------------------------->
+
+Every time a build is executed on the manifest, the changelog is also generated
+from one of the scripts accessing the GitLab API in the [scripts folder][6]. In particular,
+the job makes it possible to obtain a file in `.md` format containing the titles
+and URL references of all the MRs merged into the various meta-layers. This gives an idea of the
+various major changes between different versions of Edgehog.
+
+[6]:https://git.seco.com/edgehog/infrastructure/gitlab-ci/-/tree/master/scripts
+
<!----------------------------------------------------------------------------->
## Notify Job
<!----------------------------------------------------------------------------->
@@ -64,43 +76,16 @@ Edgehog group. The notification is allowed by using an [Incoming WebHook][5],
saved as a [group variable][3]
[5]:https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/how-to/add-incoming-webhook
-
-<!----------------------------------------------------------------------------->
-## Test jobs
<!----------------------------------------------------------------------------->
-
-TODO
-
+## Release
<!----------------------------------------------------------------------------->
-## Job generation, parent child jobs (NOT USED)
-<!----------------------------------------------------------------------------->
-
-As we support images for multiple machines/architectures we are building a lot
-of different image variants, where each needs at least build, deploy and test
-job.
-To simplify the generation of all these jobs, without a lot of copy and paste
-and the possibility to configure the actual build images, we are using gitlab's
-dynamic-child-pipeline feature. [See gitlab docs.][1]
-
-[1]: https://docs.gitlab.com/ee/ci/pipelines/parent_child_pipelines.html#dynamic-child-pipeline-example
-
-There is a *'generate-build-jobs'* job, that creates a yaml file containing the
-pipeline with all needed jobs.
-There are the following CI variables in the 'generate-build-jobs' job controlling
-the content (make sure these are not set in a more global scope, as this
-would overwrite the settings in the generated yml):
-
-* `CI_PARAM_MACHINES`: Space separated list of machines to build for, like "santaro santoka"
-* `CI_PARAM_IMAGE`: The name of the image to build. If set to an empty string,
- related jobs are not created.
-* `CI_PARAM_DISTRO`: The name of the distro to build
-* `CI_PARAM_IMAGE_FNG`: The name of the fngsystem image to build. If set to an
- empty string, related jobs are not created.
-* `CI_PARAM_DISTRO_FNG`: The name of the fngsystem distro to build
+Only when the tag is executed will a final job also be executed that allows a [GitLab
+release][7] associated with that tag to be created. This job makes it possible
+to associate with the GitLab release all the links to the SW artifacts produced by
+the build jobs in such a way that they are easily accessible, and allows the
+previously generated changelog file to be associated with the release description.
-It uses a python script called `generate_job_from_template.py` to convert the
-`build-jobs.jinja2` to `build-jobs.yml`. This yml file is then used by the
-*'trigger-build-jobs'* job, to setup the pipeline described by it.
+[7]:https://docs.gitlab.com/ee/user/project/releases/
-
+
diff --git a/docs/recipe-mr-pipeline.png b/docs/recipe-mr-pipeline.png
new file mode 100644
index 0000000000000000000000000000000000000000..df396f8799ba95253b50ded3b22522ad8a1ae79d
Binary files /dev/null and b/docs/recipe-mr-pipeline.png differ
diff --git a/docs/weekly_pipeline.png b/docs/weekly_pipeline.png
new file mode 100644
index 0000000000000000000000000000000000000000..0b83f4b52c90d7386916237145eed293b9f6e200
Binary files /dev/null and b/docs/weekly_pipeline.png differ
diff --git a/old/manifest-integration_seco_ne.yml b/old/manifest-integration_seco_ne.yml
deleted file mode 100644
index 0c17db3eb7fa32297774b4676c24b55443b676be..0000000000000000000000000000000000000000
--- a/old/manifest-integration_seco_ne.yml
+++ /dev/null
@@ -1,144 +0,0 @@
----
-# --------------------------------------------------------------------------------------
-# Global
-# --------------------------------------------------------------------------------------
-include:
- - local: common.yml
-
-stages:
- - testing
- - manifest-integration
-
-variables:
- MANIFEST_FILE: "distro/seco-distro.xml"
- # The BB_RECIPE_NAME is used for projects referenced in the SRCREV file
- # to match the repository and the bitbake recipe name.
- # We set it here to none, as every project needing it
- # has to specify it in its own gitlab-ci.yml file.
- # The BB_RECIPE_NAME is passed to the python scripts below anyway, but not
- # used for projects referenced in the manifest file.
- MANIFEST_PROJECT: seco-manifest
- MASTER_BRANCH_MANIFEST: kirkstone
- MASTER_BRANCH_PROJECT: main
- BB_RECIPE_NAME: none
-
-workflow:
- rules:
- # Explicitly allow externally triggered pipelines in every case
- - if: $CI_PIPELINE_SOURCE == "pipeline" || $CI_PIPELINE_SOURCE == "api"
- # Do not run pipelines on forked projects.
- # The pipelines would not work anyway because of the users permissions.
- # There are two cases catched here:
- # 1. The project is forked into someones gitlab namespace and a MR to
- # include a change into this forked project is created. In this case
- # is the CI_PROJECT_ROOT_NAMESPACE not seco-ne but the
- # namespace the fork lives in.
- # 2. The MR from the forked project is created to merge the change into this
- # the project in the seco-ne namespace (customer sending
- # change to us). Here the the IDs used below differ.
- #
- - if: $CI_PROJECT_ROOT_NAMESPACE != "seco-ne"
- when: never
- - if: $CI_MERGE_REQUEST_SOURCE_PROJECT_ID != $CI_MERGE_REQUEST_PROJECT_ID
- when: never
- # FIXME: Unfortunately we cannot use variables in regular expressions due to a
- # GitLab limitation: https://gitlab.com/gitlab-org/gitlab/-/issues/209904
- # As soon as this get fixed, use the regex based rules below instead of checking
- # against the MASTER_BRANCH variable.
- # Run pipeline if target branch of the merge request has an integration target, i.e.
- # INTEGRATION contains a line beginning with the target branch followed by a colon.
- # This also implies that the pipeline runs in merge request context only.
- # - if: $INTEGRATION =~ /^$CI_MERGE_REQUEST_TARGET_BRANCH_NAME:/m
- - if: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == $MASTER_BRANCH
- # Run pipeline on target branch after merging a merge request.
- # - if: $INTEGRATION =~ /^$CI_COMMIT_BRANCH:/m
- - if: $CI_COMMIT_BRANCH == $MASTER_BRANCH
- - if: $CI_MERGE_REQUEST_IID
- - if: $CI_PIPELINE_SOURCE == "schedule"
- when: always
-
-.skip-for-gitlab-ci-integrations:
- rules:
- - if: $CI_COMMIT_REF_NAME !~ /^integrate\/gitlab-ci\/.*/
-
-# --------------------------------------------------------------------------------------
-# Manifest integration jobs
-# --------------------------------------------------------------------------------------
-testing:
- tags:
- - infrastructure
- rules:
- - if: $CI_PIPELINE_SOURCE == "schedule"
- when: always
- stage: testing
- timeout: 10 m
- image: secodocker/manifest-test:vtemp
- needs: []
-
-generate-pipelines:
- rules:
- - if: $CI_PIPELINE_SOURCE == "schedule"
- when: always
- extends:
- - .infrastructure
- - .skip-for-gitlab-ci-integrations
- stage: manifest-integration
- script:
- # The job generation script implicitly passes the OS environment to the template, so
- # that the template has access to all GitLab CI variables. Hence there is no need
- # to explicitly pass any of them as command line arguments.
- - .gitlab-ci/scripts/generate_job_from_template.py
- --template=.gitlab-ci/manifest-integration-pipelines.yml.jinja2
- > manifest-integration-pipelines.yml
- artifacts:
- expire_in: 4 weeks
- paths:
- - manifest-integration-pipelines.yml
-
-trigger-pipelines:
- rules:
- - if: $CI_PIPELINE_SOURCE == "schedule"
- when: always
- extends:
- - .skip-for-gitlab-ci-integrations
- stage: manifest-integration
- needs:
- - generate-pipelines
- trigger:
- include:
- - artifact: manifest-integration-pipelines.yml
- job: generate-pipelines
- strategy: depend
-
-check:
- extends:
- - .infrastructure
- - .skip-for-gitlab-ci-integrations
- stage: manifest-integration
- rules:
- - if: $CI_MERGE_REQUEST_IID && $CI_COMMIT_REF_NAME !~ /^integrate\/gitlab-ci\/.*/
- needs:
- - trigger-pipelines
- allow_failure: true
- script:
- - cd ${CI_PROJECT_DIR}
- # Loop over all integrations and check each integration branch
- - while read -r integration; do
- SOURCE_BRANCH=$(echo $integration | cut -d':' -f1);
- TARGET_PROJECT=$(echo $integration | cut -d':' -f2);
- TARGET_BRANCH=$(echo $integration | cut -d':' -f3);
- if [[ "$SOURCE_BRANCH" == "$CI_MERGE_REQUEST_TARGET_BRANCH_NAME" ]]; then
- .gitlab-ci/scripts/check_if_integration_branch_is_up_to_date.py
- --gitlab-url=${CI_SERVER_URL}
- --token=${GITBOT_TOKEN}
- --target-project=${TARGET_PROJECT}
- --target-branch=${TARGET_BRANCH}
- --source-project=${CI_PROJECT_PATH}
- --merge-request=${CI_MERGE_REQUEST_IID}
- ;
- fi;
- done <<< "$INTEGRATION"
-
-yamllint:
- extends: .yamllint
- stage: manifest-integration
diff --git a/old/manifest-pipeline-yocto-original.yml b/old/manifest-pipeline-yocto-original.yml
deleted file mode 100644
index 631ebd9306f7dc1319bbb3709ecac8813027b374..0000000000000000000000000000000000000000
--- a/old/manifest-pipeline-yocto-original.yml
+++ /dev/null
@@ -1,38 +0,0 @@
----
-include:
- - local: manifest-pipeline.yml
-
-variables:
- # The id of the gitlab project used in the rules section to not run pipelines in
- # forked projects. Using variable here, to allow override in other projects including
- # this file.
- MANIFEST_PROJECT_ID: 1725
-
- # In the manifest, the remotes are specified by an identifier. This is used to find
- # out included projects for the retrigger job. In custom manifests, the remote may be
- # named differently, so we need a variable that may be overriden.
- CI_PARAM_SECO_REMOTE: seco-ne
-
- # GitLab group to search for projects to retrigger
- RETRIGGER_GROUP: ${CI_PROJECT_ROOT_NAMESPACE}
-
- BUILD_TIMEOUT: 1h
-
- # This is the jinja2 template file used to generate the build jobs
- BUILD_JOBS_TEMPLATE: build-jobs-yocto.yml.jinja2
-
-generate-build-jobs:
- variables:
- # Default image and distro
- CI_PARAM_IMAGE: guf-image
- CI_PARAM_DISTRO: guf-wayland
-
- # Flash-N-Go image and distro
- # In the past, the buildfng job overwrote the image and distro itself. Due to the
- # transition to the new seconorth names, image and distro for the buildfng must be
- # settable from outside of the job.
- CI_PARAM_IMAGE_FNG: fngsystem-image
- CI_PARAM_DISTRO_FNG: guf-fngsystem
-
- # List of machines to build images for
- CI_PARAM_MACHINES: imx6guf imx6ullguf imx8mguf imx8mpguf