diff --git a/docs/manifest-parent-child.png b/docs/manifest-parent-child.png index 9c3001def9c0e735a0b46091febe213631cbd4fa..3cc4d88de9b998670dd8a9409ca4df9fca749aa8 100644 Binary files a/docs/manifest-parent-child.png and b/docs/manifest-parent-child.png differ diff --git a/docs/manifest-pipeline.md b/docs/manifest-pipeline.md index 81f980c8808f6f46a0feba50f29e0c9a61208e97..0afd32b24a26dcf3b24db8e7610304b7171df620 100644 --- a/docs/manifest-pipeline.md +++ b/docs/manifest-pipeline.md @@ -31,35 +31,118 @@ TODO TODO <!-----------------------------------------------------------------------------> -## Job generation, parent child jobs +## Dynamic Pipeline Generation <!-----------------------------------------------------------------------------> -As we support images for multiple machines/architectures we are building a lot -of different image variants, where each needs at least build, deploy and test -job. -To simplify the generation of all these jobs, without a lot of copy and paste -and the possibility to configure the actual build images, we are using gitlab's -dynamic-child-pipeline feature. [See gitlab docs.][1] +### Parent/child pipelines + +We support multiple machines/architectures, and for each one of them we need the +same set of build, package, deploy and test jobs. To simplify the definition +of all these jobs without lots of copy and paste actions, and with the +possibility to configure the actual build images, we are using [GitLab's +dynamic-child-pipeline feature][1]. [1]: https://docs.gitlab.com/ee/ci/pipelines/parent_child_pipelines.html#dynamic-child-pipeline-example -There is a *'generate-build-pipeline'* job, that creates a yaml file containing -the pipeline with all needed jobs. -There are the following CI variables in the 'generate-build-pipeline' job -controlling the content (make sure these are not set in a more global scope, as -this would overwrite the settings in the generated yml): +There is a `generate-build-pipeline` job that creates a YAML file containing +the pipeline with all the jobs for each machine, based on its input variable +`MACHINES`: + +* `MACHINES` + Space separated list of machines to build for, e.g. "santaro santoka" + +This is done using the Python script [`render_jinja2_template.py`][2], which +renders the template [`build-pipeline-yocto.yml.jinja2`][3] to +`build-pipeline.yml`. -* `CI_PARAM_MACHINES`: Space separated list of machines to build for, like "santaro santoka" -* `CI_PARAM_IMAGE`: The name of the image to build. If set to an empty string, - related jobs are not created. -* `CI_PARAM_DISTRO`: The name of the distro to build +[2]: ../scripts/render_jinja2_template.py +[3]: ../build-pipeline-yocto.yml.jinja2 -* `CI_PARAM_IMAGE_FNG`: The name of the fngsystem image to build. If set to an - empty string, related jobs are not created. -* `CI_PARAM_DISTRO_FNG`: The name of the fngsystem distro to build +For each Yocto image to be built there is an `<image>-pipeline` job which +triggers the generated pipeline. The standard Yocto manifest pipeline includes +three pre-configured pipelines: -It uses a python script called `render_jinja2_template.py` to convert the -`build-pipeline.yml.jinja2` to `build-pipeline.yml`. This yml file is then used -by the *'trigger-build-pipeline'* job, to setup the pipeline described by it. +* **yocto-pipeline**: Builds a Yocto image +* **sdk-pipeline**: Builds a Yocto SDK +* **fngsystem-pipeline**: Builds a Flash-N-Go System image + +See the following screenshot for how GitLab displays these pipelines:  + +### Pipeline configuration + +A project including the manifest pipeline can alter the configuration for the +pipelines by overriding these variables in the trigger job: + +* `YOCTO_IMAGE` + Name of the Yocto image to build +* `YOCTO_DISTRO` + Name of the Yocto distro to build +* `BITBAKE_TASK` + Name of the BitBake task to execute +* `MANUAL_BUILD` + Boolean value to set whether the build jobs are manual +* `INSTALL_SCRIPT` + Name of the Flash-N-Go install script file +* `PACKAGE_TYPE` + Type of packaging (supported values: `image`, `sdk`) +* `TEST_STAGE` + Boolean value to set whether to include the Test stage +* `ALPHAPLAN_STAGE` + Boolean value to set whether to include the Alphaplan stage +* `CONFLUENCE_SPACE` + Confluence Space to publish the release overview in +* `CONFLUENCE_PARENT_ID` + Confluence page to publish the release overview under +* `DEPLOY_RELEASE_TARGET` + Path to the mounted SoftwareStore folder within the deploy Docker container +* `DEPLOY_RELEASE_TARGET_LINK` + Equivalent URL on Z: +* `DEPLOY_INTERNAL_RELEASE_TARGET` + See `DEPLOY_RELEASE_TARGET`, but for internal releases +* `DEPLOY_INTERNAL_RELEASE_TARGET_LINK` + See `DEPLOY_RELEASE_TARGET_LINK`, but for internal releases +* `DEPLOY_FTP_TARGET` + See `DEPLOY_RELEASE_TARGET`, but for FTP +* `DEPLOY_FTP_TARGET_LINK` + See `DEPLOY_RELEASE_TARGET_LINK`, but for FTP +* `AZURE_CONTAINER_NAME` + Name of the Azure blob container to deploy the images to +* `AZURE_TARGET_FOLDER` + Name of the Azure folder to deploy the images to + +**Example:** + +```yaml +yocto-pipeline: + variables: + CI_PARAM_IMAGE: seconorth-image + CI_PARAM_DISTRO: seconorth-wayland + AZURE_CONTAINER_NAME: seco-ne-private + AZURE_TARGET_FOLDER: path/to/my/private/image +``` + + +### Disabling default pipelines + +The including project can decide to remove one of the default pipelines: + +```yaml +fngsystem-pipeline: + rules: + - when: never +``` + +### Custom pipelines + +The including project can add custom pipelines: + +```yaml +yocto-custom-pipeline: + extends: + - yocto-pipeline + variables: + CI_PARAM_IMAGE: seconorth-image-custom + CI_PARAM_DISTRO: seconorth-wayland-custom +```