Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • seco-ne/yocto/infrastructure/gitlab-ci
1 result
Show changes
Commits on Source (9)
  • Tim Jaacks's avatar
    Documentation: update job generation chapter · 3d2dbe4d
    Tim Jaacks authored
    3d2dbe4d
  • Tim Jaacks's avatar
    Remove LOG_PREFIX variable · 1860ca8e
    Tim Jaacks authored 
    This was used in times when the same code was executed from different
places. We don't do that anymore, so the variable is obsolete.
    1860ca8e
  • Tim Jaacks's avatar
    Clean build.env before writing · a91baa55
    Tim Jaacks authored 
    This had been accidentally removed in
5e36715e.
    a91baa55
  • Tim Jaacks's avatar
    LAVA: rename MACHINE to LAVA_MACHINE · 073ae81e
    Tim Jaacks authored 
    Rename variable in order to avoid confusion with the original MACHINE
variable used everywhere else. Also rename the local Jinja2 variable to
include an underscore to make naming consistent.
    073ae81e
  • Tim Jaacks's avatar
    Add yocto version job · 922f49f8
    Tim Jaacks authored 
    This adds a machine-independent job "build-version" which populates the
RELEASE_VERSION and RELEASE_NAME variables, so that following jobs can
use these without depending on the various build jobs.
The variables can be set from the trigger job in a project's
`.gitlab-ci.yml` file. They are eval'ed before saving them to
version.env, so we can use deferred variable expansion or even command
execution to construct their values. This mechanism is already used for
the Flash-N-Go System variables.
    922f49f8
  • Tim Jaacks's avatar
    Remove RELEASE_SUFFIX variable · cad30c20
    Tim Jaacks authored 
    The RELEASE_NAME variable can be set directly now, so there's no need to
have a dedicated RELEASE_SUFFIX anymore. A previous configuration like

  RELEASE_SUFFIX: "-custom"

can now be achieved using:

  RELEASE_NAME: "Yocto-${RELEASE_VERSION}-custom"
    cad30c20
  • Tim Jaacks's avatar
    Use RELEASE_VERSION and RELEASE_NAME from build-version job · 372d192b
    Tim Jaacks authored 
    Instead of passing these variables from stage to stage or regenerating
their values in later stages, use the ones set in the build-version job
at all places.
    372d192b
  • Tim Jaacks's avatar
    Introduce separate variables to set RELEASE_NAME and RELEASE_VERSION · cdd47023
    Tim Jaacks authored 
    The expressions to calculate RELEASE_NAME and RELEASE_VERSION cannot be
stored within these variables themseselves. If set on the trigger level,
they would override the calculated values in build-version.env then,
as trigger variables always have the highest precedence.

Use separate variables RELEASE_VERSION_EXPRESSION and
RELEASE_NAME_EXPRESSION to define how RELEASE_VERSION and RELEASE_NAME
are calculated instead.
    cdd47023
  • Tim Jaacks's avatar
    Define MACHINE variable on job level · 1d5e479f
    Tim Jaacks authored 
    Instead of passing MACHINE from stage to stage or loading it from
testdata.json, use the original value from the Jinja2 loop and set it
directly for each job where it is used.
    1d5e479f
Hide whitespace changes
Inline Side-by-side
build-pipeline-ci-test.yml
View file @ 1d5e479f
......@@ -39,7 +39,6 @@ build:files:
timeout: 2m
variables:
GIT_STRATEGY: none
LOG_PREFIX: "CI:build:"
before_script:
- !reference [.docker_check]
- !reference [.setup_ssh]
......
build-pipeline-yocto.yml.jinja2
View file @ 1d5e479f
......@@ -33,6 +33,12 @@ variables:
changelog:
extends: .changelog
build-version:
extends: .build_version
variables:
# We have to set a machine, even if it is not actually needed for what the job does,
# so we just choose the first.
MACHINE: {{ MACHINES.split(' ')[0] }}
# --------------------------------------------------------------------------------------
# Generated jobs
......@@ -82,16 +88,16 @@ build-{{ machine }}:
{% if machine == 'seco-mx6' or machine == 'imx6guf' %}
{% set platforms = "santaro santoka santino santino-lt" %}
{% set lavamachine = "imx6guf" %}
{% set lava_machine = "imx6guf" %}
{% elif machine == 'seco-mx6ull' or machine == 'imx6ullguf' %}
{% set platforms = "nallino" %}
{% set lavamachine = "imx6ullguf" %}
{% set lava_machine = "imx6ullguf" %}
{% elif machine == 'seco-mx8mm' or machine == 'imx8mguf' %}
{% set platforms = "tanaro" %}
{% set lavamachine = "imx8mguf" %}
{% set lava_machine = "imx8mguf" %}
{% else %}
{% set platforms = '' %}
{% set lavamachine = '' %}
{% set lava_machine = '' %}
{% endif %}
{% if platforms %}
......@@ -104,8 +110,8 @@ smoketest:{{ machine }}:
- job: build-{{ machine }}
variables:
BUILD_JOB: build-{{ machine }}
MACHINE: {{ lavamachine }}
PLATFORMS: {{ lavamachine }}
LAVA_MACHINE: {{ lava_machine }}
PLATFORMS: {{ lava_machine }}
TEST_SUITE: boot.jinja2
TEST_EXTRA_ARGUMENTS: --all-devices
......@@ -117,7 +123,7 @@ platformtest:{{ machine }}:
- job: build-{{ machine }}
variables:
BUILD_JOB: build-{{ machine }}
MACHINE: {{ lavamachine }}
LAVA_MACHINE: {{ lava_machine }}
PLATFORMS: {{ platforms }}
{% endif %}
......@@ -128,12 +134,14 @@ platformtest:{{ machine }}:
package-{{ machine }}:
extends: .package
variables:
MACHINE: {{ machine }}
PACKAGE_TYPE: image
ASSOCIATED_BUILD_JOB: build-{{ machine }}
needs:
- job: build-{{ machine }}
artifacts: false
- job: changelog
- job: build-version
# --------------------------------------------------------------------------------------
......@@ -144,12 +152,14 @@ deploy-{{ machine }}:
{% if CI_COMMIT_TAG is defined %}
stage: Deploy SoftwareStore
variables:
MACHINE: {{ machine }}
DEPLOY_SOURCE: release/$${RELEASE_NAME}
DEPLOY_TARGET: ${DEPLOY_RELEASE_TARGET}
DEPLOY_TARGET_LINK: ${DEPLOY_RELEASE_TARGET_LINK}
{% else %}
stage: Deploy SoftwareStore Internal
variables:
MACHINE: {{ machine }}
DEPLOY_SOURCE: release/$${RELEASE_NAME}
DEPLOY_TARGET: ${DEPLOY_INTERNAL_RELEASE_TARGET}
DEPLOY_TARGET_LINK: ${DEPLOY_INTERNAL_RELEASE_TARGET_LINK}
......@@ -159,6 +169,7 @@ deploy-{{ machine }}:
- job: package-{{ machine }}
artifacts: false
- job: changelog
- job: build-version
# --------------------------------------------------------------------------------------
......@@ -166,8 +177,11 @@ deploy-{{ machine }}:
# --------------------------------------------------------------------------------------
generate-alphaplan-data-{{ machine }}:
extends: .generate_alphaplan_data
variables:
MACHINE: {{ machine }}
needs:
- deploy-{{ machine }}
- build-version
import-alphaplan-data-{{ machine }}:
extends: .import_alphaplan_data
......@@ -191,6 +205,7 @@ ftp-{{ machine }}:
tags:
- ftp
variables:
MACHINE: {{ machine }}
DEPLOY_SOURCE: release/$${RELEASE_NAME}
DEPLOY_TARGET: ${DEPLOY_FTP_TARGET}
DEPLOY_TARGET_LINK: ${DEPLOY_FTP_TARGET_LINK}
......@@ -199,6 +214,7 @@ ftp-{{ machine }}:
- job: package-{{ machine }}
artifacts: false
- job: changelog
- job: build-version
{% endif %}
{% endif %}
......@@ -218,6 +234,7 @@ azure-{{ machine }}:
when: manual
allow_failure: true
variables:
MACHINE: {{ machine }}
DEPLOY_SOURCE: release
DEPLOY_TARGET: ${AZURE_TARGET_FOLDER}
CONTAINER_NAME: ${AZURE_CONTAINER_NAME}
......@@ -252,8 +269,6 @@ generate-confluence-page:
publish-confluence-page:
extends:
- .publish-confluence-page
needs:
- generate-confluence-page
{% endif %}
{% endif %}
......
build-pipeline.yml
View file @ 1d5e479f
......@@ -29,7 +29,7 @@ workflow:
# CI/CD variable GITLAB_PRIVATE_KEY of the manifest repository. The public key has
# been added as a deploy key in GitLab's repository settings for all contained
# repositories.
- echo "${LOG_PREFIX} Setup SSH"
- echo "Setup SSH"
- eval $(ssh-agent -s)
- echo "$GITLAB_PRIVATE_KEY" | tr -d '\r' | ssh-add -
- mkdir -p ~/.ssh
......@@ -46,7 +46,7 @@ workflow:
- echo "$SSH_CONFIG" > ~/.ssh/config
.repo_checkout: &repo_checkout
- echo "${LOG_PREFIX} Perform repo checkout"
- echo "Perform repo checkout"
- cd ${CI_PROJECT_DIR}
- repo init --submodules -u ${CI_REPOSITORY_URL}
-b refs/pipelines/${CI_PIPELINE_ID}
......@@ -59,7 +59,7 @@ workflow:
SRCREVS_FILE="${CI_PROJECT_DIR}/${IMAGE_PATH}/BUILD_SRCREVS.log"
if [ -d "$( dirname "${SRCREVS_FILE}" )" ];then
buildhistory-collect-srcrevs > ${SRCREVS_FILE}
echo "${LOG_PREFIX} buildhistory-collect-srcrevs:"
echo "buildhistory-collect-srcrevs:"
cat ${SRCREVS_FILE}
fi
......@@ -96,24 +96,25 @@ workflow:
fi
fi
.build_script: &build_script
# setup build environment
- echo "${LOG_PREFIX} Build configuration MACHINE=${MACHINE}
.setup_build: &setup_build
- echo "Build configuration MACHINE=${MACHINE}
DISTRO=${YOCTO_DISTRO} IMAGE=${YOCTO_IMAGE}"
- echo "${LOG_PREFIX} Using build dir ${BUILD_PATH}"
- echo "Using build dir ${BUILD_PATH}"
- export MACHINE="${MACHINE}"
- export DISTRO="${YOCTO_DISTRO}"
- export EULA="1"
- source ./"${SETUP_SCRIPT}" "${BUILD_PATH}"
# start build
- source "${SETUP_SCRIPT}" "${BUILD_PATH}"
.run_build: &run_build
- echo -e "section_start:`date +%s`:bitbake_run\r\e[0KBitbake Log"
- echo "${LOG_PREFIX} bitbake ${YOCTO_IMAGE} -c ${BITBAKE_TASK}"
- echo "bitbake ${YOCTO_IMAGE} -c ${BITBAKE_TASK}"
- bitbake "${YOCTO_IMAGE}" -c "${BITBAKE_TASK}"
- echo -e "section_end:`date +%s`:bitbake_run\r\e[0K"
.save_build_env: &save_build_env
- cd ${CI_PROJECT_DIR}
# Artifact paths are needed for packaging
- echo "IMAGE_PATH=${IMAGE_PATH}" >> build.env
- echo "IMAGE_PATH=${IMAGE_PATH}" > build.env
- echo "SDK_PATH=${SDK_PATH}" >> build.env
- echo "LICENSES_PATH=${LICENSES_PATH}" >> build.env
# Install script location is needed in test job
......@@ -140,6 +141,40 @@ workflow:
paths:
- "changelog.md"
.build_version:
extends:
- .build_yocto
stage: Infrastructure
tags:
- infrastructure
needs: []
rules:
- when: always
variables:
BITBAKE_ENV_COMMAND: bitbake ${YOCTO_IMAGE} -e
# RELEASE_VERSION_EXPRESSION and RELEASE_NAME_EXPRESSION are eval'ed and then saved
# as RELEASE_VERSION and RELEASE_NAME to version.env, so we can use deferred
# variable expansion or even command execution to construct their values.
RELEASE_VERSION_EXPRESSION: $${DISTRO_VERSION}
RELEASE_NAME_EXPRESSION: Yocto-$${RELEASE_VERSION}
script:
# Extract all variable assignments from the BitBake environment and source them.
# The rather complicated regex also regards values containing escaped quotes.
- source <( ${BITBAKE_ENV_COMMAND} | egrep '^[A-Z]\w*="([^\\"]+|\\.)*"$' )
- echo ${RELEASE_VERSION_EXPRESSION}
- echo ${RELEASE_NAME_EXPRESSION}
- RELEASE_VERSION=$(eval echo "${RELEASE_VERSION_EXPRESSION}")
- RELEASE_NAME=$(eval echo "${RELEASE_NAME_EXPRESSION}")
- echo "RELEASE_VERSION=${RELEASE_VERSION}" >> ${CI_PROJECT_DIR}/version.env
- echo "RELEASE_NAME=${RELEASE_NAME}" >> ${CI_PROJECT_DIR}/version.env
- cat ${CI_PROJECT_DIR}/version.env
artifacts:
expire_in: 1 week
reports:
dotenv: version.env
paths: []
cache: []
# --------------------------------------------------------------------------------------
# Stage: build
# --------------------------------------------------------------------------------------
......@@ -210,7 +245,6 @@ workflow:
variables:
GIT_STRATEGY: none
SETUP_SCRIPT: "setup-environment"
LOG_PREFIX: "CI:build:"
BUILD_PATH: "build-${YOCTO_DISTRO}-${MACHINE}"
IMAGE_PATH: "${BUILD_PATH}/tmp/deploy/images/${MACHINE}"
LICENSES_PATH: "${BUILD_PATH}/tmp/deploy/licenses"
......@@ -221,10 +255,11 @@ workflow:
- *docker_check
- *setup_ssh
- *repo_checkout
- *setup_build
script:
- *save_build_env
- *build_script
- *run_build
- *collect_srcrevs
- *save_build_env
- *dump_install_command
# --------------------------------------------------------------------------------------
......@@ -270,7 +305,7 @@ workflow:
- .gitlab-ci/scripts/submit_test.py
--fng-install "$FNG_INSTALL_URL"
--name
"Gitlab $MACHINE $YOCTO_DISTRO $YOCTO_IMAGE ($CI_PIPELINE_ID)"
"Gitlab $LAVA_MACHINE $YOCTO_DISTRO $YOCTO_IMAGE ($CI_PIPELINE_ID)"
--results-path "results"
--report-name "testresults-${CI_JOB_NAME}.md"
--test-repo $TESTS_GIT_URL
......@@ -296,26 +331,27 @@ workflow:
artifacts:
paths:
- release/**/**/*
reports:
dotenv: package.env
timeout: 90m
before_script:
# We do this manually since we do not use GitLab's default artifact downloader
- source build.env
script:
# IMAGE_PATH, SDK_PATH and LICENSES_PATH are available via build.env from build job
# RELEASE_NAME is available via version.env from build-version job
- if [[ "${PACKAGE_TYPE}" == "image" ]]; then
- .gitlab-ci/scripts/package_release.py
--images-dir="${IMAGE_PATH}"
--licenses-dir="${LICENSES_PATH}"
--doc-dir=.
--output-dir=release
--release-suffix="${RELEASE_SUFFIX}"
--output-dir=release/${RELEASE_NAME}
--release-version=${RELEASE_VERSION}
--machine=${MACHINE}
- elif [[ "${PACKAGE_TYPE}" == "sdk" ]]; then
- .gitlab-ci/scripts/package_release.py
--sdk-dir="${SDK_PATH}"
--output-dir=release
--release-suffix="${RELEASE_SUFFIX}"
--output-dir=release/${RELEASE_NAME}
--release-version=${RELEASE_VERSION}
--machine=${MACHINE}
- fi
cache:
- key: ${CI_PIPELINE_ID}-${ASSOCIATED_BUILD_JOB}
......@@ -325,7 +361,6 @@ workflow:
policy: push
paths:
- release
- package.env
# --------------------------------------------------------------------------------------
# Stage: deploy
......@@ -335,12 +370,9 @@ workflow:
- when: manual
allow_failure: true
before_script:
# We do this manually since we do not use GitLab's default artifact downloader
- source package.env
# Save dotenv data for next stage
- echo "RELEASE_NAME=${RELEASE_NAME}" > deploy.env
- echo "VERSION=${VERSION}" >> deploy.env
- echo "MACHINE=${MACHINE}" >> deploy.env
# Save MACHINE for confluence stage. This variable cannot be passed directly on the
# job definition level because the confluence stage is machine-independent.
- echo "MACHINE=${MACHINE}" > deploy.env
# Expand eventual nested variables contained within the DEPLOY_* variables
# FIXME: For now we need a double 'eval' here due to a GitLab bug:
# https://gitlab.com/gitlab-org/gitlab/-/issues/273409
......@@ -411,6 +443,8 @@ workflow:
rules:
- when: manual
allow_failure: true
needs:
- build-version
script:
- .gitlab-ci/scripts/collect_release_information.py
--project $CI_PROJECT_ID
......@@ -424,12 +458,9 @@ workflow:
- .gitlab-ci/scripts/render_jinja2_template.py
--template="${PAGE_TEMPLATE}"
> confluence-page.xml
- echo "RELEASE_NAME=${RELEASE_NAME}" >> confluence.env
artifacts:
paths:
- confluence-page.xml
reports:
dotenv: confluence.env
.publish-confluence-page:
extends:
......@@ -438,6 +469,9 @@ workflow:
rules:
- when: manual
allow_failure: true
needs:
- generate-confluence-page
- build-version
script:
- .gitlab-ci/scripts/confluence_create_or_update_page.py
--username="${CONFLUENCE_USERNAME}"
......@@ -459,7 +493,7 @@ workflow:
rules:
- if: $ALPHAPLAN_STAGE == "true"
script:
# MACHINE and RELEASE_NAME are available from deploy.env
# RELEASE_NAME is available from build-version.env
- .gitlab-ci/scripts/generate_alphaplan_fwr_file.py
--machine="${MACHINE}"
--release-name="${RELEASE_NAME}"
......
docs/manifest-parent-child.png
View replaced file @ 17b87f84 View file @ 1d5e479f
docs/manifest-parent-child.png

47.7 KiB | W: 0px | H: 0px

docs/manifest-parent-child.png

66.9 KiB | W: 0px | H: 0px

docs/manifest-parent-child.png
docs/manifest-parent-child.png
docs/manifest-parent-child.png
docs/manifest-parent-child.png
  • 2-up
  • Swipe
  • Onion skin
docs/manifest-pipeline.md
View file @ 1d5e479f
......@@ -31,35 +31,118 @@ TODO
TODO
<!----------------------------------------------------------------------------->
## Job generation, parent child jobs
## Dynamic Pipeline Generation
<!----------------------------------------------------------------------------->
As we support images for multiple machines/architectures we are building a lot
of different image variants, where each needs at least build, deploy and test
job.
To simplify the generation of all these jobs, without a lot of copy and paste
and the possibility to configure the actual build images, we are using gitlab's
dynamic-child-pipeline feature. [See gitlab docs.][1]
### Parent/child pipelines
We support multiple machines/architectures, and for each one of them we need the
same set of build, package, deploy and test jobs. To simplify the definition
of all these jobs without lots of copy and paste actions, and with the
possibility to configure the actual build images, we are using [GitLab's
dynamic-child-pipeline feature][1].
[1]: https://docs.gitlab.com/ee/ci/pipelines/parent_child_pipelines.html#dynamic-child-pipeline-example
There is a *'generate-build-pipeline'* job, that creates a yaml file containing
the pipeline with all needed jobs.
There are the following CI variables in the 'generate-build-pipeline' job
controlling the content (make sure these are not set in a more global scope, as
this would overwrite the settings in the generated yml):
There is a `generate-build-pipeline` job that creates a YAML file containing
the pipeline with all the jobs for each machine, based on its input variable
`MACHINES`:
* `MACHINES`
Space separated list of machines to build for, e.g. "santaro santoka"
This is done using the Python script [`render_jinja2_template.py`][2], which
renders the template [`build-pipeline-yocto.yml.jinja2`][3] to
`build-pipeline.yml`.
* `CI_PARAM_MACHINES`: Space separated list of machines to build for, like "santaro santoka"
* `CI_PARAM_IMAGE`: The name of the image to build. If set to an empty string,
related jobs are not created.
* `CI_PARAM_DISTRO`: The name of the distro to build
[2]: ../scripts/render_jinja2_template.py
[3]: ../build-pipeline-yocto.yml.jinja2
* `CI_PARAM_IMAGE_FNG`: The name of the fngsystem image to build. If set to an
empty string, related jobs are not created.
* `CI_PARAM_DISTRO_FNG`: The name of the fngsystem distro to build
For each Yocto image to be built there is an `<image>-pipeline` job which
triggers the generated pipeline. The standard Yocto manifest pipeline includes
three pre-configured pipelines:
It uses a python script called `render_jinja2_template.py` to convert the
`build-pipeline.yml.jinja2` to `build-pipeline.yml`. This yml file is then used
by the *'trigger-build-pipeline'* job, to setup the pipeline described by it.
* **yocto-pipeline**: Builds a Yocto image
* **sdk-pipeline**: Builds a Yocto SDK
* **fngsystem-pipeline**: Builds a Flash-N-Go System image
See the following screenshot for how GitLab displays these pipelines:
![Manifest's parent child pipeline](manifest-parent-child.png)
### Pipeline configuration
A project including the manifest pipeline can alter the configuration for the
pipelines by overriding these variables in the trigger job:
* `YOCTO_IMAGE`
Name of the Yocto image to build
* `YOCTO_DISTRO`
Name of the Yocto distro to build
* `BITBAKE_TASK`
Name of the BitBake task to execute
* `MANUAL_BUILD`
Boolean value to set whether the build jobs are manual
* `INSTALL_SCRIPT`
Name of the Flash-N-Go install script file
* `PACKAGE_TYPE`
Type of packaging (supported values: `image`, `sdk`)
* `TEST_STAGE`
Boolean value to set whether to include the Test stage
* `ALPHAPLAN_STAGE`
Boolean value to set whether to include the Alphaplan stage
* `CONFLUENCE_SPACE`
Confluence Space to publish the release overview in
* `CONFLUENCE_PARENT_ID`
Confluence page to publish the release overview under
* `DEPLOY_RELEASE_TARGET`
Path to the mounted SoftwareStore folder within the deploy Docker container
* `DEPLOY_RELEASE_TARGET_LINK`
Equivalent URL on Z:
* `DEPLOY_INTERNAL_RELEASE_TARGET`
See `DEPLOY_RELEASE_TARGET`, but for internal releases
* `DEPLOY_INTERNAL_RELEASE_TARGET_LINK`
See `DEPLOY_RELEASE_TARGET_LINK`, but for internal releases
* `DEPLOY_FTP_TARGET`
See `DEPLOY_RELEASE_TARGET`, but for FTP
* `DEPLOY_FTP_TARGET_LINK`
See `DEPLOY_RELEASE_TARGET_LINK`, but for FTP
* `AZURE_CONTAINER_NAME`
Name of the Azure blob container to deploy the images to
* `AZURE_TARGET_FOLDER`
Name of the Azure folder to deploy the images to
**Example:**
```yaml
yocto-pipeline:
variables:
CI_PARAM_IMAGE: seconorth-image
CI_PARAM_DISTRO: seconorth-wayland
AZURE_CONTAINER_NAME: seco-ne-private
AZURE_TARGET_FOLDER: path/to/my/private/image
```
### Disabling default pipelines
The including project can decide to remove one of the default pipelines:
```yaml
fngsystem-pipeline:
rules:
- when: never
```
### Custom pipelines
The including project can add custom pipelines:
```yaml
yocto-custom-pipeline:
extends:
- yocto-pipeline
variables:
CI_PARAM_IMAGE: seconorth-image-custom
CI_PARAM_DISTRO: seconorth-wayland-custom
```
manifest-pipeline-ci-test.yml
View file @ 1d5e479f
......@@ -72,7 +72,9 @@ yocto-simulation-pipeline:
https://git.seco.com/seco-ne/yocto/manifest/-/jobs/artifacts/kirkstone/7.0/download?job=build
YOCTO_IMAGE: seconorth-image
YOCTO_DISTRO: seconorth-wayland
SETUP_SCRIPT: /dev/null
INSTALL_SCRIPT: fng-install.sh
BITBAKE_ENV_COMMAND: echo DISTRO_VERSION="kirkstone-7.0"
ARTIFACTS_PATH: build-*/tmp/deploy/images/**/*
PACKAGE_TYPE: image
TEST_STAGE: "true"
......@@ -92,7 +94,9 @@ sdk-simulation-pipeline:
https://git.seco.com/seco-ne/yocto/manifest/-/jobs/artifacts/kirkstone/7.0/download?job=buildsdk
YOCTO_IMAGE: seconorth-image
YOCTO_DISTRO: seconorth-wayland
SETUP_SCRIPT: /dev/null
INSTALL_SCRIPT: fng-install.sh
BITBAKE_ENV_COMMAND: echo DISTRO_VERSION="kirkstone-7.0"
ARTIFACTS_PATH: build-*/tmp/deploy/sdk/*
MANUAL_BUILD: "true"
PACKAGE_TYPE: sdk
manifest-pipeline-yocto.yml
View file @ 1d5e479f
......@@ -96,6 +96,15 @@ fngsystem-pipeline:
YOCTO_IMAGE: fngsystem-image
YOCTO_DISTRO: guf-fngsystem
INSTALL_SCRIPT: fngsystem-self-update.sh
# FIXME: For now we need a two-step variable assignment here due to a GitLab bug:
# https://gitlab.com/gitlab-org/gitlab/-/issues/273409
# Escaped variables are not correctly passed to child pipelines. Proposed workaround
# is to use raw variables, but we need at least GitLab 15.6 for that. As soon as we
# update our GitLab, we can get rid of the DEFERRED_RELEASE_VERSION variable and
# assign its value to RELEASE_VERSION_EXPRESSION directly.
DEFERRED_RELEASE_VERSION: $$(echo $$DISTRO_VERSION | sed "s/fngsystem-//")
RELEASE_VERSION_EXPRESSION: $DEFERRED_RELEASE_VERSION
RELEASE_NAME_EXPRESSION: FNGSystem-${DEFERRED_RELEASE_VERSION}
ARTIFACTS_PATH: build-*/tmp/deploy/images/**/*
PACKAGE_TYPE: image
ALPHAPLAN_STAGE: "true"
scripts/package_release.py
View file @ 1d5e479f
......@@ -82,15 +82,21 @@ def main():
)
parser.add_argument(
"--output-dir",
help="""Base directory name for output artifacts (can be specified multiple times)""",
help="""Directory name for output artifacts""",
dest="output_dir",
action="append",
required=True,
)
parser.add_argument(
"--release-suffix",
help="""Suffix to append to the release folder""",
dest="release_suffix",
"--release-version",
help="""Release version""",
dest="release_version",
required=True,
)
parser.add_argument(
"--machine",
help="""Machine""",
dest="machine",
required=True,
)
args, _ = parser.parse_known_args()
......@@ -118,29 +124,13 @@ def main():
with open(testdata_files[0], "r", encoding="utf-8") as f:
buildvars = json.load(f)
machine = buildvars["MACHINE"]
version = buildvars["DISTRO_VERSION"]
sdkname = buildvars["TOOLCHAIN_OUTPUTNAME"]
image_artifacts = buildvars["DISTRO_IMAGES"].split()
artifacts = buildvars["DISTRO_RELEASE_ARTEFACTS"].split()
artifacts.append("BUILD_SRCREVS.log")
# Set release name
if version.startswith("fngsystem"):
release_name = version.replace("fngsystem", "FNGSystem")
else:
release_name = f"Yocto-{version}"
# Append release suffix
if args.release_suffix is not None:
release_name = release_name + args.release_suffix
# Create output directories
output_dirs = []
for output_dir in args.output_dir:
full_output_dir = os.path.join(output_dir, release_name)
output_dirs.append(full_output_dir)
os.makedirs(full_output_dir, exist_ok=True)
# Create output directory
os.makedirs(args.output_dir, exist_ok=True)
# Package documentation files
if args.doc_dir is not None:
......@@ -160,8 +150,7 @@ def main():
files.append(doc_md5sums_file)
# Copy files
for output_dir in output_dirs:
copy_files(files, output_dir)
copy_files(files, args.output_dir)
# Package image files
if args.images_dir is not None:
......@@ -192,8 +181,8 @@ def main():
# Generate metadata file
generate_metadata(
machine,
version,
args.machine,
args.release_version,
artifacts,
sdkname,
os.path.join(args.images_dir, "metainfo.json"),
......@@ -201,27 +190,19 @@ def main():
artifacts.append(os.path.join(args.images_dir, "metainfo.json"))
# Copy files
for output_dir in output_dirs:
copy_files(artifacts, os.path.join(output_dir, machine))
copy_files(artifacts, os.path.join(args.output_dir, args.machine))
# Package SDK
if args.sdk_dir is not None:
sdkfiles = glob.glob(os.path.join(args.sdk_dir, f"{sdkname}*"))
# Generate MD5 sums file
sdk_md5sums_file = os.path.join(machine, "sdk", "md5sums.txt")
sdk_md5sums_file = os.path.join(args.machine, "sdk", "md5sums.txt")
generate_md5sums_file(sdkfiles, sdk_md5sums_file)
sdkfiles.append(sdk_md5sums_file)
# Copy files
for output_dir in output_dirs:
copy_files(sdkfiles, os.path.join(output_dir, machine, "sdk"))
# Store pathes and other stuff in environment variable file
with open("package.env", "w", encoding="utf-8") as env_file:
env_file.write(f"RELEASE_NAME={release_name}\n")
env_file.write(f"VERSION={version}\n")
env_file.write(f"MACHINE={machine}\n")
copy_files(sdkfiles, os.path.join(args.output_dir, args.machine, "sdk"))
if __name__ == "__main__":
......