CI: handle_artifacts,image_diff: various fixes for pipeline in charge of preserving build artifacts
This MR addresses the first problem articulated in following JIRA post: https://secogroup.atlassian.net/browse/YT-54?focusedCommentId=45947 and problems from https://secogroup.atlassian.net/browse/YT-55?focusedCommentId=45999
In particular, the MR contains following fixes:
- got rid of duplicated code and moved particular classes to dedicated header files;
- fixed a failure when a previously launched pipeline is canceled. Just added a 10 sec delay before proceeding further, if pipeline was canceled;
- added a try-except block to ignore exceptions produced by job.keep_artifacts()/job.delete_artifacts() when artifacts for old pipelines are automatically removed;
- got rid of hard coded values. Reworked the script to parse machine, pipeline names and distro from .gitlab-ci.yml;
- fixed timeout checking.
Merge request reports
Activity
assigned to @dmitry.petrov
requested review from @jonas.hoeppner
113 113 # job.keep_artifacts() 114 114 job = manifest_project.jobs.get(pipelinejob.id, lazy=True) 115 115 116 try: 116 117 if commit.id in keep_artifacts_sha: 117 118 print(f"keep_artifacts() for {pipelinejob.web_url}") 118 119 job.keep_artifacts() 119 120 else: 120 121 print(f"delete_artifacts() for {pipelinejob.web_url}") 121 122 job.delete_artifacts() 122 123 except GitlabCreateError as e: 124 print(f"Error: {e.error_message}") Hello @jonas.hoeppner,
[Jonas] Could you add a comment why we expect the 404 to happen here? So which actual case is this?
I've added such protection against the case when artifacts have been removed by the script itself or by GitLab after the expiration date. For example, such event occurred 4 days ago for following job: https://git.seco.com/seco-ne/yocto/manifest/-/jobs/935679. So, you can easily reproduce the 404 error by trying to access non-existing artifacts for this job via following link: https://git.seco.com/seco-ne/yocto/manifest/-/jobs/935679/artifacts/browse
Best regards,
Dmitry Petrov.
added 11 commits
-
006a2b52...bf6de920 - 5 commits from branch
master - 1a4a5ab1 - Refactoring: move common code to separate files
- b0be529b - handle_artifacts: fix case when pipeline was canceled
- 3077d252 - handle_artifacts: add "try" block for keep_/delete_artifacts
- 05d6a8db - fullbuildpipeline.py: fix timeout
- 4fd7f8a8 - report_image_diff: get rid of hardcode
- 54f17a69 - Update CI_IMAGES_REVISION
Toggle commit list-
006a2b52...bf6de920 - 5 commits from branch
enabled an automatic merge when the pipeline for 54f17a69 succeeds
Integration merge request: https://git.seco.com/seco-ne/yocto/infrastructure/ci-test/minimal-srcrev/-/merge_requests/155
-
Integration merge request: https://git.seco.com/seco-ne/yocto/infrastructure/ci-test/minimal-foo/-/merge_requests/431
-
Integration merge request: https://git.seco.com/seco-ne/yocto/infrastructure/ci-test/minimal-bar/-/merge_requests/313
-
Integration merge request: https://git.seco.com/seco-ne/yocto/infrastructure/ci-test/minimal-foo/-/merge_requests/432
-
Integration merge request: https://git.seco.com/seco-ne/yocto/infrastructure/ci-test/minimal-manifest/-/merge_requests/698
