CI: add "report-image-diff" job

This is a solution for [YT-55] CI: Add a 'test' job to grab information on the build result: Changed packages, change in image size.

Added a python script with corresponding GitLab CI job to launch this script.

1. The script retrieves build artifacts for two branches:

   • for the default/main branch (currently kirkstone) in the manifest repo;
   • for the integration branch from merge request.

   After that, the script compares those two sets of artifacts and creates a summary about following differences between these two sets:

   • the size of compressed artifacts (artifacts.zip);
   • the size of built image;
   • the difference between two manifest files.

   Herewith this comparison is done only for the machines and distributions hard coded in the script:

   machines = (
       "seco-genio510",
       "seco-genio700",
       "seco-mx6",
       "seco-mx6ull",
       "seco-mx8mm",
       "seco-mx8mp",
   )
   pipelines_and_distros = {
       "fngsystem-pipeline": "seconorth-fngsystem",
       "yocto-pipeline": "seconorth-wayland",
   }

   The produced summary is reported both to stdout and as a comment to the merge request.

   Notes:

   1. there are a lot of lines in diff file for manifests of linux kernel when it’s updated. So we:
      a) remove all these lines from the diff to keep summary readable
      b) print a message about kernel update.
      The result looks like:
      The kernel was updated from 304b502415 to 0691f0a31a commit. 1586 lines were removed from diff.

   So the report for non kernel diff looks like as follows:

   ⮘ seconorth-wayland | seco-mx6 ⮚
   ├── artifacts.zip size: [ 342.08 MiB → 388.44 MiB ] | +46.36 MiB
   ├── image size: [ 332 MiB → 378 MiB ]
   └── manifest diff:

   --- seconorth-image-seco-mx6-20240405130924.rootfs.manifest
   +++ seconorth-image-seco-mx6-20240410145347.rootfs.manifest
   +autoconf cortexa9t2hf_neon 2.71
   +automake cortexa9t2hf_neon 1.16.5
   +binutils cortexa9t2hf_neon 2.38
   +cpp cortexa9t2hf_neon 11.3.0
   +cpp-symlinks cortexa9t2hf_neon 11.3.0
   +g++ cortexa9t2hf_neon 11.3.0
   +g++-symlinks cortexa9t2hf_neon 11.3.0
   +gcc cortexa9t2hf_neon 11.3.0
   +gcc-symlinks cortexa9t2hf_neon 11.3.0
   +gettext cortexa9t2hf_neon 0.21
   +gnu-config cortexa9t2hf_neon 20211108+git0+191bcb948f
   +libatomic-dev cortexa9t2hf_neon 11.3.0
   +libc-malloc-debug0 cortexa9t2hf_neon 2.35
   +libc6-dev cortexa9t2hf_neon 2.35
   +libc6-extra-nss cortexa9t2hf_neon 2.35
   +libcroco cortexa9t2hf_neon 0.6.13
   +libgcc-s-dev cortexa9t2hf_neon 11.3.0
   +libgettextlib cortexa9t2hf_neon 0.21
   +libgettextsrc cortexa9t2hf_neon 0.21
   +libmpc3 cortexa9t2hf_neon 1.2.1
   +libmpfr6 cortexa9t2hf_neon 4.1.0
   +libnss-db2 cortexa9t2hf_neon 2.35
   +libstdc++-dev cortexa9t2hf_neon 11.3.0
   +libtool cortexa9t2hf_neon 2.4.7
   +linux-libc-headers-dev cortexa9t2hf_neon 5.16
   +m4 cortexa9t2hf_neon 1.4.19
   +make cortexa9t2hf_neon 4.3
   +packagegroup-core-buildessential cortexa9t2hf_neon 1.0
   +pkgconfig cortexa9t2hf_neon 0.29.2+git0+d97db4fae4

2. The CI job is triggered when the build is complete.

   Each run of the pipeline causes:

   • the removal of the old summary comment;
   • the creation of a new comment with summar a new one when the build is finished.

Demo MR: seco-ne/yocto/layers/meta-seconorth-distro!750 (closed)

scripts/report_image_diff.py

@@ -2,6 +2,8 @@
 import argparse
 import fnmatch
 import logging
+import sys
+import time
 from difflib import unified_diff
 from typing import Optional
 
@@ -52,6 +54,21 @@ class BuildPipelines:
         if not build_pipeline:
             return {}
 
+        timeout = 1500  # 25 nin
+        check_interval = 5
+
+        not_rdy_status = ["pending", "running"]
+        if build_pipeline.status in not_rdy_status:
+            print(f"The build pipeline ({build_pipeline.web_url}) is not ready.")
+            print(f"Wait for it to complete:", end="", flush=True)
+
+        while build_pipeline.status in not_rdy_status:
+            print(".", end="", flush=True)
+            time.sleep(check_interval)
+            timeout -= check_interval
+            if timeout < 0:
+                sys.exit("timeout")
+
         ret = {}
         for bridge in build_pipeline.bridges.list():
             if not bridge.downstream_pipeline: