--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@bab6f811

Package: move SDK files to subfolder "sdk"

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@8bcaa426

scripts: Add delete_job_artifacts script to delete all artifacts from untagged jobs

For a given project all artifacts may be deleted. This was needed due to
space limitations on the gitlab server.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@28906544

Reduce artifact expiration of build jobs to 1 week

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@d404f257

Alphaplan: allow failure on import

Otherwise the parent pipeline is stuck in "running" state until the
manual action has been performed. See for reference:
https://gitlab.com/gitlab-org/gitlab/-/issues/361574

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@dd2a6d7d

Fix error message

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@04ac29ad

CI-Test: fix deploy-files job

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@22c519f1

Move FTP jobs to FTP runner

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@84d0616b

Add new deploy pipeline to Yocto

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@f0436868

Move reusable parts from CI-Test to common

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@3d3b4d09

Yocto: fix SDK package

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@37360288

Yocto: fix SDK build

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@2b6fc739

Package: add comment

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@95d6e368

alphaplan_fwr_import: add error handling

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@277d4af7

Package & deploy: do not allow missing dotenv file

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@3c68ccab

Alphaplan: add script for importing alphaplan data

The script has mostly been copied from alphaplan_fwr.py, but with
different input data.

Adding a job to the CI-test environment for testing this.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@3b3238a1

Alphaplan: add script for generating alphaplan data

This script has mostly been copied from alphaplan_fwr.py, but with
different input data (artifacts from the SoftwareStore deploy job) and
without actually importing the data to the Alphaplan database.

Adding a job to the CI-test environment for testing this.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@387b8184

Deploy: save variables in deploy.env

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@902c9276

Deploy: save rsync file list for later stages

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@15622149

CI-test: use caching between the pipeline stages

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a96a923e

CI-test: add deploy stages for SoftwareStore and FTP

Rename all stages to uppercase on this occasion, since we're adding
stages with names containing multiple words.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@056f1af1

Add package class and stage

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a1972724

Add script to query job logs filtered by status, tag, and log pattern

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@d501fed4

Update isort config

The "wp-gpios" assignment was missing for both platforms.

BCS 746-001170

The folder was still named 'garzfricke'.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@9982711f

Move deploy jobs to deploy runner

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@e48b4c0b

package_release: Do not fail if a file does not exist

Tobias Kahlki authored 2 years ago and Jonas Höppner committed 2 years ago

A duplicate of the seco logo was present in the logos dir.

(cherry picked from commit 348ae9ef8e5439fa62a2d9eceae008bdd5d96348)

Tobias Kahlki authored 2 years ago and Jonas Höppner committed 2 years ago

Both logos need to be enabled by default. The logo that is added to the
kernel is picked through the defconfig and the override in the kernel
include file.

(cherry picked from commit 4dd7c7bbba066a73d06fcafe091e7a46454560d8)

Tobias Kahlki authored 2 years ago and Jonas Höppner committed 2 years ago

Cherry-pick from af765e15

(cherry picked from commit e03aff3ac82c5aadaa1b755f80074c90a57accda)

Tobias Kahlki authored 2 years ago and Jonas Höppner committed 2 years ago

Updated the seco logo from the marketing's eps file.

(cherry picked from commit fa277d894d440a993ff24cb93f1a5c94b91fb102)
(cherry picked from commit a70c95c0bac3db1cb2d5c750232bdd2cef634527)

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@a5e27f6e

FTP upload: Add flag to hide FTP upload stage

This is a workaround to prevent customer releases being uploaded to our
public FTP area. It should be removed as soon as we support uploading to
different FTP target folders.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@543a2170

manifest_package: add release suffix

Customer specific manifest projects can now set the RELEASE_SUFFIX
variable, which then gets appended to the deployment folder names.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c763c43f

alphaplan_fwr: change variable "release_name_local" to "release_name"

Otherwise it can be assumed that it might be correlated with
outputdir_local, which is not the case.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@2d8d26a5

package_release: more refactoring

- Make output-dir an argument that can be passed multiple times instead
  of hard-coding two dirs with certain meanings.
- Remove LOCALDIR variable from package.env, because it is not used
  anywhere (only occurence was job class ".uploadsdkftp" which wasn't
  used anywhere either).

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@dc3698c8

package_release: refactoring

Improve readability of package_release.py and its submodule
generate_release_metadata.py.

- Add describing comments throughout the file.
- Remove duplicate code from functions. Simplify them, so that they do
  what their names say (and nothing more).
- Rename variables and function arguments to make their purpose clearer.
- Harmonize variable names with their corresponding command line arguments.
- Add pydoc for functions.
- Add type declarations to function arguments.
- Generate MD5 and metadata files in the source directories so that they
  can be copied along with the rest of the files.

--

Commit: seco-ne/yocto/infrastructure/gitlab-ci@c831d1da

get_integration_sources: check if CI is enabled

Trying to query CI variables on a project which does not have the CI
feature enabled leads to an HTTP 403 (forbidden) error.
Check whether CI is enabled using the "jobs_enabled" flag.

The workflow for the integration of the gitlab-ci submodule has changed.
Updated the gitlab-ci yaml file to use the new workflow and also updated
the CI to the current revision.

Note: The name of linux-guf was changed to linux-seconorth for the 5.15
kernel.

The VDD_3P3 should not be disabled, even not logically. Thus prevent
this by fixing the typo regulator_always_on to regulator-always-on.

Add support for the Garz& Fricke NALLINO platform. The NALLINO platform
is based on an NXP i.MX6 ULL. It includes support for a parallel RBG
Display, I2C Touch, RS232, RS485, microSD-Card.

Add support for the Garz & Fricke SANTINO-LT platform. The SANTINO-LT
platform is based on an NXP i.MX6 Solo or DualLite. It includes support
for a parallel RBG Display, CAN, Audio, I2C Touch, RS232, RS485,
microSD-Card.

Add support for the Garz& Fricke SANTARO platform. The SANTARO platform
is based on an NXP i.MX6 Solo, DualLite, Dual or Quad. It includes
support for an LVDS Display, Audio, RTC, Temperature Sensor, I2C Touch,
RS232, RS485, SD-Card.

Add support for the Garz& Fricke SANTOKA platform. The SANTOKA platform
is based on an NXP i.MX6 Solo, DualLite, Dual, Quad or QuadPlus. It
includes support for an LVDS Display, CAN, Audio, Temperature Sensor,
PCIe, RTC, I2C Touch, RS232, RS485, SD-Card and USB.

 Add support for the Garz& Fricke SANTINO platform. The SANTINO platform
 is based on an NXP i.MX6 Solo or DuaslLite. It includes support for an
 RGB Display, I2C Touch, RS232, RS485, SD-Cards and USB.

Add create Garz & Fricke directory below arch/arm/boot/dts to keep the
Garz & Fricke specific files separated. This is not the "common"
mainline approach for ARM dts definitions, however it is the practice
for ARM64 platform.

Add the common imx6qdl-san.dtsi include file that is common for all garz
& Fricke SAN* platforms. Later commits will add those separately.

* Add aliases for all platforms
* Add default std-out
* Add common audio parts like audio amplifier, audio-codec-clock,
  sgtl5000, ssi
* Add backlight
* Add gfplatdetect
* Add common regulators
* Configure AUDMUX for different platforms. The actual routing is done
  in the platform definition
* Add CAN
* Add common clock definition for the IPU
* Add Fast Ethernet Controller (FEC)
* Add I2C1, I2C2 and I2C3
* Add all pinctrl entries.
* Add UARTs
* Add USB ports
* Add USDHC
* Add watchdog reset pin

Felix Gerking authored 3 years ago and Clemens Terasa committed 3 years ago

BCS 746-000016

Jonas Höppner authored 3 years ago and Clemens Terasa committed 3 years ago

As per request form the marketing department, add a neutral Garz &
Fricke boot splash. This also fits into the Garz & Fricke group.

Also add a Flash-N-Go System logo.

BCS 746-000275
BCS 746-000468

This patchset adds Garz & Fricke specific version info to the mainline
kernel. It adds information to /proc/cpuinfo, exports symbols to be used
by a separate Garz & Fricke versioning module and exports the
ATAG_Version (used for the Flash-N-Go Boot bootloader version) to the
device tree.

On previous implementations we introduced the board revision and the
bootloader revision in the /proc/cpuinfo. Because of legacy consumers
namely the install script, the demos and maybe more tools, we decided to
add those also to this more mainline kernel.

The detection itself should come from a platform-detection module and
may be built out of kernel.

Add the variables for the board revision and the Flash-N-Go Boot
bootloader version, export them and show the contents in the
/proc/cpuinfo entry.

For a platform detection module more information of the actual hardware
is needed. Specifically the mxc_cpu_type and the iMX SoC revision
information is needed.

Avoid exporting the __mxc_cpu_type though, as it should not be
overwritten by an external module.

Thus add a new function mxc_get_cpu_type and export both, the newly
added function and the imx_get_soc_revision.

The modification of the machine name is not possible if the machine
name is not exported.  Thus export the machine_name variable.

By default the machine name comes from ATAGS or the machines_desc
structure that most often is created by the {DT_}MACHINE_START macros.

For our custom boards, however, we want to set the machine name
externally from a module. The alternative would be to creta a custom
board-* oder mach-* .c file in arch/arm/mach-imx/. This, however, would copy
most of the sensible platform code, which I regard as disencouraged.

A "platform-detection" module would be a clean and nicely separated
solution.

The ATAGS have various tags useful to determine platform specific data
already acquired from or generated by the bootloader.
In this case it is the revision tag that the bootloader might fill in
the ATAGS structure.

In the G&F case and using the Flash-N-Go Boot bootloader the bootloader
version is propagated in the ATAGS_REVISION tag.

To be able to handle this later after the ATAGS are destroyed parse
the ATAG_REVISION and add it to a atag-revision entry in the device
tree.

Carsten Behling authored 4 years ago and Clemens Terasa committed 3 years ago

On Garz & Fricke linux ports for historic and legacy reasons a
different virtual kernel address offset is being used.

Set the textofs from 0x00008000 to 0x00010000 and do so as well for the
KERNEL_RAM_VADDR.

TODO: Perhaps add a Garz & Fricke specific config or preprocessor
variable.

Jonas Höppner authored 4 years ago and Clemens Terasa committed 3 years ago

The lagacyfb_depth was set to 16 by default, but we want 32 to have all
colors of a 24bit display also during framebuffer access.

BCS 746-000038

This introduces a new panel driver called panel-dt for panels defined in
the device tree.
The goal is to create a panel driver similar to the panel-simple but
introducing more device tree features, and make the definition via
device tree definition mandatory, instead of overriding an exiting
pre-defined panel.
This would enable us to define most of the functionalities in the device
tree.

This approach has already been discussed in
https://patchwork.kernel.org/patch/10842593/

 but was not considered to
be valuable for the mainline linux kernel.

For some users and for us, however, this might be very helpful.

The question is still open, if the changes are better suited in the
panel-simple and perhaps introducing another variation, similar to the
panel-simple-dsi variant or in a separate file but this uses the latter
approach.

Steps taken:
* Introduce the new panel-dt file, copied from panel-simple.c.
  Set the license to GPLv2.
* Add the panel-dt driver to the Kconfig and conditionally build.
* Remove the panel descriptions as well as the DSI interface.
  With the removal of the  DSI interace the module can be initialized by
  a simple module_platform_driver() call.
* Remove all timings and modes.
* Also rewrite the probe routine to require the panel-timings entry
  similar to the panel-lvds case.
* Get the panel properties form the device tree instead of the
  simple-panel hard-coded way. Import the delays, dimensions (in mm),
  the bus-format and -flags and bits-per-color.
* Add the device tree bindings documentation for the panel-dt driver.
  Regression: I was not able to test it with `make dt_binding_check` or
  `make dtbs_check` as per Documentation/devicetree/writing-schema.md.
* Add the support for the special LVDS SEL6_8 GPIO that on some displays
  enables switches between 18 and 24 bit modes.
  This is somewhat Garz & Fricke specific.
* Regression: The "guf," compatible prefix is not yet defined in
  Documentation/devicetree/bindings/vendor-prefixes.yaml
* In recent kernel the rotation property was introduced in the panel
  device tree definitions and the panel-simple driver.
  Do the same for panel-dt and add the rotation information.
  However, I currently see no effect, but Weston should use this property
  since version 9.0.0
* Preliminary uncomment the drm_connector_set_panel_orientation call
  in the get_modes implementation.
* The bus flags can be supplied by the similar enum display_flags. the
  display_flags are part of the videomode and can be converted to
  bus_flags by using the drm_bus_flags_from_videomode() function.  By
  using this in the panel-dt driver we get the possibility to use the
  polarity settings like hsync-active, vsync-active, de-active,
  pixelclk-active and syncclk-active.

  BCS 746-000430
  BCS 746-000415
  BCS 746-000415

Signed-off-by: Clemens Terasa <clemens.terasa@garz-fricke.com>

Norman Stetter authored 4 years ago and Clemens Terasa committed 3 years ago

* pull in the PF1550 PMIC driver from linux-fscl at commit
f2c1392ff3473a396e4d177ff5ad368b9d6cd211

Add the DRM_BUS_FLAG_SYNC_DRIVE_{POS,NEG}EDGE as valid bus flag setting.

Let's assume following function call order form a panel driver:
of_parse_display_timing() -> videomode_from_timing() ->
drm_bus_flags_from_videomode() and let's assume the result will end up
as bus_flags in the parallel-display driver.

The of_parse_display_timing() will set the DISPLAY_FLAGS_SYNC_*
flags either when given explicitly using the "syncclk-active" device
tree property or implicitly when the "pixelclk-active" property is being
used.

This results in the DRM_BUS_FLAG_SYNC_DRIVE_{POS,NEG}EDGE being set. So
it is necessary to include the DRM_BUS_FLAG_SYNC_DRIVE_{POS,NEG}EDGE in
the validity check as well.

BCS 746-000430

Jonas Höppner authored 3 years ago and Clemens Terasa committed 3 years ago

Integrate software implementation for mark and space from guf mdb driver.
Rewrite most of the code:
 Use controller_parity from the uart register
 Use available flags instead of new enum.
 Resort some code to remove dupplicated code snippets.
 Rewrite logic, more readable.
 Move mark and space parity to inline functions
 Disable DMA when CMSPAR is set, still port DMA needs to be disable to
 always work, as disable DMA after open (when setting CMSPAR via ioctl)
 is not implemented. Close and reopen after setting the bit does work.

 Also cleanup the reconfiguration of ODD/EVEN during TX, as RX lost
 bytes in the mdb driver variant. Now the rx_interrupt routine is called
 until all buffers are empty before disalbe RX and change ODD/EVEN parity.
 This way the received bytes could be mapped to the parity config used
 during rx. Still a short moment RX is disabled which could lead to
 missed bits.

 Theres also a busy wait until TX queues are empty, which could probably
 lead to high system load on slow datarates, as this might wait one byte
 long.

BCS 746-000322

Stolen from by https://lore.kernel.org/linux-arm-kernel/1397544101-18135-6-git-send-email-wens@csie.org/

Add device tree support for the rfkil-gpio driver.
Do not use the "new" gpio lookup from the same patchset.

Carsten Behling authored 5 years ago and Clemens Terasa committed 3 years ago

It is good practice to show the world the provider of a kernel build.
For Garz & Fricke build we always used "-guf" do to here as well.

Link: https://lore.kernel.org/r/20220314112743.029192918@linuxfoundation.org


Tested-by: Florian Fainelli <f.fainelli@gmail.com>
Tested-by: Ron Economos <re@w6rz.net>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Tested-by: Linux Kernel Functional Testing <lkft@linaro.org>
Tested-by: Jon Hunter <jonathanh@nvidia.com>
Tested-by: Bagas Sanjaya <bagasdotme@gmail.com>
Tested-by: Sudip Mukherjee <sudip.mukherjee@codethink.co.uk>
Tested-by: Fox Chen <foxhlchen@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

commit 95932ab2ea07b79cdb33121e2f40ccda9e6a73b5 upstream.

Commit e2ae38cf3d91 ("vhost: fix hung thread due to erroneous iotlb
entries") tries to reject the IOTLB message whose size is zero. But
the size is not necessarily meaningful, one example is the batching
hint, so the commit breaks that.

Fixing this be reject zero size message only if the message is used to
update/invalidate the IOTLB.

Fixes: e2ae38cf3d91 ("vhost: fix hung thread due to erroneous iotlb entries")
Reported-by: Eli Cohen <elic@nvidia.com>
Cc: Anirudh Rayabharam <mail@anirudhrb.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
Link: https://lore.kernel.org/r/20220310075211.4801-1-jasowang@redhat.com


Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Tested-by: Eli Cohen <elic@nvidia.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

This reverts commit 2566a89b which is
commit a2614140dc0f467a83aa3bb4b6ee2d6480a76202 upstream.

The above change depends on upstream commit 0faf890f ("net: dsa:
drop rtnl_lock from dsa_slave_switchdev_event_work"), which is not
present in linux-5.15.y. Without that change, waiting for the switchdev
workqueue causes deadlocks on the rtnl_mutex.

Backporting the dependency commit isn't trivial/desirable, since it
requires that the following dependencies of the dependency are also
backported:

df405910 net: dsa: sja1105: wait for dynamic config command completion on writes too
eb016afd net: dsa: sja1105: serialize access to the dynamic config interface
2468346c net: mscc: ocelot: serialize access to the MAC table
f7eb4a1c net: dsa: b53: serialize access to the ARL table
cf231b43 net: dsa: lantiq_gswip: serialize access to the PCE registers
338a3a47 net: dsa: introduce locking for the address lists on CPU and DSA ports

and then this bugfix on top:

8940e6b669ca ("net: dsa: avoid call to __dev_set_promiscuity() while rtnl_mutex isn't held")

Reported-by: Daniel Suchy <danny@danysek.cz>
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

commit b81e0c23 upstream.

Drop various include not actually used in genhd.h itself, and
move the remaning includes closer together.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Link: https://lore.kernel.org/r/20210920123328.1399408-15-hch@lst.de


Signed-off-by: Jens Axboe <axboe@kernel.dk>
Reported-by: Sudip Mukherjee <sudipm.mukherjee@gmail.com&gt;a>
Reported-by: "H. Nikolaus Schaller" <hns@goldelico.com>
Reported-by: Guenter Roeck <linux@roeck-us.net>
Cc: "Maciej W. Rozycki" <macro@orcam.me.uk>
[ resolves MIPS build failure by luck, root cause needs to be fixed in
  Linus's tree properly, but this is needed for now to fix the build - gregkh ]
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

commit 74583f1b upstream.

Commit 67d96729 ("riscv: Update Canaan Kendryte K210 device tree")
incorrectly removed two entries from the PLIC interrupt-controller node's
interrupts-extended property.

The PLIC driver cannot know the mapping between hart contexts and hart ids,
so this information has to be provided by device tree, as specified by the
PLIC device tree binding.

The PLIC driver uses the interrupts-extended property, and initializes the
hart context registers in the exact same order as provided by the
interrupts-extended property.

In other words, if we don't specify the S-mode interrupts, the PLIC driver
will simply initialize the hart0 S-mode hart context with the hart1 M-mode
configuration. It is therefore essential to specify the S-mode IRQs even
though the system itself will only ever be running in M-mode.

Re-add the S-mode interrupts, so that we get working IRQs on hart1 again.

Cc: <stable@vger.kernel.org>
Fixes: 67d96729 ("riscv: Update Canaan Kendryte K210 device tree")
Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

commit 4e6f55120c7eccf6f9323bb681632e23cbcb3f3c upstream.

On TGL/RKL the BIOS likes to use some kind of bogus DBUF layout
that doesn't match what the spec recommends. With a single active
pipe that is not going to be a problem, but with multiple pipes
active skl_commit_modeset_enables() goes into an infinite loop
since it can't figure out any order in which it can commit the
pipes without causing DBUF overlaps between the planes.

We'd need some kind of extra DBUF defrag stage in between to
make the transition possible. But that is clearly way too complex
a solution, so in the name of simplicity let's just sanitize the
DBUF state by simply turning off all planes when we detect a
pipe encroaching on its neighbours' DBUF slices. We only have
to disable the primary planes as all other planes should have
already been disabled (if they somehow were enabled) by
earlier sanitization steps.

And for good measure let's also sanitize in case the DBUF
allocations of the pipes already seem to overlap each other.

Cc: <stable@vger.kernel.org> # v5.14+
Closes: https://gitlab.freedesktop.org/drm/intel/-/issues/4762


Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20220204141818.1900-3-ville.syrjala@linux.intel.com


Reviewed-by: Stanislav Lisovskiy <stanislav.lisovskiy@intel.com>
(cherry picked from commit 15512021eb3975a8c2366e3883337e252bb0eee5)
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

commit d96b34248c2f4ea8cd09286090f2f6f77102eaab upstream.

We don't allow send and balance/relocation to run in parallel in order
to prevent send failing or silently producing some bad stream. This is
because while send is using an extent (specially metadata) or about to
read a metadata extent and expecting it belongs to a specific parent
node, relocation can run, the transaction used for the relocation is
committed and the extent gets reallocated while send is still using the
extent, so it ends up with a different content than expected. This can
result in just failing to read a metadata extent due to failure of the
validation checks (parent transid, level, etc), failure to find a
backreference for a data extent, and other unexpected failures. Besides
reallocation, there's also a similar problem of an extent getting
discarded when it's unpinned after the transaction used for block group
relocation is committed.

The restriction between balance and send was added in commit 9e967495
("Btrfs: prevent send failures and crashes due to concurrent relocation"),
kernel 5.3, while the more general restriction between send and relocation
was added in commit 1cea5cf0 ("btrfs: ensure relocation never runs
while we have send operations running"), kernel 5.14.

Both send and relocation can be very long running operations. Relocation
because it has to do a lot of IO and expensive backreference lookups in
case there are many snapshots, and send due to read IO when operating on
very large trees. This makes it inconvenient for users and tools to deal
with scheduling both operations.

For zoned filesystem we also have automatic block group relocation, so
send can fail with -EAGAIN when users least expect it or send can end up
delaying the block group relocation for too long. In the future we might
also get the automatic block group relocation for non zoned filesystems.

This change makes it possible for send and relocation to run in parallel.
This is achieved the following way:

1) For all tree searches, send acquires a read lock on the commit root
   semaphore;

2) After each tree search, and before releasing the commit root semaphore,
   the leaf is cloned and placed in the search path (struct btrfs_path);

3) After releasing the commit root semaphore, the changed_cb() callback
   is invoked, which operates on the leaf and writes commands to the pipe
   (or file in case send/receive is not used with a pipe). It's important
   here to not hold a lock on the commit root semaphore, because if we did
   we could deadlock when sending and receiving to the same filesystem
   using a pipe - the send task blocks on the pipe because it's full, the
   receive task, which is the only consumer of the pipe, triggers a
   transaction commit when attempting to create a subvolume or reserve
   space for a write operation for example, but the transaction commit
   blocks trying to write lock the commit root semaphore, resulting in a
   deadlock;

4) Before moving to the next key, or advancing to the next change in case
   of an incremental send, check if a transaction used for relocation was
   committed (or is about to finish its commit). If so, release the search
   path(s) and restart the search, to where we were before, so that we
   don't operate on stale extent buffers. The search restarts are always
   possible because both the send and parent roots are RO, and no one can
   add, remove of update keys (change their offset) in RO trees - the
   only exception is deduplication, but that is still not allowed to run
   in parallel with send;

5) Periodically check if there is contention on the commit root semaphore,
   which means there is a transaction commit trying to write lock it, and
   release the semaphore and reschedule if there is contention, so as to
   avoid causing any significant delays to transaction commits.

This leaves some room for optimizations for send to have less path
releases and re searching the trees when there's relocation running, but
for now it's kept simple as it performs quite well (on very large trees
with resulting send streams in the order of a few hundred gigabytes).

Test case btrfs/187, from fstests, stresses relocation, send and
deduplication attempting to run in parallel, but without verifying if send
succeeds and if it produces correct streams. A new test case will be added
that exercises relocation happening in parallel with send and then checks
that send succeeds and the resulting streams are correct.

A final note is that for now this still leaves the mutual exclusion
between send operations and deduplication on files belonging to a root
used by send operations. A solution for that will be slightly more complex
but it will eventually be built on top of this change.

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>