Earlier Mainlining Approaches

In 2014, two parallel attempts by David Vincent and Stefan Müller-Klieser aimed to provide a recipe for Barebox. Unfortunately, these efforts stalled after the initial submission, at a time when Pengutronix was not yet involved in Yocto/OpenEmbedded.

In the following years, support for barebox fragmented across various external layers like meta-fsl-arm, meta-phtytec, the dedicated meta-barebox and finally 'our' layer meta-ptx.

From the Idea to the Initial Recipes

The plan to bring Barebox into OE-Core grew gradually as we maintained our Barebox recipe in meta-ptx. By 2019, it became a concrete goal. The actual effort kicked off during the first post-COVID Pengutronix TechWeek in September 2021.

By the end of the 2021 TechWeek, we had developed a Barebox recipe based on meta-ptx. We adapted mechanisms from other layers to better fit a more generic setup. One notable improvement was splitting out the barebox-tools recipe, to build the tools separately and for both target and native. As the target and native tools usually need to link against different libraries, the bitbake recipe turned out quite complex.

This was simplified a great deal by our Barebox developer Ahmad Fatoum, stepping in to rework the barebox tools build system for this purpose.

We planned to wait until Ahmad's patches made it into a release, avoiding the need for barebox patches in the first OE-Core submission.

Initial Submission: Concerns and Challenges

But things don't always go according to plan, priorities shift and I became busy with other projects.

In December 2022, Pengutronix developer Marco Felsch took over and a few internal patch reviews later, Marco sent our first series of two patches to the OE-Core mailing list by February 2023:

[OE-Core] [PATCH 1/2] barebox: add initial support
[OE-Core] [PATCH 2/2] barebox-tools: add initial barebox tools support

This could have been a simple story, but then Richard Purdie, the Yocto Project main architect, asked the critical question:

"In order to add something to OE-Core, we need to see it being used by a reasonable portion of the ecosystem. Is there enough usage of barebox on common boards that justifies this?"

I responded by highlighting Barebox's maturity and wide adoption, its advantages as a bootloader, and how it adds little maintenance overhead. Barebox, I argued, aligns with OE-Core's goal of providing essential recipes for device bring-up.

While OE developer Otavio Salvador supported the idea of including Barebox in OE, highlighting the potential for reuse and integration with existing U-Boot-centric tooling, another OE-Core developer, Alexander Kanavin, was more skeptical. He raised concerns about the maintainability of adding another bootloader.

However, after a productive discussion on IRC, we reached a consensus: a strong maintenance commitment, combined with QEMU-based test cases for the OE-selftest suite, could justify Barebox's inclusion in OE-Core.

Adding Test Cases

In the following month, we worked to address the concerns raised and the issues identified by the autobuilder. I fixed barebox' musl support and other minor compatibility issues, added myself to the maintainers.inc file, and set reasonable defconfigs for QEMU machines.

I thought, since I was already asked to add QEMU-based test cases for the Barebox recipe — while the U-Boot recipe never had any — it would be a smart move to add tests for both (targeting arm, aarch64, and x86_64). It quickly became clear that testing a bootloader hadn't been done in OE-Core before, and we needed to make some additions to the test code, such as support for capturing the bootloader prompt.

Once those adjustments were made, running the first OE bootloader tests became as easy as:

$ MACHINE=qemuarm64 oe-selftest -r uboot barebox -v -K

This work finally resulted in a v2 series of 7 patches and the first review comment I got from Alex sounded quite promising:

Thanks for persevering with this! The patchset is mostly fine (I'll write a couple comments now), and just needs to get a verdict from the autobuilder.

A further issue we had to address was that with barebox there were suddenly two concurrent providers of virtual/bootloader for the OE-Core QEMU machines. To handle this properly, we adapted the BAREBOX_CONFIG parsing and set PREFERRED_PROVIDER_virtual/bootloader ?= "u-boot" in the QEMU machine configurations to ensure the U-Boot remained the default bootloader.

But, despite this and much more fixed in v3, the autobuilder still failed. Unfortunately, other changes in the test setup interfered with the results, making further debugging quite difficult.

Techweek 2024 — A Turning Point Decision

Our 2024 Techweek in Goslar in the Harz Mountains finally gave me the opportunity to look again into the autobuilder issues. Updating the recipe to a more recent version of Barebox and running the test suite locally revealed a couple of additional issues, which we managed to fix.

However, I couldn't resolve a remaining issue with my adaptations for the hand-crafted QEMU boot log reading test code, which were necessary to handle CRLF line breaks under UEFI. We ultimately decided that, without a proper embedded testing framework like labgrid, it wouldn't be worth the effort to further debug x86 support in the self-tests for now — especially since both U-Boot and Barebox are predominantly used on non-x86 targets. As a result, we removed the x86 tests along with the log readout modifications and were able to run the entire oe-selftest suite without errors.

Barebox: A Class of Its Own

With renewed optimism, and the outstanding review comments of v3 addressed, I finally submitted v4 on 10th September 2024.

In a fruitful discussion with Alexander Kanavin about the purpose and usage of .inc files in our recipe we agreed that although the u-boot recipe has a long history of misusing .inc files by including it across layers as 'external API', a new barebox recipe should do better and put the generic code into a barebox.bbclass.

However, the autobuilder still failed early. This finally turned out to be a false-positive in the maintainer checks that fail to handle recipes that are not parsed by default (e.g. due to PREFERRED_PROVIDER settings). Gladly we were able to fix this and, together with some other minor changes (e.g. for UNPACKDIR), with v7 we had all the missing pieces together and were cautiously optimistic that we could pass both the human review and the autobuilder.

The Last Tough Autobuilder Nut

But, the autobuilder logs that we got from Richard the same day showed that the tests don't pass on an ARM build server of all places: Both barebox and U-Boot hang while running in QEMU without providing any meaningful error output.

After some tinkering and help from Richard, Ahmad finally managed to reproduce this on a ARMv8 SBC running Debian, but only if KVM (Kernel-based Virtual Machine) was enabled.

After some local debugging, Ahmad isolated the problem into two lines of ARM A64 assembly that trigger a data abort under KVM, but work fine without and brought this to the attention of the Linux KVM maintainers.

In the ensuing discussion, it turns out that neither U-Boot nor barebox will be able to run under KVM unmodified, so for the OE-Core test suite, it was decided that explicitly disabling KVM would be the best course of action.

This may change though in the future: Ahmad has since posted patches for barebox adding KVM compatibility and reported the issue for u-boot in case the community is interested in porting the barebox changes.

Mainline!

And finally, for the v8 series, the autobuilder gave green lights and with a tiny extra round Richard accepted the v9 series for OE-Core by the 11. October 2024. 5 years after the initial idea, 1.5 years after the initial submission attempt.

(But the actual journey of maintaining barebox in OE-Core has just begun!...)