Summary: Automated Testing Summit 2018
The Automated Testing Summit (ATS) was a co-located event at the ELC-E 2018 in Edinburgh. The summit was organized by Tim Bird and Kevin Hilman and brought together over 35 (embedded) testing experts from different projects and companies.
The Idea
At ELC-E 2017 Andrew Murray did his BoF Farming Together. This BoF showed that lots of companies and individuals in the embedded world are running some kind of board farm: They all have the need to remote-control real hardware and test software on that hardware.
During the discussion the idea of more collaboration in this area arose. Tim Bird suggested to collect e-mail addresses in the eLinux.org Wiki. Afterwards, the existing automated-testing mailing list run by the Yocto Project was used for further coordination.
The Summit
During the next year, Tim Bird and Kevin Hilman organized the Automated Testing Summit as a co-located event during the ELC-E 2018. Linaro and DENSO TEN sponsored the summit.
The summit brought together 35 testing experts from over 22 projects. Most of the people represented testing of embedded devices. But some also represented high-level projects that test software without hardware. eLinux.org has a list of represented projects.
During eight hours of productive discussion, we worked on:
- defining a common terminology and reference testing workflow
- identifying interfaces where we could share test suites, build and test artifacts (by creating an interchange format)
- learning how other board farms are built and used
- and many smaller topics
(For some attendees the discussion even continued in a pub near the venue.) The minutes are available on the wiki as well.
Action Items
Taking into account the number of companies, projects and individuals involved in the summit the list of resulting action items is quite long. The summit also showed that there are even more possibilities to collaborate in testing -- even if some parts of the process are proprietary.
In the short term, the following action items will probably have the most relevance for us at Pengutronix.
pdudaemon
All projects that need to control embedded hardware switch the board's power supply in one or another way. The attendees agreed that a common interface would help all projects: a driver only has to be developed once. The idea is to support pdudaemon as the common interface to control power switches and to upstream drivers for new switches to pdudaemon.
Design for Testing Document
The attendees agreed that a design for embedded testing document could help test engineers to communicate with the hardware developers. The document should describe measures that ensure software development and automated testing are possible.
Since the Automated Testing has no formal body yet, the document could be signed by multiple projects, companies and individuals that support its contents.
I volunteered to coordinate the creation of the document. I will post an update as soon as there are any resources available online.
Next meeting
The attendees agreed that we would like to meet again next year. The meeting will probably be during the ELC-E 2019 in Lyon, France.
Further Readings
Pengutronix at Electronica 2024 in Munich
Electronica trade fair in Munich, Germany is just around the corner and Pengutronix is currently gearing up to showcase some of our latest topics and developments. You find us in Hall B4 Booth 102 (map).
Pengutronix at FrOSCon 2024
On August 17th and 18th, 2024, it's that time again: FrOSCon will take place at the Bonn-Rhein-Sieg University of Applied Sciences in Sankt Augustin - and Pengutronix will be there again as a Partner.
Pengutronix at Embedded World 2024
Meet Pengutronix at the Embedded World 2024 in Nurnberg! You find us, as always, in hall 4, booth 4-261. As usual, we will be showing demonstrators on current topics at our exhibition stand.