Home Multimedia and Automation Systems

My first talk of the day was Jan Schmidt talking about home automation. As he is working on GStreamer topics during his daytime job, he looked at which home multimedia solutions are around that work with GStreamer.

One thing to look at is distributed speakers. In order to synchronize sound on the speakers, you need precisely synchronized clocks. GStreamer has a proprietary protocol for that, but there are also NTP and PTP around for a more standardized mechanism. An open source project using GStreamer is Aurena, synchronizing the different outputs to something below 1 ms. In addition to audio, it can synchronize video as well, and with that synchronization accuracy, it is frame synchronized.

The next thing he looked at is cameras: you can basically take Raspberry Pis or any kind of RTSP based cameras, so this part is easy.

A more challenging task was to use different synchronized microphone arrays to find out the location of audio sources (realtime microphone array processing). After some experiments with Android devices, he also went to a GStreamer based variant, by wrapping some GStreamer code around the ODAS project.

Integrate ROS into EdgeX

ROS, the Robot Operating System, is an interesting technology if you have an automation background, however, so far we didn't do much with it (basically due to cross compiling issues because of their crappy build system), but it is always a good idea to see what people are doing with it, so I joined Tejun Chen's talk.

He gave a short introduction into ROS and the EdgeX project, which is a Linux Foundation project to create an IoT edge platform based on a set of REST services. They are building quite a large infrastructure, and one of the goals is to bridge data from ROS into the cloud and use machine learning and AI techniques on top of that. However, my impression is not that this project is of any real help for our work on industrial devices...

Running Linux on Constrained IoT Devices

Rui Miguel Silva then talked about their activities to create a reference platform for constrained devices/rich IoT devices. The motivation for their work is that Linux has a robust networking stack with wide application support - something that is not necessarily true with microcontroller stacks. The hardware they are working on is either standalone Cortex-M style devices or coprocessors in larger systems, operated with OpenAMP.

One interesting target is the new Cortex-M32, which is used to bring IoT data to the Microsoft Azure cloud, using enhanced security features. The team is using Yocto (poky-tiny) and OpenEmbedded for their tinification activities, currently their CramFS-XIP is about 8 MB. The read/write filesystem on SPI-NOR flash uses LittleFS, a block-pair filesystem which is currently being ported to Linux.

In his demo, he showed the boot of a corstone 700 system which basically boots instantaneously. Currently, there are no SoCs available, just FPGA based reference platforms.

Taken that Pengutronix has mainlined Cortex-M support several years ago, it's interesting to see that people actually make use of it - at the time we mainlined the code, we didn't even have hardware available where it would have made sense commercially. However, back at that time, the expectation was that some day there would be hardware capable of running Linux on controller style hardware and making use of the high quality of the kernel even on low-end hardware; this is obviously what happens now.

Offloading Network Traffic Classification to Hardware

Maxime Chevallier reported about his activities. He explained how packages do normally move from the network hardware through the networking stack up to the applications. However, the preprocessing steps take a lot of CPU time and can be offloaded into hardware.

In order to offload things, the kernel looks at flows - a group of packets that have a common source and destination. TC (traffic control) is the tool to configure traffic shaping, scheduling, policing and filtering; 'tc flower' can operate on ingress filters and can either use hardware or software. The main interface to interact with the drivers is ethtool, and in the future, offloading will also be possible with netfilter. When offloading flows to hardware, the kernel might not see any packets any more, so special care needs to be taken that it doesn't miss important information. Offloading works by classifying traffic in the switch and then moving it into the right queues. The actual matching happens on-die; the parsing and classifying against some rules create the necessary information to decide what finally happens with the packet. With receive side scaling (RSS), the traffic can be spread across multiple CPUs, but one needs to make sure the same flow ends up on the same CPU.

Maxime is working with PPv2, on Marvell SoCs such as Armada 70xx and 80xx. However, it turned out that parts of the classification engines are so complex that you cannot represent everything with the existing userspace interfaces, but within the given constraints, many classification steps can be offloaded to hardware.

Authenticated and Encrypted Storage on Embedded Linux

In the last slot of the day, Jan Lübbe talked about possibilities for authenticated and encrypted storage on embedded Linux systems. There are basically two lowlevel mechanisms that can be used for storing things: block devices and MTD devices (NAND). On top of that, several mechanisms exist to authenticate/encrypt things. In all variants, authentication, encryption and authenticated encryption are necessary; Jan then explained the different variants that exist in Linux:

dm-verity works between the filesystem and the block layer; it is read-only and is for example used by Chrome OS and Android for the rootfs and can be combined with ext4, SquashFS or EROFS. dm-verity works fine for read-only data (such as the root filesystem).

At the same layer in devicemapper, there is dm-integrity. For each block of data, there is an additional block of data where the integrity data can be stored, so the mechanism allows read/write access.

dm-crypt makes use of sector based encryption on block devices. The mechanism does only crypt, not authenticate, because that would need additional space. This makes the mechanism the best choice on read/write block devices (such as laptops). Recently, a mechanism that combines dm-integrity and dm-crypt was added and makes it possible to also authenticate the encrypted data. However, unfortunately the mechanism is not secure against replay attacks, as you can inject an old block.

On filesystem layer, there is fs-verity ("dm-verity for files"); the system makes sure that every time a file is accessed, it is unmodified. The main user is Android, for .apk authentication. The mechanism could be integrated with IMA/EVM to improve performance. Also on filesystem layer, fscrypt does file-based encryption and allows different keys for multiple users. There is no authentication and files can be removed without the key.

On NAND, Pengutronix recently implemented UBIFS Authentication; it makes use of copy-on-write, which is necessary on NAND ("wandering tree"). The mechanism currently provides the only fs which authenticates full data and metadata.

IMA/EVM is there since 2009/2011 and is a layer on top of the filesystem. It is a mechanism used to make remote attestation possible, so a remote machine can make sure a device is running the intended software. However, it is possible to for example overwrite /sbin/init with /bin/sh (which is also integer)...

One important question is where to store keys in a secure way, so an attacker does not get access. While this is easy on laptops (where the user can enter the password), it is more difficult on embedded systems. Modern hardware often has hardware to wrap keys with fixed per-device keys or store the key in trusted execution environments on on a TPM.

For developers, it is important to think about the case that locked down systems have to be debugged somehow; a possible way to do this is to have a field return mode that implements an authenticated method to erase the keys for private data and disable verified boot.

As locked down devices tend to become really, really complex, a good advise is to avoid as much complexity as possible and only select the necessary components.

End Game & Party

Thanks to Tim Bird, ELC-E always ends with the famous end game, and afterwards many of the attendees met at the party on a boat on Rhône river.