Graphics driver background

Graphics support in embedded devices has changed quite a bit over the last few years. Let's dive a bit into the history of graphics hardware and the accompanying software support to set the scene.

Historically, embedded devices would only have very minimal graphics support, if at all. They could drive simple LC displays and all the work of actually producing the picture to show on the display had to be handled by the CPU, with no hardware acceleration whatsoever. On one hand, this was convenient, as the CPU is very flexible in what it can do and driver support for the simple hardware units driving the displays didn't require much effort: all they needed to do was to get a piece of bitmap memory filled by the CPU out on the display data bus with the right timing. On the other hand, the limited processing power available on the CPU set very strict bounds on what was possible to do on the graphics side without impacting the primary use of the device, as most devices are built to fulfill other jobs besides displaying pretty graphics.

Around 15 years ago Moore's law still progressed at full speed and it was finally possible to fit real Graphics Processing Units (GPUs) into the space, power and cost envelope required in the embedded space. With GPUs taking over most of the actual work of drawing the content to show on the display, the CPU was free again for other jobs and while at the same time providing much richer user interfaces.

This however came with a new cost: the complexity of the software needed to drive those GPUs is a huge step up from the simple drawing implementation on the CPU. Also, the GPU marketing segment is very competitive, with multiple companies trying to sell GPUs for use in embedded systems. In this environment, most of those companies favored a closed-source approach for the driver software, in order to not reveal any secrets that may set them apart from the competition.

Issues with closed-source drivers in embedded devices

Embedded systems have some unique requirements in that they are usually designed to do a very specific task at the lowest possible price point. So there is little room to compensate inefficiencies in the processing pipelines with raw compute power. Thus the software has to be optimized to provide the most efficient processing pipelines possible. Also, most of them are designed to be used for a very long time. With more and more of those devices connected to the network, this means it must be easy to update the software running on them in order to be able to fix functional or security issues.

Both of those requirements are very hard to meet when depending on closed source components that one has no direct control over. Even if the GPU vendor is able and willing to provide support, changes to the driver software can take a very long time, leading to failures to meet the window of opportunity where a device can be brought into the market. Having to take into account the requirements of the closed-source bits in the system may also prevent timely updates. This may apply to new system components like kernel and base libraries in the system when those components leave their support period.

Closed-source components might also bring in assumptions and dependencies that severely limit the ability to alter the software stack later on in the lifetime of the device.

Alternative solutions

The Pengutronix graphics team has an extensive track record in building open-source drivers for these hardware units.

On the GPU side, this journey started when some of our customers were not willing to accept the risk of having a third-party closed-source component at the core of their product. They worked with Pengutronix to bring to life a open-source alternative. The NXP i.MX line of SoCs is very popular in the industrial and medical embedded ecosystems, as it ticks a lot of the required feature boxes. Starting with the i.MX6 those SoCs include GPUs from VeriSilicon (formerly Vivante). Some volunteers in the free software community had already started a reverse-engineering effort targeting these GPUs, called Etnaviv. Pengutronix jumped on-board, helped to get it into shape and provide a commercial quality open alternative to the closed-source vendor driver.

Today the Etnaviv driver stack is in use in a breadth of embedded devices, from medical appliances to in-flight entertainment systems to automotive and industrial control systems. While Pengutronix, as a strong partner with a really deep technical expertise in this area, is always standing by to help you along if the need arises, all the components are available as open-source and can be used, extended and improved by anyone, eliminating the risk of vendor lock-in for this critical component.

The Etnaviv driver stack integrates in the upstream frameworks like DRM in the Linux kernel and Mesa on the userspace side, so it's quite close in features and behavior to the general Linux desktop graphics stack, making embedded projects less special.

i.MX8MP GPU enablement with upstream drivers

Recently, the Pengutronix graphics team was tasked with enabling the GPU on the NXP i.MX8M Plus, which is currently in pre-production. This SoC is very interesting to many industrial customers, as the i.MX8M line is a successor to the really popular i.MX6 series. A lot of the software support, written and refined over the last 10 years, can be reused, significantly reducing the effort to get the basic system functionality working. The i.MX8MP also packs some interesting features like CAN FD and TSN Ethernet.

The GPU is a slightly different variant of the GC7000L, found in the already supported i.MX8MQ SoC. While some small adaptions were needed to run the open-source driver stack on this specific GPU, it took the Pengutronix graphics team less than a week total effort to get the GPU up and running. Years of experience working on those parts of the software stack really helped in getting this working quickly.

If you are planning to start a new project involving graphics input/output support we are happy to talk to you!