State of the union

Tim-Philipp Müller started off the conference with the "State of the Union" talk about the release schedule, notable features of the current 1.14 release, upcoming features for the 1.16 release in December 2018, and planned future development. The new features are too numerous to list them all, but a few highlights for us are V4L2 codec improvements, OpenGL dmabuf uploader updates for zero-copy, the Meson build system, which should be almost complete for 1.16, and the upcoming move to gitlab.freedesktop.org.

Debugging, tracing, and testing

Our colleague Michael Olbrich gave an overview of his GDB plugins created to ease post-mortem debugging of crashed GStreamer processes. The talk was well received, according to feedback received during the remainder of the conference. Marcin Kolny of Amazon gave a talk about profiling with HawkTracer, and Michał Budzyński presented about the scenario driven multimedia testing setup used for full-system testing at Samsung R&D Poland.

Video4Linux

Ezequiel Garcia and Nicolas Dufresne of Collabora spoke about how the GStreamer V4L2 video codec plugins use the stateful and stateless V4L2 codec APIs, which are currently under review. Nicolas Dufresne also updated everyone on V4L2 in his usual GStreamer V4L2 lightning talk. The annual reading of the V4L2 developer patch count high score wasn't enough to boost any egos this time, but is a sign that GStreamer V4L2 is stable and useful enough, unchanged, for many of our embedded use cases.

Feature and performance improvements

Furthermore, some of the upcoming and future improvements were presented in detail. Sebastian Dröge introduced his work on thread-sharing between elements to allow massively parallel pipelines with less overhead. He also reported about the current state of GStreamer & Rust integration. George Kiagiadakis from Collabora reported about the non-interleaved audio format support in 1.16, and Edward Hervey mastered technical difficulties, improvising his talk about Closed Captions support without any slides. He also gave a talk about future stream selection and collection support in playbin3. Mathieu Duponchelle of Centricular reported about the ongoing GStreamer Documentation efforts, which should lead to a unified developer portal soon.

Web technologies

One of the more prominent topics this year was web technologies, starting off with a talk by Philippe Normand from Igalia about GStreamer based Multimedia handling in WebKitGTK and WPE (WebKit port for embedded), a talk by Manish Goregaokar about Servo Media, the WebAudio API implementation for Mozilla's Servo Browser Engine, as well as a lightning talk by Victor Jáquez from Igalia about Servo and GStreamer integration and two lightning talks about WebRTC progress updates and data channel support by Mathieu Duponchelle and Matthew Waters of Centricular.

Various: streaming video, neural networks, signal analysis

Wim Taymans presented the current work in progress on PipeWire used as audio server.

Streaming and cloud topics were covered by talks about cloud-based live video handling by Matthew Clark and Luke Moscrop from BBC, a talk about streaming Direct3D video games on Windows by Florian Nierhaus from Bebo, and a talk by Håvard Graff from Pexip about their Microsoft Teams Connector.

There were a few talks about integrating deep neural networks with GStreamer, mostly just for inferencing purposes, implementing different abstraction layers for different deep learning backends. Time will tell whether the different approaches will move towards unification or whether one of them will become accepted.

Further lightning talks were concerned with using GStreamer on various platforms, such as webOS OSE, iOS, or Windows (DirectShow), as well as on vendor frameworks, e.g. Intel MSDK or Nvidia CUDA, or even using GStreamer as a backend for FFmpeg by implementing libav API on top, and an entertaining lightning talk about the history of MeTV, a live TV viewing application, up to the latest rewrite in Rust using GStreamer.

One surprise lightning talk was given in the form of a question by Martin Ling from the Edinburgh Hacklab, who wondered whether GStreamer could be a good fit for use in sigrok.

The talks were recorded by Ubicast and can be found in the GStreamer Conference 2018 channel.