Release notes for the Genode OS Framework 24.11

During the discussion of this year's road-map roughly one year ago, the usability concerns of Sculpt OS stood out. Besides suspend/resume, which we addressed earlier this year, multi-monitor support ranked highest on the list of desires. We are more than happy to wrap up the year with the realization of this feature. Section Multi-monitor support presents the many facets and outcomes of this intensive line of work.

Over the course of 2024, our Goa SDK has received tremendous advances, which make the development, porting, debugging, and publishing of software for Genode - and Sculpt OS in particular - a breeze. So far however, the learning curve for getting started remained rather steep because the underlying concepts largely deviate from the beaten tracks known from traditional operating systems. Even though there is plenty of documentation, it is rather scattered and overwhelming. All the more happy we are to announce that the current release is accompanied by a new book "Genode Applications" that can be downloaded for free and provides a smooth gateway for application developers into the world of Genode (Section New "Genode Applications" book).

Regarding hardware-related technical topics, the release focuses on the ARM-based i.MX SoC family, taking our ambition to run Sculpt OS on the MNT Pocket Reform laptop as guiding theme. Section Device drivers and platforms covers our driver and platform-related work in detail.

New "Genode Applications" book Multi-monitor support   Tickless GUI server and display drivers   Paving the ground for tearing-free motion   Mirrored and panoramic monitor setups   Sculpt OS integration   Revised inter-component interfaces   Current state and ongoing work Device drivers and platforms   Linux device-driver environment (DDE)     MNT Pocket Reform   Platform driver for i.MX 8M Plus   Wifi management rework   Audio driver updated to OpenBSD 7.6   AVX and hardware-based AES in virtual machines Build system and tools

New "Genode Applications" book

Complementary to our Genode Foundations and Genode Platforms books, we have been working on a new book that concentrates on application development. Genode Applications centers on the Goa SDK that we introduced with Genode 19.11 and which has seen significant improvements over the past year (23.08, 24.02, 24.08).

The book intends to provide a beginner-friendly starting point for application development and porting for Genode and Sculpt OS in particular. It starts off with a getting-started tutorial for the Goa tool, and further recapitulates Genode's architecture and a subset of its libraries, components, and conventions such as the C runtime, VFS, NIC router, and package management. With these essentials in place, the book is topped off with instructions for application debugging and a collection of advanced tutorials.

Aligned with the release of Sculpt 24.10, we updated the Goa tool with the corresponding depot archive versions. Furthermore, the Sculpt-integrated and updated Goa testbed preset is now prepared for remote debugging.

Multi-monitor support

Among the users of the Genode-based Sculpt OS, the flexible use of multiple monitors was certainly the most longed-after desire raised during our public road-map discussion roughly one year ago. We quickly identified that a profound solution cannot focus on piecemeal extensions of individual components but must embrace an architectural step forward. The step turned out being quite a leap. In fact, besides reconsidering the roles of display and input drivers in version 20.08, the GUI stack has remained largely unchanged since version 14.08. So we took our multi-monitor ambitions as welcome opportunity to incorporate our experiences of the past ten years into a new design for the next ten years.

Tickless GUI server and display drivers

Up to now, the nitpicker GUI server as well as the display drivers used to operate in a strictly periodic fashion. At a rate of 10 milliseconds, the GUI server would route input events to the designated GUI clients and flush graphical changes of the GUI clients to the display driver. This simple mode of execution has benefits such as the natural ability of batching input events and the robustness of the GUI server against overload situations. However, in Sculpt OS, we observed that the fixed rate induces little but constant load into an otherwise idle system, rendering energy-saving regimes of modern CPUs less effective than they could be. This problem would become amplified in the presence of multiple output channels operating at independent frame rates. Moreover, with panel self-refresh support of recent Intel graphics devices, the notion of a fixed continuous frame rate has become antiquated.

Hence, it was time to move to a tickless GUI-server design where the GUI server acts as a mere broker between events triggered by applications (e.g., pushing pixels) and drivers (e.g., occurrence of input, scanout to a display). Depending on the behavior of its clients (GUI applications and drivers alike), the GUI server notifies the affected parties about events of interest but does not assert an active role.

For example, if a display driver does not observe any changed pixels for 50 ms, it goes to sleep. Once an application updates pixels affecting a display, the GUI server wakes up the respective display driver, which then polls the pixels at a driver-defined frame rate until observing when the pixels remain static for 50 ms. Vice versa, the point in time when a display driver requests updated pixels is reflected as a sync event to GUI applications visible on that display, enabling such applications to synchronize their output to the frame rate of the driver. The GUI server thereby asserts the role of steering the sleep cycles of drivers and applications. Unless anything happens on screen, neither the GUI server nor the display driver are active. When two applications are visible on distinct monitors, the change of one application does not induce any activity regarding the unrelated display. This allows for scaling up the number of monitors without increasing the idle CPU load.

This change implies that the former practice of using sync signals as a time source for application-side animation timing is no longer viable. Sync signals occur only when a driver is active after all. GUI applications may best use sync signals for redraw scheduling but need to use a real time source as basis for calculating the progress of animations.

Paving the ground for tearing-free motion

Tearing artifacts during animations are rightfully frowned upon. It goes without saying that we strive to attain tearing-free motion in Genode. Two preconditions must be met. First, the GUI server must be able to get hold of a consistent picture at any time. Second, the flushing of the picture to the display hardware must be timed with vsync of the physical display.

Up to now, the GUI stack was unable to meet the first precondition by design. If the picture is composed of multiple clients, the visual representation of each client must be present in a consistent state. The textures used as input of the compositing of the final picture are buffers shared between server and client. Even though clients traditionally employ double-buffering to hide intermediate drawing states, the final back-to-front copy into the shared buffer violated the consistency of the buffer during the client-side copy operation - when looking at the buffer from the server side. To overcome this deficiency, we have now equipped the GUI server with atomic blitting and panning operations, which support atomic updates in two fashions.

Atomic back-to-front blitting allows GUI clients that partially update their user interface - like regular application dialogs - to implement double buffering by placing both the back buffer and front buffer within the GUI session's shared buffer and configuring a view that shows only the front buffer. The new blit operation (Framebuffer::Session::blit) allows the client to atomically flush pixels from the back buffer to the front buffer.

Atomic buffer flipping allows GUI clients that always update all pixels - like a media player or a game - to leverage panning (Framebuffer::Session::panning) to atomically redirect the displayed pixels to a different portion of the GUI session's shared buffer without any copy operation needed. The buffer contains two frames, the displayed one and the next one. Once the next frame is complete, the client changes the panning position to the portion containing the next frame.

Almost all GUI clients of the Genode OS framework have been updated to use these new facilities.

The vsync timing as the second precondition for tearing-free motion lies in the hands of the display driver, which can in principle capture pixel updates from the GUI server driven by vsync interrupts. In the presence of multiple monitors with different vsync rates, a GUI client may deliberately select a synchronization source (Framebuffer::Session::sync_source). That said, even though the interfaces are in place, vsync timing is not yet provided by the current display drivers.

Mirrored and panoramic monitor setups

A display driver interacts with the nitpicker GUI server as a capture client. One can think of a display driver as a screen-capturing application. Up until now, the nitpicker GUI server handed out the same picture to each capture client. So each client obtained a mirror of the same picture. By subjecting each client to a policy defining a window within a larger panorama, a driver creating one capture session per monitor becomes able to display the larger panorama spanning the connected displays. The assignment of capture clients to different parts of the panorama follows Genode's established label-based policy-selection approach as explained in the documentation of the nitpicker GUI server.

Special care has been taken to ensure that the pointer is always visible. It cannot be moved to any area that is not captured. Should the only capture client displaying the pointer disappear, the pointer is warped to the center of (any) remaining capture client.

A mirrored monitor setup can in principle be attained by placing multiple capture clients at the same part of nitpicker's panorama. However, there is a better way: Our Intel display-driver component supports both discrete and merged output channels. The driver's configuration subsumes all connectors listed within a <merge> node as a single encompassing capture session at the GUI server. The mirroring of the picture is done by the hardware. Each connector declared outside the <merge> node is handled as a discrete capture session labeled after the corresponding connector. The driver's documentation describes the configuration in detail.

Sculpt OS integration

All the changes described above are featured in the recently released Sculpt OS version 24.10, which gives the user the ability to attain mirrored or panoramic monitor setups or a combination thereof by the means of manual configuration or by using interactive controls.

You can find the multi-monitor use of Sculpt OS covered by the documentation.

Revised inter-component interfaces

Strict resource partitioning between GUI clients

Even though Genode gives server components the opportunity to strictly operate on client-provided resources only, the two prominent GUI servers - nitpicker and the window manager (wm) - did not leverage these mechanisms to full extent. In particular the wm eschewed strict resource accounting by paying out of its own pocket. This deficiency has been rectified by the current release, thereby making the GUI stack much more robust against potential resource denial-of-service issues. Both the nitpicker GUI server and the window manager now account all allocations to the resource budgets of the respective clients. This change has the effect that GUI clients must now be equipped with the actual cap and RAM quotas needed.

Note that not all central parts of the GUI stack operate on client-provided resources. In particular, a window decorator is a mere client of the window manager despite playing a role transcending multiple applications. As the costs needed for the decorations depend on the number of applications present on screen, the resources of the decorator must be dimensioned with a sensible upper bound. Fortunately, however, as the decorator is a plain client of the window manager, it can be restarted, replaced, and upgraded without affecting any application.

Structured mode information for applications

Up to now, GUI clients were able to request mode information via a plain RPC call that returned the dimensions and color depth of the display. Multi-monitor setups call for more flexibility, which prompted us to replace the mode information by XML-structured information delivered as an info dataspace. This is in line with how meta information is handled in other modern session interfaces like the platform or USB sessions. The new representation gives us room to annotate information that could previously not be exposed to GUI clients, in particular:

• The total panorama dimensions.

• Captured areas within the panorama, which can be used by multi-monitor aware GUI clients as intelligence for placing GUI views.

• DPI information carried by width_mm and height_mm attributes. This information is defined by the display driver and passed to the GUI server as Capture::Connection::buffer argument.

• The closed state of a window interactively closed by the user.

Note that the window manager (wm) virtualizes the information of the nitpicker GUI server. Instead of exposing nitpicker's panorama to its clients, the wm reports the logical screen hosting the client's window as panorama and the window size as a single captured rectangle within the panorama.

Mouse grabbing

Since the inception of the nitpicker GUI server, its clients observed absolute pointer positions only. The GUI server unconditionally translated relative mouse-motion events to absolute motion events. To accommodate applications like games or a VM emulating a relative pointer device, we have now extended the GUI server(s) with the ability to selectively expose relative motion events while locking the absolute pointer position. This is usually called pointer grabbing. It goes without saying that the user must always retain a way to forcefully reassert control over the pointer without the cooperation of the application.

The solution is the enhancement of the Input::Session interface by a new RPC function that allows a client to request exclusive input. The nitpicker GUI server grants this request if the application owns the focus. In scenarios using the window manager (wm), the focus is always defined by the wm, which happens to intercept all input sessions of GUI applications. Hence, the wm is in the natural position of arbitrating the grabbing/ungrabbing of the pointer. For each GUI client, the wm records whether the client is interested in exclusive input but does not forward this request to nitpicker. Only if a GUI client receives the focus and has requested exclusive input, the wm enables exclusive input for this client at nitpicker when observing a mouse click on the application window. Whenever the user presses the global wm key (super), the wm forcefully releases the exclusive input at nitpicker until the user clicks into the client window the next time.

Furthermore, an application may enable exclusive input transiently during a key sequence, e.g., when dragging the mouse while holding the mouse button. Transient exclusive input is revoked as soon as the last button/key is released. It thereby would in principle allow for GUI controls like knobs to lock the pointer position while the user adjusts the value by moving the mouse while the mouse button is held. So the pointer retains its original position at the knob.

While operating in exclusive input mode, there is no useful notion of an absolute pointer position at the nitpicker GUI server. Hence, nitpicker hides GUI domains that use the pointer position as coordinate origin. Thereby, the mouse cursor automatically disappears while the pointer is grabbed.

Current state and ongoing work

All the advances described above are in full effect in the recently released version 24.10 of Sculpt OS. All components hosted in Genode's main and world repositories have been updated accordingly, including Genode-specific components like the widget toolkit used by the administrative user interface of Sculpt OS, window decorators, over Qt5 and Qt6, to SDL and SDL2.

Current work is underway to implement multi-monitor window management and to make multiple monitors seamlessly available to guest OSes hosted in VirtualBox. Furthermore, the Intel display driver is currently getting equipped with the ability to use vsync interrupts for driving the interaction with the GUI server, taking the final step to attain tearing-free motion.

Device drivers and platforms

Linux device-driver environment (DDE)

With our recent update of the DDE Linux kernel to version 6.6 for PC platforms and as a prerequisite to support the MNT Pocket Reform, we have adapted all drivers for the i.MX5/6/7/8 platforms to Linux kernel version 6.6.47. The list of drivers includes Wifi, NIC, display, GPU, USB and SD-card.

MNT Pocket Reform

Platform driver for i.MX 8M Plus

While enabling support for the MNT Pocket Reform (Section MNT Pocket Reform), it was necessary to adjust the i.MX8MP specific platform driver, which was originally introduced in the previous release 24.08 to drive the Compulab i.MX 8M Plus IOT Gateway.

Some of the I/O pin configurations necessary to set up the SoC properly are statically compiled into this driver because they do not change at runtime. However, the pin configuration is specific to the actual board. Therefore, the i.MX8MP platform driver now needs to distinguish between different boards (IOT Gateway and MNT Pocket) by evaluating the platform_info ROM provided by core.

Moreover, while working on different drivers, we detected a few missing clocks that were added to the platform driver. It turned out that some clocks that we initially turned off to save energy, have to be enabled to ensure the liveliness of the ARM Trusted Firmware (ATF) and thereby the platform. Also, we had to adapt the communication in between ATF and our platform driver to control power-domains. The first version of the i.MX8MP platform driver shared the ATF power-domains protocol with the i.MX8MQ version. However, the power-domain enumerations of the different firmwares varies also and we adapted that.

Finally, the watchdog hardware is now served by the platform driver in a recurrent way. Originally our driver used the watchdog only to implement reset functionality. But in case of the MNT Pocket Reform, the watchdog hardware is already armed by the bootloader. Therefore, it needs to get served in time, to prevent the system from rebooting. As a consequence, the platform driver is mandatory on this platform if it needs to run longer than a minute.

Wifi management rework

Our management interface in the wifi driver served us well over the years and concealed the underlying complexity of the wireless stack. At the same time it gained some complexity itself to satisfy a variety of use-cases. Thus, we took the past release cycle as opportunity to rework the management layer to reduce its complexity by streamlining the interaction between various parts, like the manager layer itself, wpa_supplicant as well as the device driver in order to provide a sound foundation for future adaptions. Included is also an update of the wpa_supplicant to version 2.11.

The following segments detail the changes made to the configuration options as they were altered quite a bit to no longer mix different tasks (e.g. joining a network and scanning for hidden networks) while removing obsolete options.

At the top-level <wifi_config> node, the following alterations were made:

• The log_level attribute was added and configures the supplicant's verbosity. Valid values correspond to levels used by the supplicant and are as follows: excessive, msgdump, debug, info, warning, and error. The default value is error and configures the least amount of verbosity. This option was introduced to ease the investigation of connectivity issues.

• The bgscan attribute may be used to configure the way the supplicant performs background-scanning to steer or rather optimize roaming decision within the same network. The default value is set to simple:30:-70:600. The attribute is forwarded unmodified to the WPA supplicant and thus provides the syntax supported by the supplicant implementation. It can be disabled by specifying an empty value, e.g. bgscan="".

• The connected_scan_interval attribute was removed as this functionality is now covered by background scanning.

• The verbose_state attribute was removed altogether and similar functionality is now covered by the verbose attribute.

The network management received the following changes:

• Every configured network, denoted by a <network> node, is now implicitly considered an option for joining. The auto_connect attribute was removed and a <network> node must be renamed or removed to deactivate automatic connection establishment.

• The intent to scan for a hidden network is now managed by the newly introduced <explicit_scan> node that like the <network> node has an ssid attribute. If the specified SSID is valid, it is incorporated into the scan request to actively probe for this network. As the node requests explicit scanning only, a corresponding <network> node is required to actually connect to the hidden network. The explicit_scan attribute of the <network> node has been removed.

The following exemplary configuration shows how to configure the driver for attempting to join two different networks where one of them is hidden. The initial scan interval is set 10 seconds and the signal quality will be updated every 30 seconds while connected to a network.

<wifi_config scan_interval="10" update_quality_interval="30">
  <explicit_scan ssid="Skynet"/>
  <network ssid="Zero"   protection="WPA2" passphrase="allyourbase"/>
  <network ssid="Skynet" protection="WPA3" passphrase="illbeback"/>
</wifi_config>

For more information please consult the driver's documentation that now features a best-practices section explaining how the driver should be operated at best, and highlights the difference between a managed (as used in Sculpt OS) and a user-generated configuration.

Audio driver updated to OpenBSD 7.6

With this release, we updated our OpenBSD-based audio driver to a more recent revision that correlates to version 7.6. It supports newer devices, e.g. Alder Lake-N, and includes a fix for using message-signaled interrupts (MSI) with HDA devices as found in AMD-based systems.

AVX and hardware-based AES in virtual machines

The current release adds support for requesting and transferring the AVX FPU state via Genode's VM-session interface. With this prerequisite fulfilled, we enabled the announcement of the AVX feature to guest VMs in our port of VirtualBox6.

Additionally, we enabled the announcement of AES and RDRAND CPU features to guest VMs to further improve the utilization of the hardware.

Build system and tools

Extended depot-tool safeguards

When using the run tool's --depot-auto-update feature while switching between different git topic branches with committed recipe hashes, a binary archive present in the depot may accidentally not match its ingredients because the depot/build tool's REBUILD= mode - as used by the depot auto-update mechanism - merely looks at the archive versions. This situation is arguably rare. But when it occurs, its reach and effects are hard to predict. To rule out this corner case early, the depot/build tool has now been extended by recording the hashes of the ingredients of binary archives. When skipping a rebuild because the desired version presumably already exists as a binary archive, the recorded hashes are compared to the current state of the ingredients (src and api archives). Thereby inconsistencies are promptly reported to the user.

Users of the depot tool will notice .hash files appearing alongside src and api archives. Those files contain the hash value of the content of the respective archive. Each binary archive built is now also accompanied by a .hash file, which contains a list of hash values of the ingredients that went into the binary archive. Thanks to these .hash files, the consistency between binaries and their ingredients can be checked quickly.

As a note of caution, when switching to the Genode 24.11 with existing depot, one will possibly need to remove existing depot archives (as listed by the diagnostic messages) because the existing archives are not accompanied by .hash files yet.