DMA Device Containment in a QNX Hypervisor System

Using a SMMU manager to protect against errant or malicious DMA access

The Problem with DMA Devices

Spatial isolation is fundamental to the safety and security of embedded software systems. Resources must be isolated, so that one agent in the system cannot interfere with the correct functioning of other agents, either through error or malicious intent. Memory is of course a key resource, and one whose integrity must be closely guarded.

When memory access is controlled by the CPU, the OS can ensure that an agent’s access to memory is limited to permitted regions and activities. For example, if a device driver attempts to read an address outside its permitted address range, the OS can catch the transgression and issue an error to the device driver.

Direct memory access (DMA) is typically used for devices that need to quickly transfer large amounts of data; for example, a GPU, network card or sound card. These bus-mastering DMA devices take control of the memory bus to gain direct access to system memory, bypassing the CPU.

Since no CPU is implicated in the memory transfer, an OS can’t manage a DMA device’s access to system memory. It can’t protect the system against incorrect (and possibly malicious) access to memory from DMA devices.

SMMUs and SMMUMAN

To manage DMA device access to memory, many modern boards have System Memory Management Units (IOMMU/SMMU)[i], which provide translation and access control for non-CPU initiated reads and writes, similar to the translation and access control that page tables provide for CPU-initiated reads and writes.

To be useful, these SMMUs must be programmed; they need a map of the memory regions each DMA device may access, and the permissions granted to the device for those regions. In systems built on the QNX Neutrino RTOS, including hypervisor systems, this programming is done with a SMMU manager, the smmuman service.

In a QNX Hypervisor system, the smmuman service service runs in the hypervisor host, and (optionally) in QNX guests (e.g., QNX Neutrino RTOS, QNX OS for Safety). In the hypervisor host, the service is used to program IOMMU/SMMUs with:

• The memory regions and permissions for DMA devices owned by the host; this mapping is exactly the same as in a non-virtualized system.
• The memory regions and permissions allocated to a guest in hypervisor virtual machines (VMs); this prevents DMA devices passed through to a guest from accessing memory that belongs to another guest or the host.

Note that the DMA device must be given access to all the memory assigned to the guest because it cannot be known beforehand where the device driver will set up its DMA regions.

In a QNX guest, the smmuman service:

• Divides up the DMA-accessible memory regions of the guest for finer boundary control. A DMA device that takes control of the memory bus can’t access just anywhere in guest memory, but only its assigned memory region. Transfers are bounded by the guest driver, for a second level of safety.
• Uses the smmu virtual device to pass on the memory regions and permissions it requires for each pass-through DMA device it controls to the smmuman service running in the host, which in turn programs the board IOMMU/SMMUs.

The figure below illustrates how the smmuman service is implemented in a QNX Hypervisor system to constrain a pass-through DMA device to its assigned memory region:

For simplicity, the diagram shows a single guest with a single pass-through DMA device:

1.      In the hypervisor host, the VM hosting the guest uses the smmuman service to program the IOMMU/SMMU with the memory range and permissions (A) for the entire physical memory regions that it will present to its guest (e.g., 0x20000000 to 0xB0000000).

This protects the hypervisor host and other guests from DMA devices passed through to the guest erroneously or maliciously accessing their memory. It doesn't prevent DMA devices passed through to the guest from improperly accessing memory assigned to that guest, however.

2.      In the guest, the driver for the pass-through DMA device uses the smmuman service’s client-side API to program the memory allocation and permissions (B) it needs into the smmuman service running in the guest.

3.      The guest’s smmuman service programs the smmu virtual device running in its hosting VM as it would an IOMMU/SMMU in hardware: it programs into the smmu virtual device the DMA device's memory allocation and permissions.

4.      The smmu virtual device uses the client-side API for the smmuman service running in the hypervisor host to program into the board’s IOMMU/SMMU the memory allocation and permissions (B) requested by the guest’s smmuman service.

5.      The host’s smmuman service programs the pass-through DMA device's memory allocation and permissions (B) into the board’s IOMMU/SMMU.

The DMA device’s access to memory (C) is now limited to the region and permissions requested by the DMA device driver in the guest (B), and the guest OS and other components are protected from this device erroneously or maliciously accessing their memory.

Linux guests don't support the smmuman service; however, you can use the smmuman service in the hypervisor host to program the IOMMU/SMMU with the memory range and permissions for the entire physical memory regions that the hypervisor host will present to each Linux guest. This will limit to scope of any erroneous or malicious DMA memory access to just the memory allocated to the Linux guest.

For more information, see QNX Hypervisor and QNX Hypervisor for Safety, the Ultimate Guide to Embedded System Security, and the SMMUMAN User’s Guide.

QNX Neutrino RTOS – proven reliability, performance, security

Thousands of companies deploy and trust QNX real-time technology to ensure the best combination of performance, security and reliability in the world’s most safety-critical and mission-critical systems. At the core of this offering is the QNX® Neutrino® RTOS, a full-featured and robust microkernel RTOS designed to enable the next-generation of products for aerospace & defense, automotive, commercial vehicles, heavy machinery, industrial controls, medical devices, rail and robotics and automation. With the QNX RTOS, embedded systems designers can create compelling, safe, and secure devices built on a highly reliable RTOS serving as the foundation that helps guard against system malfunctions, malware and security breaches.

Learn more in the blog post How to Choose the Right OS for Your Embedded System: 10 Criteria for Success or visit the BlackBerry QNX website to explore the Ultimate Guide to RTOS. 

*On ARM platforms, IOMMU/SMMU components are usually called “System Memory Management Units” (SMMUs); on Intel x86 platforms, this technology is usually called “Virtualization Technology for Directed I/O” (VT-d).