Linux DMA Mapping¶

Overview¶

Direct Memory Access (DMA) allows a hardware device to transfer data between the device and system memory without requiring the CPU to copy every byte.

The actual transfer is still performed by a hardware DMA engine, DMA controller, or DMA-capable device.

The Linux DMA API does not create DMA capability for hardware that does not support DMA. Instead, it provides a portable interface that allows a driver to prepare CPU memory for access by a DMA-capable device.

CPU Buffer
        │
        ▼
Linux DMA Mapping API
        │
        ▼
DMA Address
        │
        ▼
DMA-Capable Device

The DMA Mapping API is primarily a memory mapping and cache-coherency interface. It does not normally start the hardware DMA transfer.

The driver still programs the device registers or descriptors that start the transfer.

Hardware DMA and the Linux DMA API¶

A DMA transfer requires hardware support.

Examples include:

A peripheral connected to a system DMA controller
An Ethernet controller with an integrated DMA engine
A USB controller with DMA capability
A storage controller that can directly access system memory

If a device has no DMA capability and no hardware path to a DMA controller, the Linux DMA API cannot make the device perform DMA.

The driver must instead transfer data through CPU-controlled register access.

Hardware Without DMA

CPU
 │
 ▼
readl() / writel()
 │
 ▼
Device Register

For DMA-capable hardware, Linux separates two responsibilities:

Linux DMA Mapping API
        │
        └── Prepares memory and returns a DMA address

Device Driver
        │
        └── Programs registers or descriptors and starts DMA

For example:

dma_addr = dma_map_single(dev,
buffer,
size,
DMA_TO_DEVICE);

if (dma_mapping_error(dev, dma_addr))
return -EIO;

writel(lower_32_bits(dma_addr), DMA_ADDR_REG);
writel(size, DMA_SIZE_REG);
writel(DMA_START, DMA_CONTROL_REG);

dma_map_single() prepares the memory mapping. The final register write starts the hardware transfer.

DMA Address Spaces¶

The CPU and DMA device may not use the same address for the same memory.

Three address types are important:

Address	Used By	Purpose
CPU virtual address	CPU and kernel code	Address used by normal C pointers
Physical address	Physical memory subsystem	Location of the backing RAM
DMA address	DMA-capable device	Address presented to the device

Conceptually:

CPU Virtual Address
        │
        │ MMU
        ▼
Physical Address
        │
        │ DMA Mapping / IOMMU
        ▼
DMA Address

A driver must not assume that:

DMA Address == Physical Address

This may appear true on simple systems without an IOMMU or bus translation, but it is not portable.

The driver must use the DMA address returned by the DMA Mapping API.

CPU Address and DMA Address¶

A DMA buffer commonly has two addresses:

CPU View
cpu_addr = 0xffff888012340000

                │
                ▼

        +----------------+
        |   DMA Buffer   |
        +----------------+

                ▲
                │

Device View
dma_addr = 0x0000000080000000

The CPU uses the virtual address:

memcpy(cpu_addr, source, size);

The device uses the DMA address:

writel(lower_32_bits(dma_addr), DMA_ADDR_REG);

The DMA address must not be dereferenced as a normal CPU pointer.

Coherent DMA and Streaming DMA¶

Linux DMA mappings are commonly divided into two models:

DMA Mapping
    │
    ├── Coherent DMA
    │
    └── Streaming DMA

The main differences are buffer lifetime, ownership, and cache synchronization.

Characteristic	Coherent DMA	Streaming DMA
Typical lifetime	Long-lived	Per transfer
Allocation	`dma_alloc_coherent()`	General memory plus `dma_map_single()`
CPU address	Returned by allocation API	Existing CPU buffer
DMA address	Returned by allocation or mapping API	Returned by mapping API
Ownership	CPU and device share the buffer	Ownership transfers between CPU and device
Cache synchronization	Usually implicit	Managed through DMA mapping and sync APIs
Typical use	Descriptor rings, command queues	Packets, frames, transfer buffers

Coherent DMA¶

Coherent DMA provides a buffer for which the CPU and device observe a consistent view of memory.

The term coherent means that different observers see consistent data.

CPU
 │
 ▼
Shared DMA Buffer
 ▲
 │
Device

A coherent DMA allocation returns:

A CPU virtual address
A DMA address

Example:

void *cpu_addr;
dma_addr_t dma_addr;

cpu_addr = dma_alloc_coherent(dev,
size,
&dma_addr,
GFP_KERNEL);
if (!cpu_addr)
return -ENOMEM;

The CPU accesses the buffer through cpu_addr.

The device accesses the same buffer through dma_addr.

Coherent DMA is commonly used for long-lived control structures such as:

DMA descriptor rings
Command queues
Completion queues
Shared control blocks
Hardware-owned metadata

The allocation is released with:

dma_free_coherent(dev,
size,
cpu_addr,
dma_addr);

Streaming DMA¶

Streaming DMA maps an existing CPU buffer for a particular transfer.

Typical flow:

CPU prepares buffer
        │
        ▼
dma_map_single()
        │
        ▼
Device owns mapping
        │
        ▼
DMA transfer
        │
        ▼
dma_unmap_single()
        │
        ▼
CPU owns buffer again

Example:

dma_addr_t dma_addr;

dma_addr = dma_map_single(dev,
buffer,
size,
DMA_TO_DEVICE);

if (dma_mapping_error(dev, dma_addr))
return -EIO;

/* Program and start the DMA-capable device. */

/* Wait for or handle DMA completion. */

dma_unmap_single(dev,
dma_addr,
size,
DMA_TO_DEVICE);

Streaming DMA is commonly used for:

Network packets
USB transfer buffers
SPI transfers
Storage blocks
Camera frames
Audio buffers

DMA Direction¶

DMA direction is always described from the DMA device's point of view.

Important

DMA_TO_DEVICE means the device reads data prepared by the CPU.

DMA_FROM_DEVICE means the device writes data that will later be read by the CPU.

Direction	CPU Write	CPU Read	Device Write	Device Read	Typical Use
`DMA_TO_DEVICE`	Yes	Yes	No	Yes	UART TX, SPI TX, Ethernet TX
`DMA_FROM_DEVICE`	No	Yes, after synchronization	Yes	No	UART RX, camera capture, Ethernet RX
`DMA_BIDIRECTIONAL`	Yes	Yes	Yes	Yes	Shared bidirectional buffers

DMA_TO_DEVICE¶

CPU
 │
 │ Writes data
 ▼
CPU Buffer
 │
 │ DMA_TO_DEVICE
 ▼
Device reads data

Examples:

Transmit an Ethernet packet
Send data through SPI
Send data through UART
Submit a command block to hardware

DMA_FROM_DEVICE¶

Device
 │
 │ Writes data
 ▼
CPU Buffer
 │
 │ DMA_FROM_DEVICE
 ▼
CPU reads data

Examples:

Receive an Ethernet packet
Capture a camera frame
Receive SPI data
Store ADC samples
Receive UART data

DMA_BIDIRECTIONAL¶

CPU  <---------->  Device

Both the CPU and device may modify the buffer.

This direction may require more cache maintenance and should not be selected merely to avoid deciding the correct direction.

Ownership Transfer¶

Streaming DMA uses an ownership model.

Before mapping:

Owner = CPU

After dma_map_single():

Owner = Device

After dma_unmap_single():

Owner = CPU

The complete transition is:

CPU owns buffer
        │
        ▼
dma_map_single()
        │
        ▼
Device owns buffer
        │
        ▼
DMA transfer
        │
        ▼
dma_unmap_single()
        │
        ▼
CPU owns buffer

While the device owns a streaming DMA buffer, the CPU should not access it unless the DMA API explicitly transfers access back to the CPU.

Incorrect simultaneous access may cause:

Partial updates
Stale data
Cache corruption
Device-visible data mismatches
Platform-dependent failures

Cache Coherency¶

DMA devices access memory independently of the CPU.

On a non-coherent platform, the CPU may modify cached data without immediately updating main memory.

CPU Cache
    New Data

Main Memory
    Old Data

DMA Device
    Reads Old Data

The reverse problem also exists.

A device may write new data into memory while the CPU still has stale data in its cache.

DMA Device
    Writes New Data

Main Memory
    New Data

CPU Cache
    Old Data

The DMA Mapping API performs the platform-specific cache maintenance required by the mapping operation.

For long-lived streaming mappings that switch between CPU and device access, Linux also provides explicit synchronization APIs.

Synchronization for Device Access¶

dma_sync_single_for_device() makes CPU-written data visible to the DMA device.

Conceptually:

CPU modifies buffer
        │
        ▼
CPU Cache = New
Memory    = Old
        │
        ▼
dma_sync_single_for_device()
        │
        ▼
CPU Cache = New
Memory    = New
        │
        ▼
Device reads latest data

This is primarily associated with:

DMA_TO_DEVICE
DMA_BIDIRECTIONAL

Synchronization for CPU Access¶

dma_sync_single_for_cpu() makes device-written data visible to the CPU.

Conceptually:

Device writes buffer
        │
        ▼
Memory    = New
CPU Cache = Old
        │
        ▼
dma_sync_single_for_cpu()
        │
        ▼
Memory    = New
CPU Cache = New
        │
        ▼
CPU reads latest data

This is primarily associated with:

DMA_FROM_DEVICE
DMA_BIDIRECTIONAL

Memory Barriers and DMA Synchronization¶

Memory barriers and DMA synchronization solve different problems.

Memory Barrier¶

A memory barrier controls ordering.

Write DMA descriptor
        │
        ▼
wmb()
        │
        ▼
Notify device

It guarantees that memory operations become visible in the required order.

DMA Synchronization¶

DMA synchronization controls CPU and device data visibility.

CPU modifies buffer
        │
        ▼
dma_sync_single_for_device()
        │
        ▼
Device reads updated memory

A memory barrier does not replace cache synchronization.

Cache synchronization does not replace ordering barriers required by a device protocol.

Both may be required in the same driver.

`ioremap()` and DMA Mapping¶

ioremap() and dma_map_single() operate in opposite directions.

`ioremap()`¶

Makes device registers accessible to the CPU.

Device Register
        │
        ▼
CPU Virtual Address

`dma_map_single()`¶

Makes CPU memory accessible to a DMA-capable device.

CPU Memory
        │
        ▼
DMA Address

This distinction is important:

API	Resource Being Mapped	Consumer
`ioremap()`	Device MMIO region	CPU
`dma_map_single()`	CPU memory buffer	DMA-capable device

Common DMA Mapping APIs¶

API	Purpose
`dma_alloc_coherent()`	Allocate a long-lived coherent DMA buffer
`dma_free_coherent()`	Release a coherent DMA buffer
`dma_map_single()`	Map an existing CPU buffer for streaming DMA
`dma_unmap_single()`	End a streaming DMA mapping
`dma_sync_single_for_device()`	Synchronize a mapped buffer for device access
`dma_sync_single_for_cpu()`	Synchronize a mapped buffer for CPU access
`dma_mapping_error()`	Check whether a DMA mapping failed

Simulator Design¶

The Day98 simulator focuses on the DMA memory model rather than a real DMA controller.

It models:

CPU Buffer
        │
        ▼
CPU Cache View
        │
        ▼
DMA Mapping
        │
        ▼
Device-Visible Memory

The simulator tracks:

CPU virtual address
Symbolic DMA address
Mapping type
DMA direction
Buffer owner
CPU cache validity
CPU cache dirty state
Device-visible memory

The simulator does not model:

A real DMA controller
PCIe transfers
IOMMU page tables
Scatter-gather lists
Hardware descriptor formats
DMA Engine framework internals

Summary¶

The Linux DMA API does not replace hardware DMA support.

It provides a portable way for drivers to expose CPU memory to DMA-capable hardware while handling:

CPU and device address-space differences
DMA address translation
Mapping lifetime
Buffer ownership
DMA direction
Cache coherency
Platform-specific synchronization

The most important distinction is:

DMA Mapping API
    Prepares memory for the device

Device Driver
    Programs and starts the hardware transfer

Day98 - Linux DMA API

Linux DMA Mapping¶

Overview¶

Hardware DMA and the Linux DMA API¶

DMA Address Spaces¶

CPU Address and DMA Address¶

Coherent DMA and Streaming DMA¶

Coherent DMA¶

Streaming DMA¶

DMA Direction¶

DMA_TO_DEVICE¶

DMA_FROM_DEVICE¶

DMA_BIDIRECTIONAL¶

Ownership Transfer¶

Cache Coherency¶

Synchronization for Device Access¶

Synchronization for CPU Access¶

Memory Barriers and DMA Synchronization¶

Memory Barrier¶

DMA Synchronization¶

ioremap() and DMA Mapping¶

ioremap()¶

dma_map_single()¶

Common DMA Mapping APIs¶

Simulator Design¶

Summary¶

Related Topics¶

Related API Reference¶

Related Labs¶

`ioremap()` and DMA Mapping¶

`ioremap()`¶

`dma_map_single()`¶