DMA Scatter-Gather Transfer¶
Scatter-gather DMA allows a DMA transaction to operate on memory represented by multiple regions without first copying the data into one physically contiguous temporary buffer.
In Linux, scatter-gather DMA involves several distinct representations:
CPU Memory Representation
↓
Original Scatterlist
↓
DMA Mapping
↓
DMA-Visible Segments
↓
DMA Engine
↓
DMA Controller Provider
↓
Hardware Descriptors / LLIs
These layers must not be treated as equivalent.
In particular:
1. Why Scatter-Gather DMA Exists¶
A DMA client does not always have one contiguous memory region that can be represented by a single DMA address and length.
Memory may instead consist of:
- Multiple independent buffers
- Multiple pages
- A virtually contiguous buffer backed by non-contiguous pages
- Buffers divided according to DMA maximum segment size
- Buffers divided according to controller-specific transfer limits
Scatter-gather provides a standard representation for these cases.
Conceptually:
Scatter-gather should therefore not be understood only as "many buffers chained together."
It is a general representation for segmented memory used by DMA and other kernel subsystems.
2. struct scatterlist¶
Linux represents each scatter-gather entry with:
An entry describes a region of memory using information such as:
At this stage, the entry describes the CPU/kernel-side memory representation.
It does not yet imply a valid DMA address for a particular device.
3. Building a Scatterlist¶
For a fixed number of entries:
Individual buffers can then be associated with entries:
Conceptually:
The initial count:
is the number of original SG entries.
4. How Is the Original SG Size Determined?¶
The SG array size is not normally chosen by asking how fragmented physical RAM happens to be.
Instead, the kernel component constructing the SG list already has some memory representation to work from.
Examples include:
Known buffers¶
Known pages¶
A subsystem may already own an array or collection of struct page objects.
It can construct SG entries from those pages using APIs such as:
Virtual buffers¶
A subsystem may divide a virtual buffer according to:
- Page boundaries
- Maximum DMA segment size
- Controller DMA length limits
- Memory type
Therefore the relevant question is:
rather than:
DMA address-space details are handled later by the DMA mapping layer.
5. DMA Mapping¶
An original scatterlist cannot simply be handed to hardware.
It first needs to be mapped for the DMA device.
For TX:
For RX:
Unlike dma_map_single(), which returns a DMA address, dma_map_sg() returns a count.
The return value is the number of DMA-visible segments produced by the mapping operation.
A return value of zero indicates failure.
6. Original SG Count vs Mapped Segment Count¶
The number returned by dma_map_sg() may be smaller than the number passed into it.
For example:
Original Scatterlist
SG0
SG1
SG2
SG3
nents = 4
│
│ dma_map_sg()
▼
DMA-Visible Segments
DMA Segment 0
DMA Segment 1
DMA Segment 2
mapped_nents = 3
The DMA mapping layer may merge entries when adjacent memory can be represented as a suitable DMA-visible segment.
Therefore:
for a successful mapping.
The two counts must remain conceptually separate:
7. DMA-Side SG Information¶
After mapping, DMA clients and providers must use the DMA-visible information associated with each mapped SG segment.
The important helpers are:
Conceptually:
Before DMA mapping
──────────────────
page
offset
length
│
│ dma_map_sg()
▼
After DMA mapping
─────────────────
DMA address
DMA length
A DMA provider must not reconstruct DMA addresses from CPU virtual addresses or assume that CPU physical addresses are directly usable by the DMA device.
8. Count Rules¶
Different APIs use different SG counts.
| API | Count |
|---|---|
dma_map_sg() |
Original nents |
dmaengine_prep_slave_sg() |
Mapped mapped_nents |
dma_sync_sg_for_cpu() |
Original nents |
dma_sync_sg_for_device() |
Original nents |
dma_unmap_sg() |
Original nents |
The lifecycle can be summarized as:
Original Scatterlist
│
│ nents = N
▼
dma_map_sg(..., N)
│
│ returns M
▼
DMA-Visible Segments
│
│ mapped_nents = M
▼
dmaengine_prep_slave_sg(..., M)
When returning to the DMA mapping API:
This distinction is essential when writing SG DMA clients.
9. struct sg_table¶
Subsystems frequently manage scatterlists using:
Conceptually, the important fields are:
A typical lifecycle is:
Internally, dma_map_sgtable() maps using:
and stores the mapped count in:
Cleanup uses the original count again:
This allows the structure to preserve both representations at the same time.
10. DMA Mapping Direction¶
Scatter-gather transfers use the same DMA mapping direction rules as single-buffer streaming mappings.
TX¶
Use:
RX¶
Use:
These values belong to:
They describe how the device accesses memory.
11. DMA Engine Direction¶
DMA Engine uses a different direction abstraction.
For slave DMA:
TX¶
Use:
RX¶
Use:
These values belong to:
Therefore:
| Transfer | DMA Mapping | DMA Engine |
|---|---|---|
| TX | DMA_TO_DEVICE |
DMA_MEM_TO_DEV |
| RX | DMA_FROM_DEVICE |
DMA_DEV_TO_MEM |
The two enums describe different abstraction layers.
12. Preparing a Slave SG Transaction¶
After mapping, the SG transaction can be prepared with:
desc = dmaengine_prep_slave_sg(chan,
sg,
mapped_nents,
DMA_MEM_TO_DEV,
DMA_PREP_INTERRUPT |
DMA_CTRL_ACK);
if (!desc) {
dma_unmap_sg(dev, sg, nents, DMA_TO_DEVICE);
return -EIO;
}
The important point is:
not the original entry count.
The DMA Engine provider receives the DMA-visible SG representation.
13. Scatter-Gather Transfer Lifecycle¶
The three important counts belong to different layers:
Original SG entries
N
│
│ DMA mapping
▼
DMA-visible segments
M
│
│ provider conversion
▼
Hardware descriptors / LLIs
H
The counts should not be assumed to be equal.
14. SG Segment vs Hardware Descriptor¶
A DMA-visible SG segment is still a software-level DMA representation.
The DMA controller provider must convert it into whatever hardware representation the controller requires.
Therefore:
A provider may split one mapped segment into multiple hardware descriptors because of:
- Maximum block transfer size
- Address alignment
- Transfer width
- Burst restrictions
- Controller-specific limits
For example:
Thus:
is entirely valid.
15. Synopsys AXI DMA Provider Example¶
The Synopsys AXI DMA provider demonstrates this conversion directly.
Its slave SG preparation path is:
dw_axi_dma_chan_prep_slave_sg()
│
├── iterate mapped SG
│
├── sg_dma_address()
│
├── sg_dma_len()
│
├── calculate_block_len()
│
├── determine required descriptor count
│
├── split segments if necessary
│
└── dw_axi_dma_set_hw_desc()
The provider first determines how many hardware descriptors are required.
Conceptually:
The count named num_sgs in this provider is therefore closer to the required hardware descriptor count than the original client-side SG entry count.
16. Hardware Block Length¶
The provider uses calculate_block_len() to determine how many bytes can fit in one hardware DMA block.
The basic relationship is:
The transfer width depends on factors such as:
- DMA controller data width
- DMA address alignment
- Buffer length alignment
- Configured peripheral width
- Transfer direction
For memory-to-device transfers, the memory-side width is derived from the memory address and length alignment and is limited by the provider implementation.
For device-to-memory transfers, block sizing uses the configured peripheral source width.
17. Building Hardware LLIs¶
The provider eventually calls:
to populate controller-specific LLI fields.
For memory-to-device:
For device-to-memory:
The helper prepares fields corresponding to:
- Source address
- Destination address
- Transfer width
- Increment behavior
- Block transfer count
- Burst length
- LLI control
- Linked-list pointer
This is the provider-specific translation from the DMA Engine transaction model to the hardware programming model.
18. SPI Core as the Mapping Owner¶
A useful real-world example is the Linux SPI subsystem.
The SPI Core can build and map the scatter-gather table before invoking the controller driver's DMA path.
The conceptual ownership is:
SPI Core
────────────────────────────
Buffer
↓
Build SG table
↓
dma_map_sgtable()
↓
Mapped tx_sg / rx_sg
SPI Controller Driver
────────────────────────────
tx_sg / rx_sg
↓
dmaengine_prep_slave_sg()
↓
submit
↓
issue
Therefore, a controller driver may use DMA-mapped SG entries without calling dma_map_sg() itself.
The correct question is:
not:
19. SPI Core SG Construction¶
The SPI Core helper spi_map_buf_attrs() demonstrates how a subsystem can determine the original SG representation.
For a normal kernel virtual buffer, the buffer can be divided according to:
For a vmalloc() buffer, the SPI Core instead walks the backing pages.
Conceptually:
vmalloc() buffer
│
▼
vmalloc_to_page()
│
▼
sg_set_page()
│
▼
Original SG table
│
▼
dma_map_sgtable()
This demonstrates that:
does not imply:
The subsystem must construct an appropriate page-based representation and map it for the DMA device.
Actual usability still depends on the subsystem, DMA device, and hardware constraints.
20. BCM2835 SPI SG Path¶
The BCM2835 SPI controller driver consumes the SG tables prepared by the SPI Core.
For TX:
For RX:
The controller driver then prepares the DMA Engine transaction:
Because the SPI Core has already performed DMA mapping:
represent the mapped DMA segment count used by the DMA Engine transaction.
21. BCM2835 Execution Ordering¶
The BCM2835 DMA path intentionally starts TX before finishing the RX-side preparation.
Conceptually:
Prepare TX
↓
Submit TX
↓
Program SPI hardware
↓
Enable DMA requests
↓
Issue TX
│
│ hardware begins useful work
│
└──────────────┐
│
Prepare RX
↓
Submit RX
↓
Issue RX
This overlaps software preparation with hardware execution.
It also demonstrates why error cleanup must distinguish whether a DMA transaction has already been issued.
22. Pre-Issue Failure¶
Suppose mapping succeeds but transaction preparation fails:
The DMA transaction has not been issued.
Therefore no active DMA transaction needs to be terminated.
The same principle applies to a submission failure before dma_async_issue_pending().
23. Post-Issue Failure and Timeout¶
After:
the DMA controller may already be accessing the SG buffers.
Therefore timeout or cancellation cleanup must first resolve DMA activity:
DMA issued
↓
timeout / cancellation
↓
dmaengine_terminate_sync()
↓
DMA and completion activity resolved
↓
dma_unmap_sg()
The central safety question remains:
Can DMA hardware or asynchronous completion activity still access this resource?
24. One-Shot SG TX Example¶
A simplified blocking TX sequence is:
mapped_nents = dma_map_sg(dev, sg, nents, DMA_TO_DEVICE);
if (!mapped_nents)
return -EIO;
desc = dmaengine_prep_slave_sg(chan,
sg,
mapped_nents,
DMA_MEM_TO_DEV,
DMA_PREP_INTERRUPT |
DMA_CTRL_ACK);
if (!desc) {
ret = -EIO;
goto err_unmap;
}
desc->callback = dma_tx_done;
desc->callback_param = &done;
cookie = dmaengine_submit(desc);
ret = dma_submit_error(cookie);
if (ret)
goto err_unmap;
dma_async_issue_pending(chan);
timeout = wait_for_completion_timeout(&done,
msecs_to_jiffies(5000));
if (!timeout) {
ret = -ETIMEDOUT;
goto err_terminate;
}
dma_unmap_sg(dev, sg, nents, DMA_TO_DEVICE);
return 0;
err_terminate:
dmaengine_terminate_sync(chan);
err_unmap:
dma_unmap_sg(dev, sg, nents, DMA_TO_DEVICE);
return ret;
The important count transition is:
dma_map_sg(..., nents)
↓
mapped_nents
↓
dmaengine_prep_slave_sg(..., mapped_nents)
cleanup
↓
dma_unmap_sg(..., nents)
25. One-Shot SG RX¶
For RX:
dma_map_sg(..., DMA_FROM_DEVICE)
↓
dmaengine_prep_slave_sg(..., DMA_DEV_TO_MEM)
↓
submit
↓
issue
↓
DMA completion
↓
dma_unmap_sg(..., DMA_FROM_DEVICE)
↓
CPU accesses RX data
When the one-shot streaming mapping is immediately unmapped, an additional:
is not required before CPU access.
The unmap operation ends the streaming mapping lifecycle and returns ownership to the CPU side.
26. Persistent SG Mapping¶
Sometimes the DMA mapping remains active across multiple transfers.
In that case the mapping is not immediately destroyed after every DMA completion.
Instead:
Device ownership
↓
DMA completion / safe handoff point
↓
dma_sync_sg_for_cpu()
↓
CPU ownership
↓
CPU accesses data
↓
dma_sync_sg_for_device()
↓
Device ownership
↓
next DMA activity
The sync APIs use the original SG entry count.
27. DMA Mapping Synchronization vs DMA Execution¶
A critical distinction is:
The DMA mapping API manages CPU/device ownership and architecture-specific cache synchronization.
It does not establish that the DMA hardware has stopped accessing a particular region.
The driver must first know from the DMA execution lifecycle that the region is safe to hand to the CPU.
For example:
DMA currently using P2
P0 completed
↓
driver knows P0 is currently safe
↓
dma_sync_sg_for_cpu()
↓
CPU reads P0
Calling dma_sync_sg_for_cpu() alone does not guarantee that hardware will not access P0.
28. Cyclic DMA and Producer/Consumer Ownership¶
Persistent SG mappings are especially important when reasoning about cyclic or ring-buffer DMA.
Consider:
DMA may act as a producer while the CPU acts as a consumer.
If DMA has moved from P0 to P1, the CPU may process P0 after the appropriate ownership synchronization.
However:
means DMA has caught up with the CPU.
This is a ring-buffer overrun problem.
dma_sync_sg_for_cpu() does not prevent the DMA controller from wrapping around and reusing a period.
29. Three Synchronization Layers¶
Persistent and cyclic DMA often involve three independent synchronization concerns.
DMA Engine Execution Synchronization¶
Examples:
This answers questions such as:
Has the DMA engine finished or stopped accessing this transaction or period?
DMA Mapping Ownership Synchronization¶
Examples:
This answers:
Is the mapped memory in the correct ownership/cache state for CPU or device access?
Producer/Consumer Synchronization¶
Examples:
This answers:
Has the producer caught up with the consumer?
These layers solve different problems and should not be treated as interchangeable.
30. Resource Ownership Model¶
A useful ownership model is:
Buffer allocation ownership
≠
SG allocation ownership
≠
DMA mapping ownership
≠
DMA transaction ownership
For example, the SPI subsystem may have:
SPI Core
owns SG construction
owns DMA mapping
owns DMA unmapping
SPI controller driver
owns DMA Engine transaction preparation
controls peripheral execution
DMA provider
owns hardware descriptor construction
programs DMA controller representation
Cleanup responsibilities should follow the corresponding ownership boundaries.
31. Common Mistakes¶
Mistake 1 — Passing Original nents to dmaengine_prep_slave_sg()¶
Wrong:
after dma_map_sg() returned a different mapped count.
Correct:
Mistake 2 — Unmapping with mapped_nents¶
Wrong:
Correct:
The unmap API receives the original count passed to dma_map_sg().
Mistake 3 — Reading RX Memory Before Returning Ownership¶
For a one-shot RX mapping:
Wrong:
Correct:
For a persistent mapping, use the appropriate sync operation instead of unmapping.
Mistake 4 — Treating dma_sync_sg_for_cpu() as a DMA Stop Operation¶
Wrong mental model:
Correct mental model:
DMA lifecycle says region is safe
↓
dma_sync_sg_for_cpu()
↓
CPU-side ownership/cache synchronization
Mistake 5 — Assuming One SG Entry Means One Hardware LLI¶
Wrong:
Correct:
SG representation
↓
DMA mapping
↓
mapped DMA segment
↓
provider hardware constraints
↓
one or more hardware descriptors
32. Mental Model¶
The complete scatter-gather slave DMA model is:
CPU Buffer / Pages
│
▼
Original Scatterlist
│
│ original count N
▼
DMA Mapping
│
│ mapped count M
▼
DMA-Visible Segments
│
▼
dmaengine_prep_slave_sg()
│
▼
DMA Controller Provider
│
│ apply hardware constraints
▼
Hardware Descriptor / LLI Chain
│
▼
DMA Hardware Execution
│
▼
Completion / Synchronization
│
▼
DMA Unmap or Ownership Sync
│
▼
CPU Access
The most important rule is:
Keep the CPU memory representation, DMA-visible representation, and hardware descriptor representation as three separate layers.
Once those layers are separated, SG counts, mapping lifetime, provider splitting, and cleanup responsibilities become much easier to reason about.
Summary¶
Scatter-gather DMA extends the slave DMA transaction model from one DMA address range to a mapped collection of memory segments.
The key relationships are:
Original SG entries
│
│ dma_map_sg()
▼
DMA-visible segments
│
│ dmaengine_prep_slave_sg()
▼
DMA provider
│
│ hardware-specific splitting
▼
Hardware descriptors / LLIs
The original SG count is used by the DMA mapping lifecycle, while the mapped count is passed to the DMA Engine provider.
The provider then converts those DMA-visible segments into the controller-specific descriptor format required by the hardware.
Understanding these boundaries is essential for writing correct DMA client drivers, especially when handling non-contiguous memory, subsystem-owned mappings, persistent mappings, cyclic buffers, timeout cleanup, and controller-specific transfer limits.