Skip to content

DMA Scatter-Gather Transfer

Scatter-gather DMA allows a DMA transaction to operate on memory represented by multiple regions without first copying the data into one physically contiguous temporary buffer.

In Linux, scatter-gather DMA involves several distinct representations:

CPU Memory Representation
Original Scatterlist
DMA Mapping
DMA-Visible Segments
DMA Engine
DMA Controller Provider
Hardware Descriptors / LLIs

These layers must not be treated as equivalent.

In particular:

Original SG entry
DMA-mapped segment
Hardware DMA descriptor / LLI

1. Why Scatter-Gather DMA Exists

A DMA client does not always have one contiguous memory region that can be represented by a single DMA address and length.

Memory may instead consist of:

  • Multiple independent buffers
  • Multiple pages
  • A virtually contiguous buffer backed by non-contiguous pages
  • Buffers divided according to DMA maximum segment size
  • Buffers divided according to controller-specific transfer limits

Scatter-gather provides a standard representation for these cases.

Conceptually:

Buffer A
Buffer B
Buffer C
Scatterlist
DMA Mapping
DMA-visible memory segments

Scatter-gather should therefore not be understood only as "many buffers chained together."

It is a general representation for segmented memory used by DMA and other kernel subsystems.


2. struct scatterlist

Linux represents each scatter-gather entry with:

struct scatterlist

An entry describes a region of memory using information such as:

page
offset
length

At this stage, the entry describes the CPU/kernel-side memory representation.

It does not yet imply a valid DMA address for a particular device.


3. Building a Scatterlist

For a fixed number of entries:

struct scatterlist sg[3];
unsigned int nents = ARRAY_SIZE(sg);

sg_init_table(sg, nents);

Individual buffers can then be associated with entries:

sg_set_buf(&sg[0], buf0, len0);
sg_set_buf(&sg[1], buf1, len1);
sg_set_buf(&sg[2], buf2, len2);

Conceptually:

buf0 + len0 ──► SG[0]
buf1 + len1 ──► SG[1]
buf2 + len2 ──► SG[2]

The initial count:

nents = 3

is the number of original SG entries.


4. How Is the Original SG Size Determined?

The SG array size is not normally chosen by asking how fragmented physical RAM happens to be.

Instead, the kernel component constructing the SG list already has some memory representation to work from.

Examples include:

Known buffers

Buffer A
Buffer B
Buffer C


3 SG entries

Known pages

A subsystem may already own an array or collection of struct page objects.

It can construct SG entries from those pages using APIs such as:

sg_set_page()

Virtual buffers

A subsystem may divide a virtual buffer according to:

  • Page boundaries
  • Maximum DMA segment size
  • Controller DMA length limits
  • Memory type

Therefore the relevant question is:

What memory representation does this subsystem currently own?

rather than:

How fragmented is physical RAM?

DMA address-space details are handled later by the DMA mapping layer.


5. DMA Mapping

An original scatterlist cannot simply be handed to hardware.

It first needs to be mapped for the DMA device.

For TX:

mapped_nents = dma_map_sg(dev,
                          sg,
                          nents,
                          DMA_TO_DEVICE);
if (!mapped_nents)
    return -EIO;

For RX:

mapped_nents = dma_map_sg(dev,
                          sg,
                          nents,
                          DMA_FROM_DEVICE);
if (!mapped_nents)
    return -EIO;

Unlike dma_map_single(), which returns a DMA address, dma_map_sg() returns a count.

The return value is the number of DMA-visible segments produced by the mapping operation.

A return value of zero indicates failure.


6. Original SG Count vs Mapped Segment Count

The number returned by dma_map_sg() may be smaller than the number passed into it.

For example:

Original Scatterlist

SG0
SG1
SG2
SG3

nents = 4

        │ dma_map_sg()

DMA-Visible Segments

DMA Segment 0
DMA Segment 1
DMA Segment 2

mapped_nents = 3

The DMA mapping layer may merge entries when adjacent memory can be represented as a suitable DMA-visible segment.

Therefore:

0 < mapped_nents <= nents

for a successful mapping.

The two counts must remain conceptually separate:

nents
    =
original memory representation count

mapped_nents
    =
DMA-visible segment count

7. DMA-Side SG Information

After mapping, DMA clients and providers must use the DMA-visible information associated with each mapped SG segment.

The important helpers are:

sg_dma_address(sg);
sg_dma_len(sg);

Conceptually:

Before DMA mapping
──────────────────

page
offset
length

        │ dma_map_sg()

After DMA mapping
─────────────────

DMA address
DMA length

A DMA provider must not reconstruct DMA addresses from CPU virtual addresses or assume that CPU physical addresses are directly usable by the DMA device.


8. Count Rules

Different APIs use different SG counts.

API Count
dma_map_sg() Original nents
dmaengine_prep_slave_sg() Mapped mapped_nents
dma_sync_sg_for_cpu() Original nents
dma_sync_sg_for_device() Original nents
dma_unmap_sg() Original nents

The lifecycle can be summarized as:

Original Scatterlist
        │ nents = N
dma_map_sg(..., N)
        │ returns M
DMA-Visible Segments
        │ mapped_nents = M
dmaengine_prep_slave_sg(..., M)

When returning to the DMA mapping API:

dma_sync_sg_for_cpu(..., N)

dma_sync_sg_for_device(..., N)

dma_unmap_sg(..., N)

This distinction is essential when writing SG DMA clients.


9. struct sg_table

Subsystems frequently manage scatterlists using:

struct sg_table

Conceptually, the important fields are:

sgl
    Scatterlist

orig_nents
    Original SG entry count

nents
    DMA-mapped segment count

A typical lifecycle is:

sg_alloc_table()
orig_nents = N
dma_map_sgtable()
nents = M

Internally, dma_map_sgtable() maps using:

sgt->orig_nents

and stores the mapped count in:

sgt->nents

Cleanup uses the original count again:

dma_unmap_sgtable()
sgt->orig_nents

This allows the structure to preserve both representations at the same time.


10. DMA Mapping Direction

Scatter-gather transfers use the same DMA mapping direction rules as single-buffer streaming mappings.

TX

CPU memory
Device

Use:

DMA_TO_DEVICE

RX

Device
CPU memory

Use:

DMA_FROM_DEVICE

These values belong to:

enum dma_data_direction

They describe how the device accesses memory.


11. DMA Engine Direction

DMA Engine uses a different direction abstraction.

For slave DMA:

TX

Memory
Peripheral

Use:

DMA_MEM_TO_DEV

RX

Peripheral
Memory

Use:

DMA_DEV_TO_MEM

These values belong to:

enum dma_transfer_direction

Therefore:

Transfer DMA Mapping DMA Engine
TX DMA_TO_DEVICE DMA_MEM_TO_DEV
RX DMA_FROM_DEVICE DMA_DEV_TO_MEM

The two enums describe different abstraction layers.


12. Preparing a Slave SG Transaction

After mapping, the SG transaction can be prepared with:

desc = dmaengine_prep_slave_sg(chan,
                               sg,
                               mapped_nents,
                               DMA_MEM_TO_DEV,
                               DMA_PREP_INTERRUPT |
                               DMA_CTRL_ACK);
if (!desc) {
    dma_unmap_sg(dev, sg, nents, DMA_TO_DEVICE);
    return -EIO;
}

The important point is:

sg_len = mapped_nents

not the original entry count.

The DMA Engine provider receives the DMA-visible SG representation.


13. Scatter-Gather Transfer Lifecycle

DMA scatter-gather transfer lifecycle

The three important counts belong to different layers:

Original SG entries
        N
        │ DMA mapping
DMA-visible segments
        M
        │ provider conversion
Hardware descriptors / LLIs
        H

The counts should not be assumed to be equal.


14. SG Segment vs Hardware Descriptor

A DMA-visible SG segment is still a software-level DMA representation.

The DMA controller provider must convert it into whatever hardware representation the controller requires.

Therefore:

Mapped SG segment
Hardware descriptor / LLI

A provider may split one mapped segment into multiple hardware descriptors because of:

  • Maximum block transfer size
  • Address alignment
  • Transfer width
  • Burst restrictions
  • Controller-specific limits

For example:

Mapped Segment 0
        ├── LLI 0
        ├── LLI 1
        └── LLI 2

Mapped Segment 1
        └── LLI 3

Thus:

mapped_nents = 2

hardware descriptor count = 4

is entirely valid.


15. Synopsys AXI DMA Provider Example

The Synopsys AXI DMA provider demonstrates this conversion directly.

Its slave SG preparation path is:

dw_axi_dma_chan_prep_slave_sg()
        ├── iterate mapped SG
        ├── sg_dma_address()
        ├── sg_dma_len()
        ├── calculate_block_len()
        ├── determine required descriptor count
        ├── split segments if necessary
        └── dw_axi_dma_set_hw_desc()

The provider first determines how many hardware descriptors are required.

Conceptually:

for_each_sg(sgl, sg, sg_len, i)
    num_sgs += DIV_ROUND_UP(sg_dma_len(sg), axi_block_len);

The count named num_sgs in this provider is therefore closer to the required hardware descriptor count than the original client-side SG entry count.


16. Hardware Block Length

The provider uses calculate_block_len() to determine how many bytes can fit in one hardware DMA block.

The basic relationship is:

block length
    =
hardware block transfer count
    ×
bytes per transfer

The transfer width depends on factors such as:

  • DMA controller data width
  • DMA address alignment
  • Buffer length alignment
  • Configured peripheral width
  • Transfer direction

For memory-to-device transfers, the memory-side width is derived from the memory address and length alignment and is limited by the provider implementation.

For device-to-memory transfers, block sizing uses the configured peripheral source width.


17. Building Hardware LLIs

The provider eventually calls:

dw_axi_dma_set_hw_desc()

to populate controller-specific LLI fields.

For memory-to-device:

Memory
    │ SAR
    │ increment
DMA Controller
    │ DAR
    │ no increment
Peripheral

For device-to-memory:

Peripheral
    │ SAR
    │ no increment
DMA Controller
    │ DAR
    │ increment
Memory

The helper prepares fields corresponding to:

  • Source address
  • Destination address
  • Transfer width
  • Increment behavior
  • Block transfer count
  • Burst length
  • LLI control
  • Linked-list pointer

This is the provider-specific translation from the DMA Engine transaction model to the hardware programming model.


18. SPI Core as the Mapping Owner

A useful real-world example is the Linux SPI subsystem.

The SPI Core can build and map the scatter-gather table before invoking the controller driver's DMA path.

The conceptual ownership is:

SPI Core
────────────────────────────

Buffer
Build SG table
dma_map_sgtable()
Mapped tx_sg / rx_sg


SPI Controller Driver
────────────────────────────

tx_sg / rx_sg
dmaengine_prep_slave_sg()
submit
issue

Therefore, a controller driver may use DMA-mapped SG entries without calling dma_map_sg() itself.

The correct question is:

Which layer owns DMA mapping?

not:

Does this driver source file contain dma_map_sg()?

19. SPI Core SG Construction

The SPI Core helper spi_map_buf_attrs() demonstrates how a subsystem can determine the original SG representation.

For a normal kernel virtual buffer, the buffer can be divided according to:

dma_get_max_seg_size()
ctlr->max_dma_len

For a vmalloc() buffer, the SPI Core instead walks the backing pages.

Conceptually:

vmalloc() buffer
vmalloc_to_page()
sg_set_page()
Original SG table
dma_map_sgtable()

This demonstrates that:

vmalloc() = virtually contiguous

does not imply:

vmalloc() = impossible for DMA

The subsystem must construct an appropriate page-based representation and map it for the DMA device.

Actual usability still depends on the subsystem, DMA device, and hardware constraints.


20. BCM2835 SPI SG Path

The BCM2835 SPI controller driver consumes the SG tables prepared by the SPI Core.

For TX:

nents = tfr->tx_sg.nents;
sgl   = tfr->tx_sg.sgl;

For RX:

nents = tfr->rx_sg.nents;
sgl   = tfr->rx_sg.sgl;

The controller driver then prepares the DMA Engine transaction:

desc = dmaengine_prep_slave_sg(chan,
                               sgl,
                               nents,
                               dir,
                               flags);

Because the SPI Core has already performed DMA mapping:

tfr->tx_sg.nents
tfr->rx_sg.nents

represent the mapped DMA segment count used by the DMA Engine transaction.


21. BCM2835 Execution Ordering

The BCM2835 DMA path intentionally starts TX before finishing the RX-side preparation.

Conceptually:

Prepare TX
Submit TX
Program SPI hardware
Enable DMA requests
Issue TX
    │ hardware begins useful work
    └──────────────┐
             Prepare RX
              Submit RX
               Issue RX

This overlaps software preparation with hardware execution.

It also demonstrates why error cleanup must distinguish whether a DMA transaction has already been issued.


22. Pre-Issue Failure

Suppose mapping succeeds but transaction preparation fails:

dma_map_sg()
dmaengine_prep_slave_sg()
dma_unmap_sg()

The DMA transaction has not been issued.

Therefore no active DMA transaction needs to be terminated.

The same principle applies to a submission failure before dma_async_issue_pending().


23. Post-Issue Failure and Timeout

After:

dma_async_issue_pending(chan);

the DMA controller may already be accessing the SG buffers.

Therefore timeout or cancellation cleanup must first resolve DMA activity:

DMA issued
timeout / cancellation
dmaengine_terminate_sync()
DMA and completion activity resolved
dma_unmap_sg()

The central safety question remains:

Can DMA hardware or asynchronous completion activity still access this resource?


24. One-Shot SG TX Example

A simplified blocking TX sequence is:

mapped_nents = dma_map_sg(dev, sg, nents, DMA_TO_DEVICE);
if (!mapped_nents)
    return -EIO;

desc = dmaengine_prep_slave_sg(chan,
                               sg,
                               mapped_nents,
                               DMA_MEM_TO_DEV,
                               DMA_PREP_INTERRUPT |
                               DMA_CTRL_ACK);
if (!desc) {
    ret = -EIO;
    goto err_unmap;
}

desc->callback = dma_tx_done;
desc->callback_param = &done;

cookie = dmaengine_submit(desc);
ret = dma_submit_error(cookie);
if (ret)
    goto err_unmap;

dma_async_issue_pending(chan);

timeout = wait_for_completion_timeout(&done,
                                      msecs_to_jiffies(5000));
if (!timeout) {
    ret = -ETIMEDOUT;
    goto err_terminate;
}

dma_unmap_sg(dev, sg, nents, DMA_TO_DEVICE);

return 0;

err_terminate:
    dmaengine_terminate_sync(chan);

err_unmap:
    dma_unmap_sg(dev, sg, nents, DMA_TO_DEVICE);

return ret;

The important count transition is:

dma_map_sg(..., nents)
mapped_nents
dmaengine_prep_slave_sg(..., mapped_nents)

cleanup
dma_unmap_sg(..., nents)

25. One-Shot SG RX

For RX:

dma_map_sg(..., DMA_FROM_DEVICE)
dmaengine_prep_slave_sg(..., DMA_DEV_TO_MEM)
submit
issue
DMA completion
dma_unmap_sg(..., DMA_FROM_DEVICE)
CPU accesses RX data

When the one-shot streaming mapping is immediately unmapped, an additional:

dma_sync_sg_for_cpu()

is not required before CPU access.

The unmap operation ends the streaming mapping lifecycle and returns ownership to the CPU side.


26. Persistent SG Mapping

Sometimes the DMA mapping remains active across multiple transfers.

In that case the mapping is not immediately destroyed after every DMA completion.

Instead:

Device ownership
DMA completion / safe handoff point
dma_sync_sg_for_cpu()
CPU ownership
CPU accesses data
dma_sync_sg_for_device()
Device ownership
next DMA activity

The sync APIs use the original SG entry count.


27. DMA Mapping Synchronization vs DMA Execution

A critical distinction is:

dma_sync_sg_for_cpu()
pause DMA controller

The DMA mapping API manages CPU/device ownership and architecture-specific cache synchronization.

It does not establish that the DMA hardware has stopped accessing a particular region.

The driver must first know from the DMA execution lifecycle that the region is safe to hand to the CPU.

For example:

DMA currently using P2

P0 completed
driver knows P0 is currently safe
dma_sync_sg_for_cpu()
CPU reads P0

Calling dma_sync_sg_for_cpu() alone does not guarantee that hardware will not access P0.


28. Cyclic DMA and Producer/Consumer Ownership

Persistent SG mappings are especially important when reasoning about cyclic or ring-buffer DMA.

Consider:

P0 → P1 → P2 → P3
↑              │
└──────────────┘

DMA may act as a producer while the CPU acts as a consumer.

If DMA has moved from P0 to P1, the CPU may process P0 after the appropriate ownership synchronization.

However:

DMA:
P0 → P1 → P2 → P3 → P0

CPU:
P0
still processing

means DMA has caught up with the CPU.

This is a ring-buffer overrun problem.

dma_sync_sg_for_cpu() does not prevent the DMA controller from wrapping around and reusing a period.


29. Three Synchronization Layers

Persistent and cyclic DMA often involve three independent synchronization concerns.

DMA Engine Execution Synchronization

Examples:

completion callback
dmaengine_terminate_sync()
period completion

This answers questions such as:

Has the DMA engine finished or stopped accessing this transaction or period?

DMA Mapping Ownership Synchronization

Examples:

dma_sync_sg_for_cpu()
dma_sync_sg_for_device()

This answers:

Is the mapped memory in the correct ownership/cache state for CPU or device access?

Producer/Consumer Synchronization

Examples:

producer index
consumer index
period state
overrun detection

This answers:

Has the producer caught up with the consumer?

These layers solve different problems and should not be treated as interchangeable.


30. Resource Ownership Model

A useful ownership model is:

Buffer allocation ownership
SG allocation ownership
DMA mapping ownership
DMA transaction ownership

For example, the SPI subsystem may have:

SPI Core
    owns SG construction
    owns DMA mapping
    owns DMA unmapping

SPI controller driver
    owns DMA Engine transaction preparation
    controls peripheral execution

DMA provider
    owns hardware descriptor construction
    programs DMA controller representation

Cleanup responsibilities should follow the corresponding ownership boundaries.


31. Common Mistakes

Mistake 1 — Passing Original nents to dmaengine_prep_slave_sg()

Wrong:

dmaengine_prep_slave_sg(chan,
                        sg,
                        nents,
                        direction,
                        flags);

after dma_map_sg() returned a different mapped count.

Correct:

dmaengine_prep_slave_sg(chan,
                        sg,
                        mapped_nents,
                        direction,
                        flags);

Mistake 2 — Unmapping with mapped_nents

Wrong:

dma_unmap_sg(dev,
             sg,
             mapped_nents,
             DMA_TO_DEVICE);

Correct:

dma_unmap_sg(dev,
             sg,
             nents,
             DMA_TO_DEVICE);

The unmap API receives the original count passed to dma_map_sg().


Mistake 3 — Reading RX Memory Before Returning Ownership

For a one-shot RX mapping:

Wrong:

DMA completion
CPU reads buffer
dma_unmap_sg()

Correct:

DMA completion
dma_unmap_sg()
CPU reads buffer

For a persistent mapping, use the appropriate sync operation instead of unmapping.


Mistake 4 — Treating dma_sync_sg_for_cpu() as a DMA Stop Operation

Wrong mental model:

dma_sync_sg_for_cpu()
DMA stops accessing buffer

Correct mental model:

DMA lifecycle says region is safe
dma_sync_sg_for_cpu()
CPU-side ownership/cache synchronization

Mistake 5 — Assuming One SG Entry Means One Hardware LLI

Wrong:

1 SG entry
    =
1 hardware descriptor

Correct:

SG representation
DMA mapping
mapped DMA segment
provider hardware constraints
one or more hardware descriptors

32. Mental Model

The complete scatter-gather slave DMA model is:

CPU Buffer / Pages
Original Scatterlist
        │ original count N
DMA Mapping
        │ mapped count M
DMA-Visible Segments
dmaengine_prep_slave_sg()
DMA Controller Provider
        │ apply hardware constraints
Hardware Descriptor / LLI Chain
DMA Hardware Execution
Completion / Synchronization
DMA Unmap or Ownership Sync
CPU Access

The most important rule is:

Keep the CPU memory representation, DMA-visible representation, and hardware descriptor representation as three separate layers.

Once those layers are separated, SG counts, mapping lifetime, provider splitting, and cleanup responsibilities become much easier to reason about.


Summary

Scatter-gather DMA extends the slave DMA transaction model from one DMA address range to a mapped collection of memory segments.

The key relationships are:

Original SG entries
        │ dma_map_sg()
DMA-visible segments
        │ dmaengine_prep_slave_sg()
DMA provider
        │ hardware-specific splitting
Hardware descriptors / LLIs

The original SG count is used by the DMA mapping lifecycle, while the mapped count is passed to the DMA Engine provider.

The provider then converts those DMA-visible segments into the controller-specific descriptor format required by the hardware.

Understanding these boundaries is essential for writing correct DMA client drivers, especially when handling non-contiguous memory, subsystem-owned mappings, persistent mappings, cyclic buffers, timeout cleanup, and controller-specific transfer limits.