Day 108 — DMA Scatter-Gather Transfer and `dmaengine_prep_slave_sg()`¶

Today's Goal¶

Understand how Linux represents non-contiguous or segmented memory for DMA using scatter-gather lists, how DMA mapping transforms the original scatterlist into DMA-visible segments, and how dmaengine_prep_slave_sg() passes those mapped segments to a DMA controller provider.

The main focus is the relationship between:

struct scatterlist
struct sg_table
Original SG entries
DMA-mapped SG segments
dma_map_sg()
dma_map_sgtable()
sg_dma_address()
sg_dma_len()
dmaengine_prep_slave_sg()
DMA controller hardware descriptors and LLIs
SG mapping lifetime
Persistent mapping and dma_sync_sg_*()

The implementation goal is to extend the Day107 one-shot DMA client workflow from a single DMA address range to a scatter-gather transaction.

1. Scatter-Gather Memory Representation¶

Scatter-gather DMA allows one DMA transaction to describe memory using multiple regions without first copying the data into one contiguous temporary buffer.

Conceptually:

CPU memory regions
        │
        ▼
struct scatterlist
        │
        ▼
DMA mapping
        │
        ▼
DMA-visible segments
        │
        ▼
DMA Engine
        │
        ▼
DMA controller hardware descriptors

A scatterlist should not be understood only as "multiple buffers chained together."

A single logical buffer may also require multiple SG entries because of:

Page boundaries
DMA maximum segment size
Controller DMA length limits
Memory type
Subsystem-specific constraints

2. Building an Original Scatterlist¶

A fixed scatterlist can be initialized with:

struct scatterlist sg[3];
unsigned int nents = ARRAY_SIZE(sg);

sg_init_table(sg, nents);

sg_set_buf(&sg[0], buf0, len0);
sg_set_buf(&sg[1], buf1, len1);
sg_set_buf(&sg[2], buf2, len2);

At this point, the scatterlist describes the CPU-side memory representation.

Conceptually:

buf0 + len0 ──► SG[0]
buf1 + len1 ──► SG[1]
buf2 + len2 ──► SG[2]

Original SG count = nents

The original SG entry count describes the memory representation known by the kernel component building the scatterlist.

It does not represent the number of final DMA-visible segments or hardware descriptors.

3. SG Entry Count Is Not Physical Fragmentation¶

A driver or subsystem does not normally guess the required scatterlist size from physical memory fragmentation.

Instead, it builds an SG representation from memory information it already owns.

Possible sources include:

A known set of kernel buffers
A known set of struct page objects
A subsystem-provided struct sg_table
A virtual buffer divided according to page or DMA constraints

The important question is:

"What memory representation do I currently have?"

not:

"How fragmented is physical RAM?"

DMA address-space fragmentation is handled later by the DMA mapping layer.

4. DMA Mapping of Scatter-Gather Lists¶

For a TX transaction:

mapped_nents = dma_map_sg(dev, sg, nents, DMA_TO_DEVICE);
if (!mapped_nents) {
    /* Handle DMA mapping failure. */
}

For RX:

mapped_nents = dma_map_sg(dev, sg, nents, DMA_FROM_DEVICE);

Unlike dma_map_single(), dma_map_sg() returns the number of DMA-visible segments rather than a DMA address.

A return value of zero indicates mapping failure.

For a successful mapping:

0 < mapped_nents <= nents

5. Original Entries vs DMA-Mapped Segments¶

DMA mapping may merge original SG entries when they can be represented as a larger DMA-visible segment.

For example:

Original SG entries

SG0
SG1
SG2
SG3

nents = 4

        │
        │ dma_map_sg()
        ▼

DMA-visible segments

Segment 0
Segment 1
Segment 2

mapped_nents = 3

Therefore:

Original SG entry
        ≠
DMA-mapped segment

The mapping layer may reduce the number of segments but does not increase the mapped segment count beyond the original entry count.

6. CPU-Side vs DMA-Side SG Information¶

Before mapping, scatterlist entries primarily describe CPU-side memory information such as:

page
offset
length

After mapping, the DMA-visible information must be obtained using:

sg_dma_address(sg);
sg_dma_len(sg);

The responsibility boundary is:

CPU-side memory representation
──────────────────────────────
sg_page()
sg->offset
sg->length

        │
        │ DMA mapping
        ▼

DMA-visible representation
──────────────────────────
sg_dma_address()
sg_dma_len()

A DMA controller provider should use the mapped DMA information rather than reconstructing DMA addresses from CPU virtual or physical addresses.

7. Count Semantics¶

The original and mapped entry counts have different responsibilities.

For the raw SG DMA APIs:

Operation	Count
`dma_map_sg()`	Original `nents`
`dmaengine_prep_slave_sg()`	`mapped_nents`
`dma_sync_sg_for_cpu()`	Original `nents`
`dma_sync_sg_for_device()`	Original `nents`
`dma_unmap_sg()`	Original `nents`

Conceptually:

dma_map_sg(..., N)
        │
        │ returns M
        ▼
dmaengine_prep_slave_sg(..., M)

DMA mapping lifecycle:
    sync(..., N)
    unmap(..., N)

This distinction is one of the most important rules when implementing scatter-gather DMA clients.

8. `struct sg_table`¶

Linux subsystems commonly use:

struct sg_table

which maintains both the original and mapped counts.

The important fields are conceptually:

sgl
    scatterlist

orig_nents
    original SG entry count

nents
    mapped DMA segment count after mapping

dma_map_sgtable() uses orig_nents as the mapping input and stores the resulting mapped count in nents.

The source path confirms:

sgt->orig_nents
        │
        ▼
__dma_map_sg_attrs()
        │
        │ returns mapped count
        ▼
sgt->nents

dma_unmap_sgtable() later uses sgt->orig_nents.

9. DMA Mapping Direction vs DMA Engine Direction¶

Scatter-gather DMA uses the same two direction abstractions learned on Day107.

For TX:

Layer	Direction
DMA Mapping API	`DMA_TO_DEVICE`
DMA Engine API	`DMA_MEM_TO_DEV`

For RX:

Layer	Direction
DMA Mapping API	`DMA_FROM_DEVICE`
DMA Engine API	`DMA_DEV_TO_MEM`

The mapping direction describes device access to memory.

The DMA Engine direction describes the transaction topology between memory and the peripheral endpoint.

10. `dmaengine_prep_slave_sg()`¶

After the SG has been mapped, a client can prepare the transaction with:

desc = dmaengine_prep_slave_sg(chan,
                               sg,
                               mapped_nents,
                               DMA_MEM_TO_DEV,
                               DMA_PREP_INTERRUPT |
                               DMA_CTRL_ACK);

The API receives the mapped segment count.

The conceptual flow is:

DMA-mapped SG
        │
        ▼
dmaengine_prep_slave_sg()
        │
        ▼
device_prep_slave_sg()
        │
        ▼
DMA controller provider

dmaengine_prep_slave_sg() does not perform DMA mapping.

The SG must remain mapped while the DMA transaction may access the memory.

11. SG Entry vs Hardware Descriptor¶

A mapped SG segment is not equivalent to one hardware DMA descriptor or LLI.

The provider may need to split one mapped segment because of:

Maximum hardware block size
Transfer width
Alignment
Burst constraints
Controller-specific limits

Conceptually:

Original SG entries
        N
        │
        │ DMA mapping
        ▼
Mapped DMA segments
        M
        │
        │ provider conversion
        ▼
Hardware LLIs
        H

M <= N, but the hardware descriptor count H is controller dependent and may be greater than M.

12. Synopsys AXI DMA Provider Source Trace¶

The Synopsys AXI DMA provider uses:

sg_dma_address(sg);
sg_dma_len(sg);

when processing mapped SG segments.

Its preparation path is conceptually:

dw_axi_dma_chan_prep_slave_sg()
        │
        ├── iterate mapped SG
        ├── obtain DMA address and length
        ├── calculate hardware block length
        ├── count required hardware descriptors
        ├── split large mapped segments
        └── build hardware LLIs

The provider first calculates how many hardware descriptors are required and then allocates descriptor storage.

For each mapped segment it may create one or more hardware descriptors using:

dw_axi_dma_set_hw_desc()

13. Hardware Block Length¶

calculate_block_len() calculates the maximum byte length that can fit in one hardware DMA block under the current width constraints.

Conceptually:

Hardware block transfer count
        ×
Bytes per transfer
        ↓
Maximum hardware block length

For memory-to-device transfers, the provider derives a safe memory transfer width from:

Controller data width
DMA address alignment
Buffer length alignment

The implementation further limits the memory-side width to 32 bits on this path.

For device-to-memory transfers, block sizing uses the configured peripheral source width.

14. Building the Hardware LLI¶

dw_axi_dma_set_hw_desc() translates the higher-level DMA transaction information into controller-specific hardware descriptor fields.

The function configures information such as:

SAR
DAR
Source and destination transfer width
Increment / no-increment behavior
Block transfer count
Burst length
LLI control fields

For TX:

Memory
    SAR
    INC

Peripheral
    DAR
    NOINC

For RX:

Peripheral
    SAR
    NOINC

Memory
    DAR
    INC

The helper constructs a hardware programming image in an LLI rather than directly programming every transfer through MMIO registers.

15. BCM2835 SPI Client Source Trace¶

The BCM2835 SPI DMA path consumes SG tables already prepared by the SPI Core.

For TX:

nents = tfr->tx_sg.nents;
sgl   = tfr->tx_sg.sgl;

For RX:

nents = tfr->rx_sg.nents;
sgl   = tfr->rx_sg.sgl;

The driver then calls:

dmaengine_prep_slave_sg(chan, sgl, nents, dir, flags);

The BCM2835 SPI driver does not perform the DMA mapping itself on this path.

This reinforces the Day107 rule:

A DMA client driver does not call dma_map_sg()
        ≠
No DMA mapping occurred

The first question should always be:

Which layer owns the DMA mapping?

16. SPI Core SG Construction¶

The SPI Core prepares SG tables in spi_map_buf_attrs().

For normal kernel buffers, it divides the buffer according to:

DMA maximum segment size
ctlr->max_dma_len

For vmalloc() or kmap-backed buffers, it creates page-based entries.

The vmalloc() path uses:

vmalloc_to_page()
sg_set_page()

Conceptually:

vmalloc buffer
        │
        ▼
walk backing pages
        │
        ▼
vmalloc_to_page()
        │
        ▼
sg_set_page()
        │
        ▼
SG table
        │
        ▼
dma_map_sgtable()

This confirms that:

vmalloc() ≠ DMA impossible

but an appropriate page-based SG representation and DMA mapping path are required.

17. SPI Core Mapping Ownership¶

The SPI Core maps the completed SG table using:

dma_map_sgtable()

After successful mapping:

sgt->orig_nents
    = original SG entry count

sgt->nents
    = mapped DMA segment count

BCM2835 SPI uses sgt->nents when preparing the DMA Engine transaction.

On cleanup, the SPI Core performs:

dma_unmap_sgtable()
        ↓
sg_free_table()
        ↓
reset nents / orig_nents

This demonstrates a subsystem-level ownership model in which the SPI Core owns the SG allocation and DMA mapping lifetime while the controller driver owns DMA Engine transaction preparation and peripheral execution.

18. BCM2835 DMA Execution Ordering¶

The BCM2835 SPI DMA path intentionally overlaps hardware execution and software preparation.

The approximate sequence is:

Prepare / submit TX DMA
        ↓
Program SPI transfer length
        ↓
Enable SPI DMA mode
        ↓
Issue TX DMA
        ↓
Prepare / submit RX DMA
        ↓
Issue RX DMA
        ↓
Asynchronous completion

This shows that real drivers do not always follow a simplified sequence in which every descriptor is prepared before hardware execution starts.

However, mapping lifetime and synchronization rules must still remain valid.

19. Guided Implementation — Blocking SG TX¶

The SG TX exercise used three fixed buffers.

The initial SG representation was:

struct scatterlist sg[3];
unsigned int nents = ARRAY_SIZE(sg);
int mapped_nents;

sg_init_table(sg, nents);

sg_set_buf(&sg[0], buf0, len0);
sg_set_buf(&sg[1], buf1, len1);
sg_set_buf(&sg[2], buf2, len2);

Mapping:

mapped_nents = dma_map_sg(dev, sg, nents, DMA_TO_DEVICE);
if (!mapped_nents)
    return -EIO;

Preparation:

desc = dmaengine_prep_slave_sg(chan,
                               sg,
                               mapped_nents,
                               DMA_MEM_TO_DEV,
                               DMA_PREP_INTERRUPT |
                               DMA_CTRL_ACK);
if (!desc) {
    dma_unmap_sg(dev, sg, nents, DMA_TO_DEVICE);
    return -EIO;
}

The transaction then follows:

Set callback
        ↓
dmaengine_submit()
        ↓
dma_submit_error()
        ↓
dma_async_issue_pending()
        ↓
wait_for_completion_timeout()
        ↓
completion
        ↓
dma_unmap_sg()

20. Pre-Issue and Post-Issue Cleanup¶

If preparation or submission fails before dma_async_issue_pending():

DMA mapped
        ↓
prep / submit failure
        ↓
dma_unmap_sg()

No active DMA transfer needs to be terminated.

After issue:

dma_async_issue_pending()
        ↓
DMA may access SG buffers
        ↓
timeout / cancellation
        ↓
dmaengine_terminate_sync()
        ↓
dma_unmap_sg()

This preserves the Day107 resource safety rule:

Can DMA hardware or asynchronous completion activity
still access this resource?

21. Guided Implementation — SG RX¶

For RX, the direction pair becomes:

DMA Mapping:
    DMA_FROM_DEVICE

DMA Engine:
    DMA_DEV_TO_MEM

After successful one-shot DMA completion:

DMA completion
        ↓
dma_unmap_sg(..., DMA_FROM_DEVICE)
        ↓
CPU accesses RX buffers

No additional dma_sync_sg_for_cpu() is required when the streaming mapping is immediately unmapped.

22. Persistent SG Mapping¶

If the mapping remains active across repeated DMA operations, CPU/device ownership can be transferred with:

Device ownership
        ↓
dma_sync_sg_for_cpu()
        ↓
CPU access
        ↓
dma_sync_sg_for_device()
        ↓
Device ownership

Both synchronization APIs use the original SG entry count, not the mapped segment count.

23. DMA Mapping Synchronization Does Not Pause DMA¶

dma_sync_sg_for_cpu() performs DMA mapping and cache ownership synchronization.

It does not stop or pause the DMA controller.

For a cyclic or persistent DMA buffer:

DMA producer
    P0 → P1 → P2 → P3 → P0 ...

CPU consumer
    processes completed periods

The driver must separately determine that a region is safe for CPU access based on DMA progress or period lifecycle.

If DMA wraps around before the CPU finishes processing a period, the problem is a producer/consumer overrun rather than something dma_sync_sg_*() can prevent.

Three different synchronization concerns should remain separate:

DMA Engine execution synchronization
        ↓
completion / terminate_sync()

DMA mapping ownership synchronization
        ↓
dma_sync_sg_for_cpu()
dma_sync_sg_for_device()

Producer / consumer synchronization
        ↓
period state / indices / overrun handling

Important Observations¶

struct scatterlist is a kernel memory representation, not a hardware DMA descriptor.
An SG list does not need to correspond one-to-one with physical fragmentation.
One virtual buffer can generate multiple original SG entries.
dma_map_sg() may merge original SG entries into fewer DMA-visible segments.
A successful mapped segment count is always less than or equal to the original entry count.
sg_dma_address() and sg_dma_len() represent the DMA-side view.
DMA Mapping API lifecycle operations use the original SG count.
dmaengine_prep_slave_sg() uses the mapped DMA segment count.
struct sg_table preserves both orig_nents and nents.
One mapped SG segment may become multiple hardware LLIs.
The Synopsys AXI DMA provider converts mapped SG information into controller-specific SAR, DAR, width, increment mode, block count, burst, and LLI fields.
The SPI Core can build page-based SG entries for vmalloc() buffers.
BCM2835 SPI consumes SG tables mapped by the SPI Core instead of performing mapping itself.
DMA mapping ownership and DMA hardware execution synchronization are separate concerns.
dma_sync_sg_for_cpu() does not pause DMA hardware.

Summary¶

Day108 extended the one-shot slave DMA model from a single DMA address range to scatter-gather memory.

The key three-layer model is:

CPU Memory Representation
        ↓
Original Scatterlist
        ↓
DMA Mapping
        ↓
DMA-Visible Segments
        ↓
DMA Engine Provider
        ↓
Hardware Descriptors / LLIs

The most important count distinction is:

Original SG count
        │
        │ dma_map_sg()
        ▼
Mapped DMA segment count
        │
        │ DMA provider
        ▼
Hardware descriptor count

These counts belong to different abstraction layers and must not be treated as interchangeable.

The Linux source trace through the SPI Core, BCM2835 SPI driver, DMA mapping API, and Synopsys AXI DMA provider confirmed the entire path from CPU buffer representation through DMA-visible segments to hardware LLIs.

Next Plan¶

Continue the DMA Engine learning path by building on the slave SG transaction lifecycle and examining the next client-side DMA pattern.

The next session should continue emphasizing:

Real DMA client implementation patterns
Resource ownership and cleanup
Existing Linux client-driver source paths
Provider/client responsibility boundaries
Practical preparation for future hardware DMA labs

Day 108 — DMA Scatter-Gather Transfer and dmaengine_prep_slave_sg()¶

Today's Goal¶

1. Scatter-Gather Memory Representation¶

2. Building an Original Scatterlist¶

3. SG Entry Count Is Not Physical Fragmentation¶

4. DMA Mapping of Scatter-Gather Lists¶

5. Original Entries vs DMA-Mapped Segments¶

6. CPU-Side vs DMA-Side SG Information¶

7. Count Semantics¶

8. struct sg_table¶

9. DMA Mapping Direction vs DMA Engine Direction¶

10. dmaengine_prep_slave_sg()¶

11. SG Entry vs Hardware Descriptor¶

12. Synopsys AXI DMA Provider Source Trace¶

13. Hardware Block Length¶

14. Building the Hardware LLI¶

15. BCM2835 SPI Client Source Trace¶

16. SPI Core SG Construction¶

17. SPI Core Mapping Ownership¶

18. BCM2835 DMA Execution Ordering¶

19. Guided Implementation — Blocking SG TX¶

20. Pre-Issue and Post-Issue Cleanup¶

21. Guided Implementation — SG RX¶

22. Persistent SG Mapping¶

23. DMA Mapping Synchronization Does Not Pause DMA¶

Important Observations¶

Summary¶

Next Plan¶

Day 108 — DMA Scatter-Gather Transfer and `dmaengine_prep_slave_sg()`¶

8. `struct sg_table`¶

10. `dmaengine_prep_slave_sg()`¶