DMA Client Driver Architecture¶
Overview¶
A Linux peripheral driver that uses DMA Engine is a DMA client.
The DMA controller driver provides DMA Engine operations, but the client driver remains responsible for coordinating:
- the peripheral
- DMA channels
- DMA mappings
- DMA descriptors
- transfer completion
- error recovery
- buffer ownership
- producer and consumer lifetimes
A complete DMA client therefore manages several related but independent lifetimes:
DMA channel lifetime
≠
DMA transaction lifetime
≠
DMA mapping lifetime
≠
buffer lifetime
≠
consumer lifetime
≠
peripheral completion lifetime
Correct DMA integration depends on keeping these lifetimes properly ordered.
DMA Client Architecture¶
At a high level:
Peripheral Driver
│
│ DMA Engine client APIs
▼
DMA Engine Framework
│
▼
DMA Controller Driver
│
▼
DMA Controller Hardware
│
├──────── Memory
│
└──────── Peripheral
The DMA controller driver knows how to program the DMA controller.
The peripheral client driver knows:
- when a transfer should begin
- which peripheral FIFO is involved
- which direction the transfer uses
- where the data buffer comes from
- when the peripheral should be enabled
- when the peripheral transaction is actually finished
- what should happen after completion
- how errors and timeouts should be recovered
- when a completed buffer region may be consumed
- when that region may safely be reused by DMA
DMA Engine does not replace these client responsibilities.
Driver Lifetime and Transfer Lifetime¶
DMA channel setup is commonly associated with the driver lifetime.
Conceptually:
probe()
│
├── request DMA channel
│
├── configure slave DMA
│
▼
READY
│
├── transfer
├── transfer
├── stream
└── ...
│
▼
remove()
│
└── release DMA channel
Transfer-specific resources usually have a shorter lifetime.
For a request-based transfer:
Request
│
▼
Obtain Buffer
│
▼
DMA Map
│
▼
Prepare Descriptor
│
▼
Submit / Issue
│
▼
Transfer
│
▼
Complete / Terminate
│
▼
Unmap
│
▼
Release Buffer
A streaming transfer may use a much longer mapping lifetime:
Enable Stream
│
▼
Map Buffer
│
▼
Prepare Stream
│
▼
Start DMA
│
├── block completion
├── block completion
├── block completion
└── ...
│
▼
Stop Stream
│
▼
Terminate / Synchronize
│
▼
Release Outstanding Consumers
│
▼
Unmap Buffer
A driver should therefore avoid treating DMA channel ownership, DMA mappings, individual descriptors, and buffer consumption as if they were one resource with one lifetime.
DMA Client Implementation Sequence¶
The following sequence shows a typical request-based slave DMA client lifecycle.
The sequence can be divided into four phases.
1. Driver Setup¶
The driver first obtains and configures the DMA channel:
The configuration describes the peripheral-side DMA characteristics, such as:
- peripheral FIFO address
- transfer direction
- bus width
- burst size
This configuration may remain valid across multiple transfers.
2. Request Setup¶
When a transfer request arrives:
The exact preparation API depends on the transfer representation.
Examples include:
Contiguous slave buffer
→ dmaengine_prep_slave_single()
Scatter-gather transfer
→ dmaengine_prep_slave_sg()
Continuous cyclic transfer
→ dmaengine_prep_dma_cyclic()
3. Submission and Peripheral Start¶
After preparation:
The distinction between ARMED and ACTIVE is important for peripheral DMA.
4. Completion or Error Cleanup¶
The transaction eventually reaches either:
or:
Both paths must reach a point where DMA can no longer access resources that are about to be unmapped or freed.
Prepare, Submit, Issue, and Start¶
A useful client-side mental model is:
PREPARED
│
│ dmaengine_submit()
▼
SUBMITTED
│
│ dma_async_issue_pending()
▼
ARMED
│
│ peripheral starts generating DMA requests
▼
ACTIVE
These states are conceptual rather than literal DMA Engine state names, but they help explain the ordering.
Prepared¶
The descriptor describes the transfer.
For example:
At this point the transaction has not yet been submitted.
Submitted¶
Calling:
submits the prepared transaction.
The descriptor is now managed by the DMA Engine/provider submission path.
Armed¶
Calling:
allows pending transactions to be issued toward the DMA controller.
For peripheral slave DMA, this does not necessarily mean that data is already moving.
Active¶
Actual data movement may depend on DMA requests generated by the peripheral.
Therefore the client often follows an ordering similar to:
prepare DMA
↓
submit DMA
↓
issue pending
↓
enable peripheral
↓
peripheral generates DMA request
↓
DMA transfers data
The exact hardware ordering remains device-specific.
Why DMA Is Often Armed Before the Peripheral Starts¶
Consider an RX peripheral such as an ADC-like device.
If the peripheral begins producing samples before the DMA path is ready:
A safer architecture is:
The client does not normally wait for a separate generic DMA Engine "ready callback" before enabling the peripheral.
Instead, the driver relies on the defined semantics of its DMA controller/provider and the peripheral-specific startup sequence.
DMA Completion Is Not Peripheral Completion¶
DMA completion describes the DMA transaction.
Peripheral completion describes the peripheral protocol or hardware operation.
These are not always the same event.
For a transmit path:
DMA may finish moving the last bytes into the peripheral before the peripheral has shifted the last bit onto the physical interface.
Therefore:
A client that must know when the physical transfer has ended may also need to inspect peripheral-specific state.
Examples include:
- TX FIFO empty
- shift register empty
- bus transaction complete
- chip-select release condition
- protocol-specific completion interrupt
This distinction is especially important before disabling clocks, resetting hardware, or declaring an external transaction finished.
DMA Completion and Mapping Lifetime¶
DMA completion does not automatically destroy a DMA mapping.
For a one-shot transfer:
The transaction lifetime and mapping lifetime therefore overlap but are not identical.
For continuous streaming, this distinction becomes even more important:
Persistent Mapping
│
├── Block 0 complete
├── Block 1 complete
├── Block 2 complete
├── Block 3 complete
├── Block 0 reused
└── ...
│
▼
Stream Termination
│
▼
Unmap
The mapping may survive many individual DMA completion events.
DMA Mapping Direction vs DMA Engine Direction¶
DMA clients commonly use two direction representations.
For an RX transfer:
For a TX transfer:
These describe related data flow but belong to different APIs.
They should not be treated as interchangeable constants.
| Physical Flow | DMA Engine Direction | DMA Mapping Direction |
|---|---|---|
| Memory → Peripheral | DMA_MEM_TO_DEV |
DMA_TO_DEVICE |
| Peripheral → Memory | DMA_DEV_TO_MEM |
DMA_FROM_DEVICE |
RX Ownership and Synchronization¶
For a streaming DMA_FROM_DEVICE mapping, CPU and device ownership must be respected.
For a one-shot buffer:
CPU owns buffer
↓
map / sync for device
↓
Device owns buffer
↓
DMA writes data
↓
DMA completion
↓
sync for CPU
↓
CPU owns valid data
For a persistent streaming mapping, ownership may change at a smaller granularity than the complete mapping.
Conceptually:
Persistent DMA Mapping
┌────────┬────────┬────────┬────────┐
│ Block0 │ Block1 │ Block2 │ Block3 │
└────────┴────────┴────────┴────────┘
Block0 → CPU-visible completed data
Block1 → DMA active
Block2 → available for later DMA
Block3 → available for later DMA
The exact synchronization behavior depends on:
- mapping type
- platform coherency
- DMA API requirements
- buffer implementation
The important architectural distinction is:
Request-Based DMA¶
A request-based DMA architecture associates a DMA transaction with one logical operation.
Examples include:
- SPI transfer
- finite ADC acquisition
- command-based RX
- individual TX request
- finite scatter-gather transfer
- FIFO read initiated by a hardware event
The typical lifecycle is:
Request
↓
Map
↓
Prepare
↓
Submit / Issue
↓
Start Peripheral
↓
Complete
↓
Sync / Unmap
↓
Request Done
Completion normally means the current logical DMA transaction is finished.
Streaming DMA¶
Continuous data acquisition uses a different lifetime model.
Examples include:
- continuous ADC sampling
- audio capture
- audio playback
- continuously operating hardware streams
A mapping and cyclic DMA transaction may remain active for much longer:
Enable Stream
↓
Persistent Buffer / Mapping
↓
Prepare Cyclic DMA
↓
Submit / Issue
↓
Start Peripheral
↓
Block Ready
↓
Consumer Processes Data
↓
Next Block
↓
...
A cyclic callback usually means:
rather than:
Continuous Sampling Does Not Always Mean Continuous DMA¶
A peripheral may produce samples continuously while still exposing a request-based data transport.
For example, an external SPI sensor or ADC may internally sample at a fixed rate:
but accumulate those samples in an internal FIFO.
A watermark interrupt may then trigger a bounded SPI transaction:
Internal Sampling
↓
Sensor FIFO
↓
FIFO Watermark
↓
GPIO IRQ
↓
Driver Starts SPI Read
↓
Finite DMA Transfer
The acquisition is continuous at the sensor level, but the DMA-facing transport consists of repeated finite transactions.
This may naturally fit repeated one-shot or scatter-gather DMA.
By contrast, a peripheral that can continuously generate DMA requests after one startup operation may fit cyclic DMA more naturally:
A useful design question is:
After DMA is started, can the hardware continue generating the required DMA requests without software initiating every block?
The answer helps determine whether cyclic DMA or repeated finite transactions better match the hardware.
Request-Based vs Streaming DMA¶
The main architectural differences are:
| Request-Based DMA | Streaming DMA |
|---|---|
| One transfer belongs to one logical request | One stream spans many data blocks |
| Completion normally ends the transaction | Completion reports another region becoming ready |
| Mapping may follow request lifetime | Mapping may remain persistent |
| One-shot or SG transfer is common | Cyclic or block-based transfer is common |
| Cleanup occurs after request completion | Cleanup occurs when the stream stops |
| Consumer handles a finite result | Consumer repeatedly processes produced data |
Both models may use the same DMA Engine framework, but their lifetime and buffering strategies are different.
Continuous DMA Buffer Architecture¶
A continuous RX DMA client may divide one persistent mapping into reusable regions.
Conceptually:
Persistent DMA Buffer
┌────────┬────────┬────────┬────────┐
│ Slot 0 │ Slot 1 │ Slot 2 │ Slot 3 │
└────────┴────────┴────────┴────────┘
│
▼
completed-region metadata
│
▼
CPU Consumer
The word slot is used here as an architectural term for a reusable buffer region.
For cyclic DMA, each slot may correspond to one DMA period.
The important point is that the payload does not necessarily need to be copied into another software buffer merely to build a producer-consumer queue.
The software queue may instead track which DMA-buffer regions are available.
DMA Buffer Slot Ownership¶
A continuous DMA client may track each reusable buffer region independently from the lifetime of the complete DMA mapping.
The diagram shows three related but distinct concepts:
- the current DMA producer position
- the ready-ring producer and consumer positions
- the ownership state of each reusable buffer slot
A useful streaming ownership model is:
FREE¶
The region may be reused by DMA.
DMA_OWNED¶
The DMA producer may currently modify the region.
READY¶
DMA completed the region and the data is waiting for a consumer.
CPU_OWNED¶
The consumer acquired the completed region and may still be reading or processing it.
The critical reuse condition is:
This ownership model is a client-side architectural model.
Exact subsystem and hardware implementations may represent these states differently.
Producer and Consumer Positions¶
A streaming ring commonly requires separate producer and consumer positions.
Conceptually:
Useful metadata may include:
| Metadata | Purpose |
|---|---|
| Producer position | Identifies where the next completion is published |
| Consumer position | Identifies the oldest completed region waiting for consumption |
| Ready count | Identifies how many completed regions are waiting |
| DMA position | Identifies the region currently being used by DMA |
These values may move together during simple operation, but they describe different responsibilities.
A DMA hardware position should not automatically be treated as equivalent to the software consumer position.
Consumer Lag¶
A consumer does not need to process every completed region immediately.
For example:
DMA:
Slot 0 complete
Slot 1 complete
Slot 2 complete
CPU:
consume Slot 0
consume Slot 1
consume Slot 2
Several completed regions may temporarily wait in the ring.
This is normal buffering behavior.
The buffer provides elasticity for:
- scheduling latency
- temporary CPU load
- userspace wake-up latency
- short processing bursts
Therefore:
as long as reusable storage remains available.
Streaming Overrun¶
An overrun occurs when the producer needs to reuse storage that has not yet been released by the consumer.
For cyclic DMA:
Slot 0 → READY
Slot 1 → READY
Slot 2 → READY
Slot 3 → READY
│
▼
DMA wraps to Slot 0
│
▼
Slot 0 still not FREE
│
▼
OVERRUN
The fundamental condition is:
Possible policies include:
- stop the stream
- overwrite old data
- drop new data
- apply backpressure
- report data loss and continue
The correct policy depends on the peripheral and application semantics.
A client driver must not silently assume that a finite buffer can absorb an indefinitely slower consumer.
Buffering Does Not Fix Throughput Mismatch¶
Buffering can absorb temporary timing differences.
For example:
A temporary scheduling pause may create a backlog that the consumer later drains.
However, if:
for a sufficiently long period, every finite buffer eventually fills.
A larger buffer only delays the failure.
Therefore streaming design must consider:
- DMA block or period size
- number of reusable regions
- completion frequency
- kernel processing latency
- userspace scheduling latency
- userspace read rate
- overflow policy
- acceptable data loss
DMA Ring vs Additional Software Buffer¶
A software ring does not necessarily imply an additional payload copy.
Two architectures should be distinguished.
Copy-Based Buffering¶
This can simplify lifetime management but adds CPU and memory-bandwidth cost.
Metadata-Based Buffering¶
The queue contains metadata rather than a second copy of the payload.
This can reduce copying, but ownership and reuse rules become more important.
The correct architecture depends on:
- throughput
- latency
- subsystem requirements
- userspace interface
- DMA controller capabilities
- buffer lifetime requirements
Zero-copy is not automatically better if the additional ownership complexity is unnecessary.
DMA and Subsystem Buffers¶
DMA and subsystem buffering solve different problems.
A continuous acquisition path may look like:
Peripheral
↓
DMA
↓
DMA-accessible kernel memory
↓
Driver / subsystem buffering
↓
Userspace interface
↓
Userspace consumer
DMA solves:
A subsystem buffer solves:
These mechanisms can therefore be used together.
For an IIO-style acquisition path:
The exact integration depends on the IIO driver and buffer backend.
The presence of multiple buffering abstractions does not by itself prove that the payload is copied between every layer.
DMA Callback Responsibilities¶
A DMA completion callback should generally remain short.
A common pattern is:
DMA Completion
↓
Callback
│
├── record completion
├── update metadata
├── advance producer state
└── wake / schedule consumer
↓
Return
Large payload processing should normally be deferred to a suitable context.
If callback-side metadata is shared with another execution context, synchronization must match the callback context.
Possible mechanisms include:
- spinlocks
- atomic operations for suitable independent state
- lockless producer-consumer designs when their assumptions are satisfied
A mutex is not appropriate in a context where sleeping is prohibited.
Atomic variables also do not automatically make a multi-field state transition consistent.
Streaming Termination¶
Stopping DMA and ending the consumer lifetime are separate events.
Conceptually:
At this point, completed data may still exist.
For example:
DMA termination means the device no longer accesses the buffer.
It does not necessarily mean that the CPU consumer has finished with every completed region.
Therefore:
A teardown path may need to:
- stop new producer activity
- terminate and synchronize DMA
- drain or discard completed data according to policy
- wait for or revoke outstanding consumers when required
- release the mapping only when it is safe
Error and Timeout Cleanup¶
The error path must preserve DMA ownership rules.
A typical request-based sequence is:
Error / Timeout
↓
Stop Peripheral
↓
Terminate DMA
↓
Synchronize DMA Completion Activity
↓
DMA Safe
↓
Unmap
↓
Release Buffer
The critical rule is:
For streaming clients, an additional rule applies:
A synchronous termination path such as:
provides a useful DMA-side teardown boundary when appropriate for the client context.
Consumer-side lifetime may still require separate synchronization.
Normal and Error Paths Must Converge¶
A useful implementation structure is to make both paths reach a common DMA-safe cleanup point:
ACTIVE
/ \
/ \
Completion Error
│ │
│ Stop Peripheral
│ │
│ Terminate / Sync
│ │
└───────┬───────┘
▼
DMA SAFE
↓
Consumer Safe
↓
Unmap
↓
Release Buffer
For one-shot transfers, DMA SAFE and Consumer Safe may occur close together.
For streaming transfers, they may be separated by outstanding completed buffers.
Client-Side One-Shot Pattern¶
A simplified one-shot RX sequence can be expressed as:
/* Driver lifetime setup. */
chan = dma_request_chan(dev, "rx");
dmaengine_slave_config(chan, &config);
/* Per-transfer setup. */
dma_addr = dma_map_single(dev,
buffer,
length,
DMA_FROM_DEVICE);
desc = dmaengine_prep_slave_single(chan,
dma_addr,
length,
DMA_DEV_TO_MEM,
flags);
desc->callback = rx_complete;
desc->callback_param = context;
cookie = dmaengine_submit(desc);
dma_async_issue_pending(chan);
/* Start the peripheral only after the DMA path is ready. */
start_rx_peripheral();
The completion and cleanup path conceptually follows:
DMA callback
↓
record completion / wake consumer
↓
perform required CPU ownership transition
↓
unmap when mapping is no longer needed
↓
consume or deliver received data
The actual callback context and synchronization mechanism depend on the driver architecture.
One-Shot DMA Simulation¶
Day109 implements a simplified one-shot RX DMA client simulator.
The simulated states are:
Three cases are tested:
| Case | Purpose |
|---|---|
| Normal completion | Verify the complete RX lifecycle and CPU data visibility |
| Abort while active | Verify synchronized error cleanup |
| Invalid cleanup | Verify that unmapping is rejected while DMA may still access the mapping |
See:
Day109 — One-Shot DMA Client Lifecycle
Continuous DMA Streaming Simulation¶
Day110 extends the client simulator into a continuous RX architecture.
The complete DMA buffer remains persistently mapped and is divided into reusable slots.
The simplified ownership lifecycle is:
The simulator separates:
DMA producer position
+
ready-ring producer position
+
ready-ring consumer position
+
per-slot ownership
Three cases are verified:
| Case | Purpose |
|---|---|
| Normal producer / consumer | Verify immediate completion, acquire, synchronization, and release |
| Consumer lag | Verify several completed slots may queue without overrun |
| Cyclic DMA overrun | Verify DMA cannot safely reuse a slot still owned by the consumer side |
The Day110 simulator also extends the lab DMA-mapping simulator with subrange synchronization support so that individual slots inside one persistent mapping can be synchronized independently.
These range-sync helpers are simulator-specific APIs and are not Linux kernel DMA API names.
See:
Day110 — Continuous DMA Streaming and Ring Buffer Ownership
Mental Model¶
A DMA client driver coordinates several independent state machines:
Peripheral State
+
DMA Transaction / Stream State
+
DMA Mapping Lifetime
+
Buffer Region Ownership
+
Producer / Consumer State
+
Peripheral Protocol State
A correct implementation does not assume that one state transition automatically completes all the others.
For a one-shot transfer:
DMA COMPLETED
│
├── DMA transaction finished
├── mapping may still exist
├── CPU ownership may still require synchronization
└── peripheral may still have protocol-level work
For a streaming transfer:
DMA BLOCK COMPLETED
│
├── stream is still running
├── mapping remains active
├── completed region becomes available
├── consumer may acquire that region
└── DMA continues toward another region
This is the central DMA client architecture model.
Related Topics¶
- Linux DMA Engine Framework
- DMA Controller and Channel
- DMA Descriptor
- DMA Completion and Callback
- DMA Termination and Synchronization
- Cyclic DMA
- DMA Slave Configuration
- DMA Slave Transfer
- DMA Scatter-Gather Transfer
Related API Reference¶
Related Labs¶
Summary¶
A DMA client is responsible for more than calling DMA Engine APIs.
It must coordinate:
DMA Channel
↓
DMA Mapping
↓
Descriptor / Stream
↓
Submission
↓
Issue
↓
Peripheral Start
↓
DMA Completion
↓
Buffer Ownership
↓
Consumer
↓
Peripheral Completion
↓
Cleanup
The most important rules are:
- Configure and arm DMA before allowing the peripheral to generate data when the hardware requires that ordering.
- Do not equate
dma_async_issue_pending()with active peripheral data movement. - Do not equate DMA completion with peripheral protocol completion.
- Do not equate DMA completion with mapping destruction.
- Respect CPU/device ownership for streaming mappings.
- Treat persistent mapping lifetime and individual buffer-region ownership as separate concepts.
- Distinguish DMA producer position from software consumer position.
- Consumer lag is normal until the producer exhausts reusable storage.
- Detect and define an explicit policy for streaming overrun.
- Terminate and synchronize DMA before unsafe teardown.
- Do not invalidate storage while DMA or a consumer may still access it.
- Treat request-based and streaming DMA as different lifetime architectures built on the same DMA Engine framework.