Day97 - Memory Barriers¶

Objective¶

In this lab, we explore why modern CPUs and compilers may reorder memory accesses, and how Linux memory barriers enforce the required ordering between CPUs, shared memory, MMIO registers, and DMA devices.

Instead of emulating a real CPU pipeline, this lab implements a simple Memory Ordering Simulator that visualizes the difference between Program Order and Visible Memory Order.

Learning Objectives¶

After completing this lab, you should understand:

The difference between program order and visible memory order
Why compiler and CPU reordering occurs
How memory barriers preserve ordering
How Release ordering protects shared-memory publication
Why DMA drivers require write barriers before notifying hardware

Simulator Design¶

The simulator models memory ordering using a deterministic operation queue.

It visualizes:

Program Order
Visible Memory Order
Ordering constraints introduced by memory barriers

It does not emulate:

CPU pipelines
Store buffers
Cache coherency
Out-of-order execution
Hardware buses

Instead, it demonstrates ordering behavior at a conceptual level.

Conceptually:

Program Order

↓

Memory Barrier

↓

Visible Memory Order

Operation Types¶

The simulator supports the following operations:

Operation	Description
STORE	Store to normal shared memory
LOAD	Load from normal shared memory
MMIO_WRITE	Write to a device register
BARRIER	Ordering boundary

Supported barrier types:

Compiler
Read
Write
Full
Release
Acquire

Lab 1 - Store Reordering Without Barrier¶

Program Order:

STORE data
STORE ready

Without any ordering constraint, the simulator allows:

STORE ready
STORE data

This models the classic producer-consumer publication problem.

Expected result:

PASS when ready becomes visible before data.

Lab 2 - Store Reordering With Release Barrier¶

Program Order:

STORE data

↓

Release Barrier

↓

STORE ready

The Release barrier prevents the publication flag from becoming visible before the shared data.

Expected result:

PASS when data remains visible before ready.

Lab 3 - DMA Doorbell Without Write Barrier¶

Program Order:

STORE descriptor

↓

MMIO_WRITE doorbell

Without a write barrier, the simulator allows the doorbell notification to become visible before the descriptor update.

This models a DMA engine receiving a notification before the descriptor is fully prepared.

Expected result:

PASS when doorbell becomes visible before descriptor.

Lab 4 - DMA Doorbell With Write Barrier¶

Program Order:

STORE descriptor

↓

Write Barrier

↓

MMIO_WRITE doorbell

The write barrier guarantees that the descriptor becomes visible before the hardware notification.

Expected result:

PASS when descriptor remains visible before doorbell.

Summary¶

This simulator demonstrates four common memory-ordering scenarios found in Linux kernel development:

Publication without ordering
Publication protected by Release ordering
DMA notification without a write barrier
DMA notification protected by a write barrier

Rather than simulating CPU microarchitecture, the simulator focuses on the observable effects of memory ordering, making it easier to understand how Linux memory barrier APIs preserve correctness.