Day65 - Linux Kernel RCU APIs and Usage¶

Overview¶

This lab explores how Linux Kernel RCU APIs are used to safely publish new objects and reclaim old objects without blocking readers.

Two experiments are implemented:

RCU API Usage Simulation
call_rcu() Deferred Reclamation Simulation

The implementation is a simplified userspace model built with C11 atomics and pthreads.

Lab1 - RCU API Usage Simulation¶

Objective¶

Understand the semantics of:

rcu_read_lock()
rcu_read_unlock()

rcu_dereference()
rcu_assign_pointer()

synchronize_rcu()

and verify the RCU publication and reclamation workflow.

Data Structure¶

struct rcu_object {
    int version;
    int value;
};

Global pointer:

static _Atomic(struct rcu_object *) g_rcu_ptr;

Reader tracking:

static atomic_int g_reader_count;

Simulated RCU APIs¶

rcu_read_lock()¶

static void rcu_read_lock(void)
{
    atomic_fetch_add_explicit(
        &g_reader_count,
        1,
        memory_order_acquire);
}

rcu_read_unlock()¶

static void rcu_read_unlock(void)
{
    atomic_fetch_sub_explicit(
        &g_reader_count,
        1,
        memory_order_release);
}

rcu_dereference()¶

static struct rcu_object *rcu_dereference(void)
{
    return atomic_load_explicit(
        &g_rcu_ptr,
        memory_order_acquire);
}

rcu_assign_pointer()¶

static void rcu_assign_pointer(struct rcu_object *obj)
{
    atomic_store_explicit(
        &g_rcu_ptr,
        obj,
        memory_order_release);
}

synchronize_rcu()¶

static void synchronize_rcu(void)
{
    while (atomic_load_explicit(
               &g_reader_count,
               memory_order_acquire) != 0) {
        sched_yield();
    }
}

Reader Flow¶

rcu_read_lock()
        ↓
rcu_dereference()
        ↓
access object
        ↓
rcu_read_unlock()

Readers keep using the object even after a writer publishes a newer version.

Writer Flow¶

allocate new object
        ↓
initialize object
        ↓
rcu_assign_pointer()
        ↓
synchronize_rcu()
        ↓
free old object

Expected Behavior¶

publish new object
        ↓
old readers continue using old object
        ↓
grace period completes
        ↓
old object reclaimed

Lab2 - call_rcu() Deferred Reclamation¶

Objective¶

Understand asynchronous object reclamation using:

call_rcu()

instead of:

synchronize_rcu()

Data Structure¶

rcu_head¶

struct rcu_head {
    struct rcu_head *next;
    void (*func)(struct rcu_head *head);
};

Reclaimable Object¶

struct rcu_object {
    int version;
    int value;

    struct rcu_head rcu;
};

Callback Queue¶

The lab implements a simple callback queue:

head
 │
 ▼
rcu_head
    ↓
rcu_head
    ↓
rcu_head
    ↓
NULL

Each queued node represents a pending callback request.

call_rcu()¶

call_rcu(&old->rcu,
         rcu_object_free_cb);

The writer does not wait for a grace period.

Instead:

enqueue callback
        ↓
return immediately

Reclaimer Thread¶

The reclaimer thread performs:

wait grace period
        ↓
dequeue callback
        ↓
execute callback
        ↓
free object

Pseudo flow:

for (;;) {

    synchronize_rcu();

    head = rcu_queue_pop();

    if (!head)
        break;

    head->func(head);
}

Callback Function¶

static void rcu_object_free_cb(
    struct rcu_head *head)
{
    struct rcu_object *obj;

    obj = container_of(
              head,
              struct rcu_object,
              rcu);

    free(obj);
}

Expected Behavior¶

Writer:

publish new object
        ↓
call_rcu()
        ↓
exit

Reclaimer:

wait grace period
        ↓
execute callback
        ↓
free old object

Results¶

Lab1¶

Verified:

Publication
    ↓
Grace Period
    ↓
Reclamation

workflow.

Readers safely continued using old objects until the grace period completed.

Lab2¶

Verified:

Publication
    ↓
Deferred Reclamation
    ↓
Callback Execution

workflow.

Writers no longer blocked while waiting for readers.

Object reclamation was performed asynchronously by the reclaimer thread.

Key Takeaways¶

RCU Reader Side¶

rcu_read_lock()
rcu_dereference()
rcu_read_unlock()

allows lockless access to published objects.

RCU Writer Side¶

rcu_assign_pointer()

safely publishes a new object.

Grace Period¶

synchronize_rcu()

blocks until all current readers leave their read-side critical sections.

Deferred Reclamation¶

call_rcu()

moves reclamation into a callback queue and allows writers to continue immediately.

Publication vs Reclamation¶

The most important RCU design principle is:

Object Publication
and
Object Reclamation
are separate operations.

A new object can be published immediately, while the old object must remain alive until the grace period completes.