fiber.h File Reference

AI Generated (🤦) Fiber subsystem overview. More...

#include <stddef.h>
#include <stdint.h>

Include dependency graph for fiber.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Data Structures
struct	srn_fiber_ctx_t
	The saved context of a suspended fiber is a single word: its stack pointer at the moment it was switched away from. More...
struct	srn_fiber_stack_t
	One stack per fiber, mapped with a guard page at the low end so an overflow faults deterministically instead of corrupting a neighbour. More...
struct	srn_fiber_t

Macros
#define	SRN_ASAN   0
#define	SRN_TSAN   0
#define	srn_fiber_get_scheduler_m(fiber)
#define	SRN_FIBER_DEFAULT_STACK_SIZE   (128U * 1024U)
	Default size of every fiber stack.

Typedefs
typedef struct srn_fiber_t	srn_fiber_t
typedef void *	srn_fiber_result_t
typedef struct srn_fiber_ctx_t	srn_fiber_ctx_t
	The saved context of a suspended fiber is a single word: its stack pointer at the moment it was switched away from.
typedef struct srn_fiber_stack_t	srn_fiber_stack_t
	One stack per fiber, mapped with a guard page at the low end so an overflow faults deterministically instead of corrupting a neighbour.
typedef enum srn_fiber_state_e	srn_fiber_state_t
typedef srn_fiber_result_t(*	srn_fiber_entry_t) (srn_context_t *ctx, void *arg)
	The function a fiber runs.
typedef bool(*	srn_fiber_park_fn) (srn_fiber_t *self, void *arg)
	Suspend commit callback.

Enumerations
enum	srn_fiber_state_e : uint8_t {   SRN_FIBER_NEW = 0 , SRN_FIBER_READY , SRN_FIBER_RUNNING , SRN_FIBER_SUSPENDED ,   SRN_FIBER_DONE }

Functions
srn_scheduler_t *	srn_sched_init (srn_engine_t *engine)
void	srn_sched_shutdown (srn_scheduler_t *sched)
	The one stop tear down of the fiber subsystem, should be called once srn_sched_run has returned.
void	srn_sched_run (srn_scheduler_t *sched, int nworkers)
	Run the scheduler with nworkers os threads draining it, returning once the pool goes quiescent (every os thread parked on an empty queue) or a stop is requested with srn_sched_stop.
void	srn_sched_stop (srn_scheduler_t *sched)
	Ask a running scheduler to stop.
void	srn_sched_register (srn_scheduler_t *sched, srn_fiber_t *fiber)
	Record a fiber in the scheduler's registry of live fibers, where it stays until it is reaped.
void	srn_sched_enqueue (srn_scheduler_t *sched, srn_fiber_t *fiber)
	Place a fiber on a scheduler's ready queue, making it eligible to run.
srn_fiber_t *	srn_fiber_make (srn_context_t *ctx, srn_scheduler_t *sched, srn_fiber_entry_t entry, void *arg, size_t stack_size)
	Create a fiber that will run entry(ctx, arg).
void	srn_fiber_yield (void)
	Yield cooperatively: re-enqueue the running fiber and run the next ready one.
void	srn_fiber_suspend (srn_fiber_park_fn commit, void *arg)
	Park the running fiber until a party calls srn_fiber_ready.
void	srn_fiber_ready (srn_fiber_t *fiber)
	Mark a suspended fiber runnable again.
srn_fiber_result_t	srn_fiber_wait_for (srn_fiber_t *target)
	Block the calling fiber until target finishes, then return its result.
srn_fiber_t *	srn_fiber_current (void)
	The fiber currently running on this os thread (the bootstrap fiber if none).
srn_fiber_t *	srn_fiber_worker_loop (void)
	The worker's loop of the worker running on the calling os thread.
void	srn_fiber_swap (srn_fiber_ctx_t *from, srn_fiber_ctx_t *to)
	Save the current execution context into from, restore to, and resume on to's stack.
void	srn_fiber_ctx_make (srn_fiber_ctx_t *fiber_ctx, srn_fiber_stack_t stack, void(*fn)(void *), void *arg)
	Initialise a fresh fiber context so the first srn_fiber_swap into it begins executing fn(arg) on stack.
void	srn_fiber_switch (srn_fiber_t *from, srn_fiber_t *to)
	Switch from from to to, both live fibers, telling AddressSanitizer about the stack change.
void	srn_fiber_switch_final (srn_fiber_t *to)
	Like srn_fiber_switch, but for a fiber that has finished and must not be resumed: control transfers to to and never comes back.
void	srn_fiber_on_entry (srn_fiber_t *from)
	Call as the first action inside a fresh fiber's entry.
void	srn_fiber_on_reap (srn_fiber_t *fiber)
	Call when a finished fiber is reaped, after it has switched away for the last time.
void	srn_fiber_init_thread (srn_fiber_t *f)
	Represent the calling OS thread as the running fiber ("#0"), so the scheduler or a test can switch away from it and back.
srn_fiber_stack_t	srn_fiber_stack_alloc (size_t size)
	Allocate a stack of at least size usable bytes plus a guard page, or SRN_FIBER_DEFAULT_STACK_SIZE when size is 0.
void	srn_fiber_stack_free (srn_fiber_stack_t stack)
static size_t	srn_fiber_stack_size (srn_fiber_stack_t s)

Detailed Description

AI Generated (🤦) Fiber subsystem overview.

Stackful, cooperative fibers for the runtime. A fiber is a function plus its own stack, so it can pause at any call depth and resume later. Nothing is preempted. A fiber runs until it explicitly yields, suspends, or finishes.

Terminology (used throughout the fiber subsystem) fiber a unit of work with its own stack. It can pause and resume. os thread a thread the operating system schedules. worker the state one os thread uses to run fibers. It holds a local queue of ready fibers and a worker's loop. There is one worker per os thread. worker's loop a special fiber that stands in for the os thread. A running fiber switches back to it to hand control over, and it picks the next fiber to run. worker routine the steps an os thread repeats. It takes a fiber, switches into it, deals with how the fiber gave up control, and parks when no fiber is runnable. scheduler there is exactly one. It owns the shared ready queue and the list of live fibers and decides what runs next. It does not run fibers itself, workers do. suspend a fiber steps off every queue and waits to be woken. Its os thread stays free to run other fibers. park an os thread blocks because no fiber is runnable anywhere. A notify wakes it.

Every fiber stack is the same fixed size (SRN_FIBER_DEFAULT_STACK_SIZE), one mapping with a guard page that never grows. Uniform size is a deliberate constraint, not an accident. It lets a finished fiber's stack be handed to any later fiber (the stack ring TODO below) without tracking size classes, and it keeps switch and mapping costs predictable.

Control flow is a hub. A fiber never jumps to another fiber. It switches back to its worker's loop, which picks the next one.

scheduler ready queue:  [F2] -> [F5] -> ...
                           |  the os thread takes the head
                           v
   +----------->  worker's loop  --- switch --->  running fiber
   |                                                  |
   |  yield    (re-enqueue, run next) ----------------+
   |  suspend  (off-queue until srn_fiber_ready) -----+
   |  finish   (reap) --------------------------------+
   +--------------------------------------------------+

A running fiber gives up control three ways:

• yield still runnable, so it moves to the back of the ready queue.
• suspend off every queue until some party calls srn_fiber_ready. The wake seam used by peer fibers, locks, and a future IO reactor.
• finish its entry returns and the fiber is reaped.

The spec book's fibers chapter (docs/spec/fibers.typ) is the long-form reference. Keep this overview in step with it.

Definition in file fiber.h.

Macro Definition Documentation

SRN_ASAN

#define SRN_ASAN   0

Definition at line 94 of file fiber.h.

SRN_FIBER_DEFAULT_STACK_SIZE

#define SRN_FIBER_DEFAULT_STACK_SIZE   (128U * 1024U)

Default size of every fiber stack.

All fiber stacks share one size (see the subsystem overview), so this is a process-wide value, not a per-fiber one: the default here, eventually overridable through a CLI argument.

A fiber stack is fixed and cannot grow (see the fibers chapter), so the size must hold realistic call chains yet stay cheap to reserve in bulk. 128 KiB strikes that balance: deep enough for the runtime's C and JIT frames, while lazy page commit means a shallow fiber faults in only the one or two pages it touches and leaves the rest a virtual reservation. For scale, an OS thread stack is far too large to spawn fibers by the thousand, and a few KiB would overflow on modest nesting.

Definition at line 138 of file fiber.h.

srn_fiber_get_scheduler_m

#define srn_fiber_get_scheduler_m

(

fiber

)

Value:

fiber->ctx->engine->scheduler

Definition at line 119 of file fiber.h.

SRN_TSAN

#define SRN_TSAN   0

Definition at line 105 of file fiber.h.

Typedef Documentation

srn_fiber_ctx_t

typedef struct srn_fiber_ctx_t srn_fiber_ctx_t

The saved context of a suspended fiber is a single word: its stack pointer at the moment it was switched away from.

The callee-saved registers live on the fiber's own stack, pushed by srn_fiber_swap and popped when it is resumed.

srn_fiber_entry_t

typedef srn_fiber_result_t(* srn_fiber_entry_t) (srn_context_t *ctx, void *arg)

The function a fiber runs.

Allocations made through ctx land in the context's block. TODO(lxsameer): The shape is generic until code generation produces fibers, at which point it adopts the Serene ABI. TODO(lxsameer): Should we support two types of entry functions? one for C calling convention and one for FastC?

Definition at line 199 of file fiber.h.

srn_fiber_park_fn

typedef bool(* srn_fiber_park_fn) (srn_fiber_t *self, void *arg)

Suspend commit callback.

The worker routine runs it on its own side once the fiber has switched out. It registers self with whatever will wake it and re-checks the awaited condition. It returns true to stay suspended, or false to resume self at once because the condition already holds. It must not switch fibers.

Definition at line 206 of file fiber.h.

srn_fiber_result_t

typedef void* srn_fiber_result_t

Definition at line 117 of file fiber.h.

srn_fiber_stack_t

typedef struct srn_fiber_stack_t srn_fiber_stack_t

One stack per fiber, mapped with a guard page at the low end so an overflow faults deterministically instead of corrupting a neighbour.

the layout is like:

|... frames ...|... guard page ...| ^ start ^ limit ^ guard

It is a bit unintuitive to see the guard pointing at the end of the the region, but conventionally, the guard is an OS page and since the stack grows downward and allocations normally grow upward, guard is actually the address that OS (POSIX in this example) will return to us as the starting point of the memory region. Or to put it in different terms, we store the stack frames, starting from the end of the allocated memory for the region and move toward the start of the allocated memory.

srn_fiber_state_t

typedef enum srn_fiber_state_e srn_fiber_state_t

srn_fiber_t

typedef struct srn_fiber_t srn_fiber_t

Definition at line 110 of file fiber.h.

Enumeration Type Documentation

srn_fiber_state_e

enum srn_fiber_state_e : uint8_t

Enumerator
SRN_FIBER_NEW	Created, stack mapped, never resumed.
SRN_FIBER_READY	On the run queue, eligible to run.
SRN_FIBER_RUNNING	Currently executing.
SRN_FIBER_SUSPENDED	Parked off the run queue, awaits srn_fiber_ready.
SRN_FIBER_DONE	Entry returned. The result is final.

Definition at line 180 of file fiber.h.

                               : uint8_t {
  /// Created, stack mapped, never resumed
  SRN_FIBER_NEW = 0,
  /// On the run queue, eligible to run
  SRN_FIBER_READY,
  /// Currently executing
  SRN_FIBER_RUNNING,
  /// Parked off the run queue, awaits srn_fiber_ready
  SRN_FIBER_SUSPENDED,
  /// Entry returned. The result is final
  SRN_FIBER_DONE,
} srn_fiber_state_t;

Function Documentation

srn_fiber_ctx_make()

void srn_fiber_ctx_make	(	srn_fiber_ctx_t *	fiber_ctx,
		srn_fiber_stack_t	stack,
		void(*	fn )(void *),
		void *	arg )

Initialise a fresh fiber context so the first srn_fiber_swap into it begins executing fn(arg) on stack.

fn is the fiber's entry: a void (*)(void *) that runs on the new stack and receives arg. It must NEVER return – there is no frame beneath it, so a return traps in the trampoline. Instead it transfers control away with srn_fiber_switch (to yield) or srn_fiber_switch_final (when finished), and as its first action it calls srn_fiber_on_entry(). Typical shape:

static void worker(void *arg) { srn_fiber_on_entry(&scheduler); // hand the switch to the sanitizer ... do the fiber's work using arg ... srn_fiber_switch_final(&scheduler); // finished, never returns }

Definition at line 29 of file ctx_x86_64.c.

                                   {
  // Lay srn_fiber_swap's frame (see rt/fiber/stack.h and switch_x86_64.S) at
  // the top of `stack`, rounded down to SRN_FIBER_STACK_ALIGN so the
  // return-address slot -- and thus the trampoline's entry rsp -- is correctly
  // aligned. The compound literal zero-fills the slots not named here. Register
  // slots are uintptr_t, so storing fn/trampoline is a
  // function-pointer-to-integer conversion (allowed), not the
  // function-to-object-pointer conversion that -Wpedantic forbids.
  uintptr_t top = (uintptr_t)stack.start;
  uintptr_t sp = (top - sizeof(srn_fiber_frame_t)) & ~(uintptr_t)(SRN_FIBER_STACK_ALIGN - 1);
 
  *(srn_fiber_frame_t *)sp = (srn_fiber_frame_t){
      .mxcsr = SRN_FIBER_MXCSR_DEFAULT,
      .fcw = SRN_FIBER_FCW_DEFAULT,
      .r13 = (uintptr_t)fn,
      .r12 = (uintptr_t)arg,
      .ret_addr = (uintptr_t)srn_fiber_trampoline,
  };
 
  fiber_ctx->sp = (void *)sp;
}

Here is the call graph for this function:

Here is the caller graph for this function:

srn_fiber_current()

srn_fiber_t * srn_fiber_current

(

void

)

The fiber currently running on this os thread (the bootstrap fiber if none).

Definition at line 930 of file scheduler.c.

                                     {
  return current_worker != nullptr ? current_worker->current : nullptr;
}

Here is the caller graph for this function:

srn_fiber_init_thread()

void srn_fiber_init_thread

(

srn_fiber_t *

f

)

Represent the calling OS thread as the running fiber ("#0"), so the scheduler or a test can switch away from it and back.

The thread's stack bounds are not queried here (there is no portable way). Under the sanitizer they are recorded by the first fiber's srn_fiber_on_entry. The saved context is written by the first switch away.

Definition at line 137 of file fiber.c.

                                           {
  // Represent the calling thread as the running fiber. Its stack bounds are not
  // queried here (there is no portable way). Under the sanitizer the first
  // fiber's srn_fiber_on_entry records them. The saved context (fiber_ctx)
  // stays empty, and the first switch away from the thread fills it.
  memset(f, 0, sizeof(*f));
  f->state = SRN_FIBER_RUNNING;
  f->name = "thread";
#if SRN_TSAN
  // The loop fiber stands in for this OS thread, so it takes the thread's own
  // current TSan fiber rather than a freshly created one.
  f->tsan_fiber = __tsan_get_current_fiber();
#endif
}

Here is the caller graph for this function:

srn_fiber_make()

srn_fiber_t * srn_fiber_make ( srn_context_t * ctx, srn_scheduler_t * sched, srn_fiber_entry_t entry, void * arg, size_t stack_size )

nodiscard

Create a fiber that will run entry(ctx, arg).

The fiber allocates into its block and never releases it. A stack_size of 0 selects SRN_FIBER_DEFAULT_STACK_SIZE.

Definition at line 169 of file fiber.c.

                                                          {
 
  srn_fiber_t *f = ALLOC(ctx, srn_fiber_t);
  memset((void *)f, 0, sizeof(srn_fiber_t));
 
  // TODO(lxsameer): Make the fiber stack configurable via cli arg or something.
  // This is the acquire side of the stack-ring TODO in fiber.h: a pooled stack
  // would be pulled from the per-thread ring here, falling back to a fresh
  // mapping only on a miss.
  f->stack = srn_fiber_stack_alloc(stack_size);
  f->ctx = ctx;
  f->state = SRN_FIBER_NEW;
  f->entry = entry;
  f->arg = arg;
#if SRN_TSAN
  f->tsan_fiber = __tsan_create_fiber(0);
#endif
  srn_fiber_ctx_make(&f->fiber_ctx, f->stack, &srn_fiber_launcher, (void *)f);
 
  // Register before enqueuing: the scheduler must know about the fiber for its
  // whole life, independent of which queue (if any) it currently sits on. The
  // registry is how a SUSPENDED fiber, off every run queue, stays reachable for
  // cleanup and cancellation.
  srn_sched_register(sched, f);
 
  // TODO(lxsameer): revisit whether make should auto-enqueue (spawn semantics)
  // or stay NEW and require an explicit srn_fiber_ready to start. For now
  // make == spawn: the fiber is runnable the moment it is created.
  srn_sched_enqueue(sched, f);
  return f;
}

Here is the call graph for this function:

Here is the caller graph for this function:

srn_fiber_on_entry()

void srn_fiber_on_entry

(

srn_fiber_t *

from

)

Call as the first action inside a fresh fiber's entry.

from is the fiber that started this one (the scheduler, or the bootstrap thread fiber). It completes the switch for AddressSanitizer and, since from's stack bounds only become discoverable here, records them into from so a later switch back is tracked. from may be null. A no-op when the sanitizer is not in use.

Definition at line 107 of file fiber.c.

                                                                      {
#if SRN_ASAN
  // A nullptr fake-stack: a fresh fiber has no previously saved bookkeeping.
  // The out-params report the stack this fiber was started from -- this is the
  // one chance to learn `from`'s bounds (there is no portable way to query a
  // thread's own stack), and a later switch back to `from` needs them, so
  // record them. They flow through the sanitizer rather than libc.
  const void *bottom = nullptr;
  size_t size = 0;
  __sanitizer_finish_switch_fiber(nullptr, &bottom, &size);
 
  if (from != nullptr) {
    // ASan reports the came-from stack as (bottom, size). The struct keeps the
    // low and high boundaries, so limit = bottom and start = bottom + size.
    from->stack.limit = (void *)bottom;
    from->stack.start = (char *)bottom + size;
  }
#else
  UNUSED(from);
#endif
}

Here is the caller graph for this function:

srn_fiber_on_reap()

void srn_fiber_on_reap

(

srn_fiber_t *

fiber

)

Call when a finished fiber is reaped, after it has switched away for the last time.

Releases the fiber's ThreadSanitizer handle. A no-op when the sanitizer is not in use.

Definition at line 129 of file fiber.c.

                                           {
#if SRN_TSAN
  __tsan_destroy_fiber(fiber->tsan_fiber);
#else
  UNUSED(fiber);
#endif
}

Here is the caller graph for this function:

srn_fiber_ready()

void srn_fiber_ready

(

srn_fiber_t *

fiber

)

Mark a suspended fiber runnable again.

This is the seam an IO reactor, timer, or peer fiber uses to wake a fiber when the event it awaited occurs.

Definition at line 918 of file scheduler.c.

                                         {
  PANIC_IF_NULL(fiber);
 
  // Wake a suspended fiber. The flip in `ready_fiber` lets exactly one of
  // several racing wakers enqueue it (an IO completion and a timeout firing on
  // it, say), while the rest find it no longer `SUSPENDED` and do nothing. The
  // scheduler is resolved from the fiber, not the calling os thread, so a waker
  // that is not itself running a worker (the IO reactor, a peer os thread) can
  // wake it.
  ready_fiber(srn_fiber_get_scheduler_m(fiber), fiber);
}

Here is the call graph for this function:

Here is the caller graph for this function:

srn_fiber_stack_alloc()

srn_fiber_stack_t srn_fiber_stack_alloc ( size_t size )

nodiscard

Allocate a stack of at least size usable bytes plus a guard page, or SRN_FIBER_DEFAULT_STACK_SIZE when size is 0.

Platform specific.

Definition at line 34 of file stack_posix.c.

                                                     {
  const size_t os_page_size = srn_mm_get_os_page_size();
  const size_t desired_size = size ? size : (size_t)SRN_FIBER_DEFAULT_STACK_SIZE;
 
  // Round upto a multiple of PAGE size
  const size_t usable = (desired_size + os_page_size - 1) & ~(os_page_size - 1);
  // Add a page to act as a guard page
  const size_t total = usable + os_page_size;
 
  // Since stack grows downward, the starting point where mmap returns will
  // be the end of the stack for us. We could of used MAP_GROWSDOWN to
  // change the behaviour, but that causes the stack to grow, but we have
  // a fix stack.
  void *end = mmap(nullptr, total, PROT_READ | PROT_WRITE, STACK_FLAGS,
                   // according to the man page, for anonymous mode fd should be -1
                   -1,
                   // according to the man page, for anonymous mode offset should be 0
                   0);
 
  PANIC_IF(end == MAP_FAILED, "Failed to allocate the stack for the fiber subsystem");
 
  if (mprotect(end, os_page_size, PROT_NONE) == -1) {
    munmap(end, total);
    PANIC("Failed to allocate the guard page for the fiber subsystem");
  }
 
  const uintptr_t start = (uintptr_t)end + (uintptr_t)total;
  const uintptr_t limit = (uintptr_t)end + (uintptr_t)os_page_size;
 
  srn_fiber_stack_t stack = {.start = (void *)start, .limit = (void *)limit, .guard = end};
  return stack;
}

Here is the call graph for this function:

Here is the caller graph for this function:

srn_fiber_stack_free()

void srn_fiber_stack_free

(

srn_fiber_stack_t

stack

)

Definition at line 67 of file stack_posix.c.

                                                   {
  munmap(stack.guard, (uintptr_t)stack.start - (uintptr_t)stack.guard);
}

Here is the caller graph for this function:

srn_fiber_stack_size()

static size_t srn_fiber_stack_size ( srn_fiber_stack_t s )

inlinestatic

Definition at line 437 of file fiber.h.

                                                               {
  return (size_t)((char *)s.start - (char *)s.limit);
}

Here is the caller graph for this function:

srn_fiber_suspend()

void srn_fiber_suspend	(	srn_fiber_park_fn	commit,
		void *	arg )

Park the running fiber until a party calls srn_fiber_ready.

The fiber switches out first. Only then does the scheduler run commit (see srn_fiber_park_fn) to register it and confirm it should stay parked. Registering only after the park completes is what makes it race free – a waker can never observe a half parked fiber. Acts on the fiber currently running on this thread.

Park the running fiber until a party calls srn_fiber_ready.

The commit callback runs on the worker's loop side once the fiber has switched out. It hands the fiber's pointer to the event source it blocks on (a peer fiber, a lock's waiter list, the IO reactor's fd table), so that party can call srn_fiber_ready when the awaited event occurs. Running commit only after the suspend completes is what makes the hand-off race free, a waker can never observe a half-suspended fiber. If commit registers the fiber nowhere, it is genuinely lost – a deadlock, like an os thread blocking on a condition nobody signals.

Definition at line 898 of file scheduler.c.

                                                            {
  PANIC_IF_NULL(commit);
 
  srn_worker_t *worker = current_worker;
  PANIC_IF_NULL(worker);
 
  srn_fiber_t *self = worker->current;
  PANIC_IF_NULL(self);
 
  // The fiber carries its own commit. The worker routine runs it after we
  // switch out -- the one safe point to publish a fully suspended fiber to its
  // waker. The routine also stamps the `SUSPENDED` state once the switch
  // completes, so the `state` never marks a fiber that is still suspending.
  // This call leaves the `state` as `RUNNING` and lets the switch carry the
  // fiber off the os thread.
  self->park_commit = commit;
  self->park_arg = arg;
  srn_fiber_switch(self, &worker->loop);
}

Here is the call graph for this function:

Here is the caller graph for this function:

srn_fiber_swap()

void srn_fiber_swap	(	srn_fiber_ctx_t *	from,
		srn_fiber_ctx_t *	to )

Save the current execution context into from, restore to, and resume on to's stack.

Returns, on from's stack, when a fiber later switches back into from.

Here is the caller graph for this function:

srn_fiber_switch()

void srn_fiber_switch	(	srn_fiber_t *	from,
		srn_fiber_t *	to )

Switch from from to to, both live fibers, telling AddressSanitizer about the stack change.

from resumes where it left off when something later switches back into it.

Switch from from to to, both live fibers, telling AddressSanitizer about the stack change.

Not instrumented by either sanitizer: the stack swaps mid-function, which confuses ASan's fake stack and TSan's shadow stack. The explicit annotations keep each sanitizer's fiber tracking correct across the swap instead.

Definition at line 62 of file fiber.c.

                                                                                                {
 
#if SRN_ASAN
  const size_t size = srn_fiber_stack_size(to->stack);
  __sanitizer_start_switch_fiber(&from->fake_stack, to->stack.limit, size);
#endif
#if SRN_TSAN
  // Move TSan's notion of the running fiber to `to` before the stack swaps. A
  // fiber built outside srn_fiber_make / srn_fiber_init_thread has no handle
  // and is simply not tracked. Only the low-level switch tests build such a
  // fiber. Every fiber the scheduler runs has one.
  if (to->tsan_fiber != nullptr) {
    __tsan_switch_to_fiber(to->tsan_fiber, 0);
  }
#endif
  srn_fiber_swap(&from->fiber_ctx, &to->fiber_ctx);
#if SRN_ASAN
  __sanitizer_finish_switch_fiber(from->fake_stack, nullptr, nullptr);
#endif
}

Here is the call graph for this function:

Here is the caller graph for this function:

srn_fiber_switch_final()

void srn_fiber_switch_final

(

srn_fiber_t *

to

)

Like srn_fiber_switch, but for a fiber that has finished and must not be resumed: control transfers to to and never comes back.

The fiber's entry function therefore ends at this call – any code after it is unreachable, which is why this is [[noreturn]]. Nothing is freed here. The spent fiber's stack is left frozen and reclaimed later by its owner (the scheduler reaps a finished fiber and frees its stack via srn_fiber_stack_free).

Definition at line 85 of file fiber.c.

                                        {
#if SRN_ASAN
  // A nullptr fake-stack tells ASan the current fiber is finished and will not
  // resume, so it can discard the bookkeeping rather than leak it.
  const size_t size = srn_fiber_stack_size(to->stack);
  __sanitizer_start_switch_fiber(nullptr, to->stack.limit, size);
#endif
#if SRN_TSAN
  // The finished fiber's handle is released later, at reap. Here just move
  // TSan to `to` before the swap, when `to` has a handle (see
  // srn_fiber_switch).
  if (to->tsan_fiber != nullptr) {
    __tsan_switch_to_fiber(to->tsan_fiber, 0);
  }
#endif
  // srn_fiber_swap always writes the outgoing sp somewhere. This fiber is done,
  // so discard it. Control loads `to` and never comes back.
  srn_fiber_ctx_t discard;
  srn_fiber_swap(&discard, &to->fiber_ctx);
  SHOULD_NOT_HAPPEN;
}

Here is the call graph for this function:

Here is the caller graph for this function:

srn_fiber_wait_for()

srn_fiber_result_t srn_fiber_wait_for

(

srn_fiber_t *

target

)

Block the calling fiber until target finishes, then return its result.

Definition at line 963 of file scheduler.c.

                                                           {
  PANIC_IF_NULL(target);
  PANIC_IF(target == srn_fiber_current(), "srn_fiber_wait_for: a fiber cannot wait for itself");
 
  // Suspend until the target finishes (wait_for_park registers us on its waiter
  // list). The target's DONE handling in the worker routine wakes us. The
  // result is read from the struct, which survives the target's reap.
  srn_fiber_suspend(wait_for_park, target);
  return target->result;
}

Here is the call graph for this function:

Here is the caller graph for this function:

srn_fiber_worker_loop()

srn_fiber_t * srn_fiber_worker_loop

(

void

)

The worker's loop of the worker running on the calling os thread.

The fiber that resumes when the current fiber yields, suspends, or finishes, the resumer a fiber's launcher hands control back to. Each worker has its own, so this is valid only on an os thread that is currently running the worker routine.

Definition at line 934 of file scheduler.c.

                                         {
  PANIC_IF_NULL(current_worker);
  return &current_worker->loop;
}

Here is the caller graph for this function:

srn_fiber_yield()

void srn_fiber_yield

(

void

)

Yield cooperatively: re-enqueue the running fiber and run the next ready one.

Acts on the fiber currently running on this thread.

Definition at line 876 of file scheduler.c.

                           {
  srn_worker_t *worker = current_worker;
  PANIC_IF_NULL(worker);
 
  // Switch to the worker's loop without enqueuing first. The worker routine
  // puts this fiber back on the ready queue once the switch has saved its
  // context. Enqueuing here, before the switch, would let another os thread
  // dequeue and resume the fiber while this os thread is still saving its
  // context -- two os threads on one fiber stack, which corrupts the switch.
  srn_fiber_t *self = worker->current;
  srn_fiber_switch(self, &worker->loop);
}

Here is the call graph for this function:

Here is the caller graph for this function:

srn_sched_enqueue()

void srn_sched_enqueue	(	srn_scheduler_t *	sched,
		srn_fiber_t *	fiber )

Place a fiber on a scheduler's ready queue, making it eligible to run.

Definition at line 593 of file scheduler.c.

                                                                   {
  PANIC_IF_NULL(sched);
  PANIC_IF_NULL(fiber);
 
  fiber->state = SRN_FIBER_READY;
  push_ready(sched, fiber);
}

Here is the call graph for this function:

Here is the caller graph for this function:

srn_sched_init()

srn_scheduler_t * srn_sched_init ( srn_engine_t * engine )

nodiscard

Definition at line 246 of file scheduler.c.

                                                      {
  PANIC_IF_NULL(engine);
  // The scheduler outlives every context and fiber, so it is allocated from the
  // immortal region rather than a releasable block.
  srn_scheduler_t *sched = srn_mm_immortal_allocate(engine->mm, srn_scheduler_t);
  PANIC_IF_NULL(sched);
 
  sched->engine = engine;
  sched->ready_head = nullptr;
  sched->ready_tail = nullptr;
  sched->registry = nullptr;
  sched->idle = 0;
  sched->runnable = 0;
  sched->nworkers = 0;
  sched->state = SRN_SCHED_IDLE;
  sched->workers = nullptr;
  sched->os_threads = nullptr;
  sched->destroyed = false;
  atomic_init(&sched->run_active, false);
 
  PANIC_IF(srn_mutex_init(&sched->lock) != SRN_THREAD_OK,
           "failed to initialise the scheduler lock");
 
  PANIC_IF(srn_cond_init(&sched->work) != SRN_THREAD_OK,
           "failed to initialise the scheduler condition");
 
  // srn_engine_make will store this scheduler in the engine
  return sched;
}

Here is the call graph for this function:

Here is the caller graph for this function:

srn_sched_register()

void srn_sched_register	(	srn_scheduler_t *	sched,
		srn_fiber_t *	fiber )

Record a fiber in the scheduler's registry of live fibers, where it stays until it is reaped.

Definition at line 305 of file scheduler.c.

                                                                    {
  PANIC_IF_NULL(sched);
  PANIC_IF_NULL(fiber);
 
  srn_mutex_lock(&sched->lock);
  registry_add(sched, fiber);
  srn_mutex_unlock(&sched->lock);
}

Here is the call graph for this function:

Here is the caller graph for this function:

srn_sched_run()

void srn_sched_run	(	srn_scheduler_t *	sched,
		int	nworkers )

Run the scheduler with nworkers os threads draining it, returning once the pool goes quiescent (every os thread parked on an empty queue) or a stop is requested with srn_sched_stop.

The calling os thread becomes worker 0, so it does not return until then. nworkers is clamped to at least 1; with 1 it is the calling os thread alone, which keeps execution single-threaded and cooperatively ordered. The spawned os threads are not joined here – srn_sched_shutdown joins them as part of tearing the subsystem down.

Definition at line 784 of file scheduler.c.

                                                         {
  PANIC_IF_NULL(sched);
  PANIC_IF(sched->destroyed, "srn_sched_run called on a scheduler that was already shut down");
 
  if (nworkers < 1) {
    nworkers = 1;
  }
 
  if (nworkers > SRN_MAX_WORKERS) {
    nworkers = SRN_MAX_WORKERS;
  }
 
  // Allocate `workers` and `os_threads` on the scheduler so
  // `srn_sched_shutdown` can join the threads and free them later. `workers`
  // has one entry per worker. `os_threads` has one per spawned thread, with
  // slot 0 left empty because the caller runs worker 0 inline (see the struct
  // comment). `runnable` is left alone, it already counts the fibers queued
  // before the run.
  sched->workers = srn_mm_allocate(sched->engine->mm, (size_t)nworkers * sizeof(srn_worker_t));
  PANIC_IF_NULL(sched->workers);
 
  sched->os_threads = srn_mm_allocate(sched->engine->mm, (size_t)nworkers * sizeof(srn_thread_t));
  PANIC_IF_NULL(sched->os_threads);
 
  for (int i = 0; i < nworkers; i++) {
    srn_worker_t *w = &sched->workers[i];
    w->sched = sched;
    w->id = i;
    w->current = nullptr;
    // The deque indices start empty. Its ring slots are written before they are
    // read, and the worker's loop is set up by worker_main on its own os
    // thread.
    atomic_init(&w->top, 0);
    atomic_init(&w->bottom, 0);
  }
 
  // Publish the coordination state before any os thread starts. `nworkers` must
  // be set first so the quiescence check counts the right total, and the state
  // must be RUNNING before an os thread can observe it. `run_active` lets
  // shutdown see that a run is in flight.
  sched->idle = 0;
  sched->nworkers = nworkers;
  atomic_store(&sched->state, SRN_SCHED_RUNNING);
  atomic_store(&sched->run_active, true);
 
  // Spawn nworkers - 1 os threads. The calling os thread runs worker 0 inline.
  // A spawn failure at startup is fatal, a partial pool would never reach `idle
  // == nworkers` and so never quiesce.
  for (int i = 1; i < nworkers; i++) {
    if (srn_thread_spawn(&sched->os_threads[i], worker_main, &sched->workers[i]) != SRN_THREAD_OK) {
      PANIC("failed to spawn an os thread");
    }
  }
 
  worker_main(&sched->workers[0]);
 
  // Worker 0 has stopped, so the run is over from the caller's point of view.
  // The spawned os threads may still be winding down, so they are NOT joined
  // here. `srn_sched_shutdown` joins them (the `os_threads` live on the
  // scheduler) as part of tearing the subsystem down. Clearing `run_active`
  // lets shutdown proceed. The state stays STOPPING, which keeps any os thread
  // still looping on its way out.
  atomic_store(&sched->run_active, false);
}

Here is the call graph for this function:

Here is the caller graph for this function:

srn_sched_shutdown()

void srn_sched_shutdown

(

srn_scheduler_t *

sched

)

The one stop tear down of the fiber subsystem, should be called once srn_sched_run has returned.

It joins the os threads, then releases the stack of every fiber still registered (any left unreaped, such as one suspended with no party to wake it, or one left queued when the run stopped early), and frees the scheduler's own resources. The fiber structs themselves live in context blocks and are reclaimed with those blocks, not here.

The scheduler is NOT usable after this. A later srn_sched_run panics, and a repeated srn_sched_shutdown is a no-op (so the engine can call it unconditionally even after a caller already did).

Must be called from outside the pool, never from an os thread that is running a worker nor from a fiber, and only after srn_sched_run has returned. Calling it on a live pool panics – stop the pool with srn_sched_stop, let srn_sched_run return, then shut down.

Definition at line 314 of file scheduler.c.

                                                {
  PANIC_IF_NULL(sched);
 
  if (sched->destroyed) {
    return;
  }
 
  // It must run on a thread outside the pool: a worker, or a fiber (which runs
  // on a worker), would be tearing down the scheduler it is itself running on.
  // `current_worker` is null only off a worker, so it is the test for that.
  PANIC_IF(current_worker != nullptr,
           "srn_sched_shutdown must be called from outside the worker pool");
 
  // And it must run after `srn_sched_run` has returned, not while a run is in
  // flight. Stop a running pool with `srn_sched_stop` and let `srn_sched_run`
  // return first.
  PANIC_IF(atomic_load(&sched->run_active),
           "srn_sched_shutdown called while srn_sched_run is active; call "
           "srn_sched_stop and let the run return first");
 
  // The run has returned, so worker 0 has stopped. The spawned os threads may
  // still be winding down (`srn_sched_run` does not join them), so join them
  // now. `os_threads` slot 0 is the inline worker 0, never spawned, so the
  // spawned os threads to join are 1..nworkers-1.
  for (int i = 1; i < sched->nworkers; i++) {
    (void)srn_thread_join(&sched->os_threads[i]);
  }
 
  // Every worker is gone, so this runs single threaded now.
  //
  // Any fiber still in the registry never finished: it was left parked in
  // SRN_FIBER_SUSPENDED with no party able to wake it (a deadlock), or was left
  // queued when the run stopped early. Release its stack so it does not leak.
  // The fiber structs themselves live in context blocks and are reclaimed with
  // those blocks, not here. The scheduler is immortal-allocated, so it is not
  // freed either.
  //
  // Unlike the reap path, this unmaps for good: shutdown runs after the workers
  // are gone, so the per-thread stack ring from the fiber.h TODO no longer
  // exists and there is nothing to recycle into. (Draining that ring, when it
  // exists, also belongs here.)
  srn_fiber_t *fiber = sched->registry;
  while (fiber != nullptr) {
    srn_fiber_t *next = fiber->reg_next;
    SCHED_LOG("shutdown reaping unfinished fiber '%s' (suspended with no waker?)",
              fiber->name != nullptr ? fiber->name : "<unnamed>");
    // TODO(lxsameer): Free up the ring here as well
    srn_fiber_stack_free(fiber->stack);
    srn_fiber_on_reap(fiber);
    fiber->reg_prev = nullptr;
    fiber->reg_next = nullptr;
    fiber = next;
  }
  sched->registry = nullptr;
 
  // Release the run-scoped storage (srn_mm_free tolerates null, so a scheduler
  // that never ran is fine).
  srn_mm_free(sched->engine->mm, sched->os_threads);
  srn_mm_free(sched->engine->mm, sched->workers);
  sched->os_threads = nullptr;
  sched->workers = nullptr;
  sched->nworkers = 0;
 
  // Destroy the synchronisation primitives. The scheduler is not usable after
  // this, so they are not re-initialised. No worker holds or waits on them now,
  // the join above made sure of that.
  (void)srn_cond_destroy(&sched->work);
  (void)srn_mutex_destroy(&sched->lock);
 
  sched->destroyed = true;
}

Here is the call graph for this function:

Here is the caller graph for this function:

srn_sched_stop()

void srn_sched_stop

(

srn_scheduler_t *

sched

)

Ask a running scheduler to stop.

Each worker routine checks the request at the top of each turn and stops once the fiber it is running yields or finishes, so a slice in flight is never cut mid-execution. srn_sched_run then returns. Fibers left queued stay unrun and are reclaimed by srn_sched_shutdown. Does not wait. Safe to call from any os thread, including a signal-driven context or a fiber. A no-op if the scheduler is not running.

Definition at line 849 of file scheduler.c.

                                            {
  PANIC_IF_NULL(sched);
  // Flip RUNNING to STOPPING once. If the scheduler is not running, or is
  // already stopping, there is nothing to do.
  srn_sched_state_t expected = SRN_SCHED_RUNNING;
 
  if (!atomic_compare_exchange_strong(&sched->state, &expected, SRN_SCHED_STOPPING)) {
    return;
  }
 
  // Running os threads see STOPPING at the top of their next turn. Parked os
  // threads are roused to observe it. The notify is under the lock, paired with
  // the park path, so no wakeup is lost.
  srn_mutex_lock(&sched->lock);
  srn_cond_notify_all(&sched->work);
  srn_mutex_unlock(&sched->lock);
  SCHED_LOG("stop requested");
}

Here is the call graph for this function:

Here is the caller graph for this function: