How a job runs
One daemon, runnerd serve, owns every slot. Clients never talk to GitHub or Docker: they read the daemon’s status and history and send it commands over its Unix socket ($XDG_RUNTIME_DIR/runnerd.sock). The dashboard runs inside the daemon’s process but uses the same socket.
Life of a job
Section titled “Life of a job”- Queued. The job waits on GitHub, asking for labels that one of the repository’s profiles serves.
- Listed. Every
poll_intervalthe engine lists each repository’s queued jobs, and three times as often while the repository is active: a job in the last 10 minutes, or a runner running one. It lists this machine’s runners of a repository first, every third time and after one of them starts or goes, so a job whose runner just took it does not get a second one. A command lists every repository at once. - Admitted. The planner picks a free slot in the repository’s profile whose labels match the job. A job starts only if its runner’s memory fits the budget: its predicted peak (its cap while the job has too little history, or under
admission = "caps"), within what the host leaves the slice. A job that does not fit stays queued, andrunnerctl statussays why. - Runner started. The engine registers an ephemeral runner with the pool’s labels and starts a container for it in the slot, under sysbox, over the profile’s snapshot. Up to four runners start at once, and their registrations go to GitHub one after another. The runner takes one job that matches those labels, not necessarily the one that prompted the start, and exits after it.
- Job. While it runs, the engine records the job’s steps, its log and, every 2 seconds, its container’s CPU and memory in the history store. Clients stream this through
GET /v1/events. - Recorded. When the runner exits, its credentials are scrubbed and, on a Sysbox profile that opts in, the image probes a sample of the calls Sysbox serves. Docker’s event wakes the engine, which inspects the container, removes it, frees the slot and starts a runner for the next listed job there without waiting for the next poll. GitHub drops the runner’s registration with its job; runnerd deletes one it kept without holding the slot. If Docker’s events are not coming through, the next poll finds the exit. The job’s record is final once GitHub reports it completed. A failed job of a profile with
record_failuresleaves a record first, whichrunnerctl shellopens.
Scheduling in detail
Section titled “Scheduling in detail”- It reads the
ghCLI’s token at start, and again when GitHub rejects it. Everypoll_interval(15 s by default) it asks GitHub for each repository’s queued jobs, and three times as often, every 5 s by default, for a repository with jobs lately. - The pool whose
labelsinclude every label in a job’sruns-onserves it: one[[repositories]]entry, or one of several. - It registers a just-in-time runner and starts it in a free slot of the pool’s profile if the slice’s
MemoryMaxhas room for the profile’smemory(under measured admission, the job’s predicted peak) and, under CPU admission, the CPU budget has room too. - A job that does not fit lets younger ones past, until it has waited 10 minutes for memory or CPU, or 2 if its pool already went first and still has jobs queued. Then it goes first. What it wins is the next runner of its pool: GitHub gives a runner any job its labels match, so another job of that pool may take it.
- A runner takes one job, then runnerd removes it. A runner with no job after 5 minutes is removed unless it is warm.
- Each tick removes the idle runners it no longer needs first, all at once, then starts runners for queued jobs. A runner that no longer fits what runs by then waits; the others start, then the tick plans again from the same listings (at most three times) instead of waiting for the next poll.
- When a runner’s container stops, Docker’s event wakes runnerd between polls: it removes the runner and starts one for the next job the last poll listed in its slot within seconds. A job cancelled or taken elsewhere since that poll can still get a runner, which then waits like any idle one. A job that went first holds these starts back too.
- A job’s page shows when runnerd first saw it queued (Seen by runnerd; a job a waiting runner took before any listing showed it has none), so its wait splits into the time before runnerd knew of it, the time it took to start a runner, and the runner’s boot.
- A restarted runnerd adopts running runner containers.
How often GitHub is asked
Section titled “How often GitHub is asked”When the hour’s rate limit runs short, runnerd reads running jobs less often first, then lists active repositories at the idle pace, then every repository two or four times less often, one step at a time, and logs each change; no registration or delete waits for this, but what a tick decides, such as stopping an idle runner or topping up a warm one, comes with the slower ticks.
Where each part lives maps these parts to the code.