Scheduler & Lifecycle
The shard lifecycle layer exists to make one connection fail without turning the whole PXE node into a single failure domain.
Primary source:
High-level states
ASSIGNED
↓
STARTING
↓
READY
│
├──── transport failure ────→ BACKOFF
│ │
│ ▼
└──────────────────────────── reset/retry
invalid Gateway configuration
↓
FATAL
The lifecycle state answers what the scheduler should do next.
Protocol-specific state remains inside the TCP/TLS/WebSocket/Discord contexts.
Per-shard failure isolation
The lifecycle tick accepts one shard context.
That design is deliberate: the scheduler can reuse the same shard pointer across:
- lifecycle check
- TCP progression
- TLS progression
- WebSocket/Gateway progression
without rescanning the global pool inside every layer.
A compatibility wrapper can still sweep all active shards when needed.
Backoff
Current reconnect backoff is bounded by a fixed delay before retry.
When backoff is scheduled, the runtime:
- marks the shard BACKOFF
- increments reconnect count
- calculates a deadline from the TSC timing calibration
- logs the last error
- logs current TCP state
- logs current TLS state
This produces enough state to explain why the shard entered recovery.
Why log before reset?
Once the workspace is zeroed, the evidence that caused the failure disappears.
The lifecycle code therefore emits state before rebuilding the connection.
This is the bare-metal equivalent of preserving crash context before restarting a service.
Detecting failed transport
The lifecycle tick currently treats:
- TCP state
FAILED - TLS state
FAILED
as reasons to enter backoff.
Higher-level fatal Gateway errors can instead mark the shard fatal and prevent retry.
Reset scope
A shard reset is intentionally local.
It rebuilds only the failing shard's private state:
TCP page
reliable queue contents
TLS workspace
WebSocket workspace
Discord workspace
RX/TX lengths
heartbeat state
error state
reconnect deadline
Other active shard contexts remain untouched.
Preserving resumable sessions
Before transport reset, the lifecycle path saves:
- last Gateway sequence
- flags
After the lower layers are reset, it checks whether a valid session object still exists.
Only when both the session and sequence are valid does it restore sequence state for resume.
Otherwise the shard returns to an invalid-sequence state and must establish a fresh session.
Rebinding layers
After reset, the runtime:
- rebuilds the private WebSocket request
- rebinds the shard's TLS path to the shard's TCP context
- logs that the shard will retry
This ordering matters because the reconstructed upper layers must point at the reconstructed lower layer.
Zero deadlines
A zero reconnect deadline is treated as "schedule the normal backoff now."
This is useful when a failure site can mark a shard for backoff without duplicating timing logic.
Fatal state
A shard in FATAL state is skipped by normal retry processing.
This distinguishes:
transient transport failure
from
configuration/protocol failure that retry cannot solve
Without that distinction, one invalid configuration could create an endless reconnect loop.
Lease interaction
Shard lifecycle also exists within a larger ownership model.
A shard should only connect while the node owns a valid local plan/lease.
When an armed lease expires, the sharded transport can be stopped so another node can take ownership without creating duplicate live sessions.
Code example of lease-expiry gating
- NASM original
- Execution order
- Rust
- C
- Python
legacy_shards_lease_tick64:
cmp dword [rel shard_lease_expired], 1
je .done
mov r8, [rel shard_lease_deadline]
test r8, r8
jz .done
rdtsc
shl rdx, 32
or rax, rdx
cmp rax, r8
jb .done
call legacy_shards_expire_lease64
.done:
ret
1. Exit immediately if the lease is already marked expired.
2. Read the global lease deadline.
3. Treat a zero deadline as “lease enforcement disabled.”
4. Read the current TSC.
5. Compare current time against the lease deadline.
6. If the deadline has passed, expire the lease and stop shard-owned network traffic.
fn lease_tick(state: &mut ShardRuntime, now_tsc: u64) {
if state.lease_expired || state.lease_deadline == 0 {
return;
}
if now_tsc >= state.lease_deadline {
state.expire_lease();
}
}
void lease_tick(struct shard_runtime *state, uint64_t now_tsc) {
if (state->lease_expired || state->lease_deadline == 0)
return;
if (now_tsc >= state->lease_deadline)
expire_lease(state);
}
def lease_tick(state, now_tsc: int) -> None:
if state.lease_expired or state.lease_deadline == 0:
return
if now_tsc >= state.lease_deadline:
state.expire_lease()
Scheduler scalability
The context model scales by iterating active local slots, not the entire possible global shard count.
A node may participate in a bot with a large global shard count while owning only a bounded local subset.
That is why:
global total shards
local start
local count
are distinct values.
Diagnostics
Useful lifecycle fields include:
- state
- reconnect count
- reconnect deadline
- last error
- lease generation
- lease deadline
- TCP state
- TLS state
- heartbeat misses
These are exposed through the shard diagnostics path rather than hidden inside opaque protocol loops.
Reconnect backoff deadline scheduling
- NASM original
- Execution order
- Rust equivalent
- C equivalent
- Python model
| Symbol / register | Meaning |
|---|---|
RBX | Failing shard context. |
SHARD_RECONNECT_BACKOFF_MS | Fixed reconnect delay, currently 5000 ms. |
tsc_cycles_per_ms | TSC timing calibration. |
RECONNECT_DEADLINE | Absolute TSC timestamp when retry becomes eligible. |
mov dword [rbx + SHARD_OFF_STATE], SHARD_STATE_BACKOFF ; Stop normal connection progression.
inc qword [rbx + SHARD_OFF_RECONNECTS] ; Record another reconnect cycle.
mov rax, [rel tsc_cycles_per_ms] ; RAX = calibrated TSC cycles per millisecond.
test rax, rax ; Is timing calibration available?
jz .no_tsc ; No: store zero deadline and let caller reschedule.
imul rax, SHARD_RECONNECT_BACKOFF_MS ; RAX = 5000 ms expressed as TSC cycles.
mov r8, rax ; Preserve delay across RDTSC.
rdtsc ; EDX:EAX = current TSC.
shl rdx, 32 ; Shift high half into 64-bit position.
or rax, rdx ; RAX = full current TSC value.
add rax, r8 ; RAX = absolute reconnect deadline.
mov [rbx + SHARD_OFF_RECONNECT_DEADLINE], rax ; Publish retry eligibility time.
jmp .log ; Continue into failure-state logging.
.no_tsc:
mov qword [rbx + SHARD_OFF_RECONNECT_DEADLINE], 0 ; Zero means schedule normal backoff later.
1. Move the failing shard into BACKOFF.
2. Increment its reconnect counter.
3. Convert the fixed 5000 ms delay into TSC cycles.
4. Read the current TSC and add the delay.
5. Store the reconnect deadline.
6. Continue into failure logging before reset destroys the old state.
fn schedule_backoff(
shard: &mut ShardContext,
cycles_per_ms: u64,
now_tsc: u64,
) {
shard.state = ShardState::Backoff;
shard.reconnects += 1;
shard.reconnect_deadline = if cycles_per_ms == 0 {
0
} else {
now_tsc + cycles_per_ms * 5_000
};
}
void schedule_backoff(struct shard_ctx *shard,
uint64_t cycles_per_ms,
uint64_t now_tsc) {
shard->state = SHARD_STATE_BACKOFF;
shard->reconnects++;
shard->reconnect_deadline =
cycles_per_ms ? now_tsc + cycles_per_ms * 5000ull : 0;
}
def schedule_backoff(shard, cycles_per_ms, now_tsc):
shard.state = "BACKOFF"
shard.reconnects += 1
shard.reconnect_deadline = (
now_tsc + cycles_per_ms * 5000
if cycles_per_ms
else 0
)
The function continues by printing last-error, TCP-state, and TLS-state values before connection reset destroys the evidence. That is the bare-metal equivalent of capturing crash context before restarting a service.
Only the failing shard enters BACKOFF; unrelated shard contexts keep their current transport and Gateway state.
Gateway resume-state preservation across transport reset
TCP/TLS/WebSocket state is disposable. A valid Discord session and sequence may not be.
- NASM original
- Execution order
- Rust equivalent
- C equivalent
- Python model
| Symbol / register | Meaning |
|---|---|
R14D | Gateway sequence saved before transport reset. |
R15D | Shard flags saved before transport reset. |
R12 | Session object pointer loaded after lower-layer reset. |
SSES_VALID | Session-structure flag proving the saved session can be resumed. |
SHARD_FLAG_SEQ_VALID | Proves the saved sequence is meaningful. |
mov r12, [rbx + SHARD_OFF_SESSION_PTR] ; R12 = session object retained across transport reset.
test r12, r12 ; Does a session object exist?
jz .no_resume ; No: resume is impossible.
cmp dword [r12 + SSES_VALID], 1 ; Is the retained session explicitly valid?
jne .no_resume ; No: force a fresh Identify path.
test r15d, SHARD_FLAG_SEQ_VALID ; Was the pre-reset Gateway sequence valid?
jz .no_resume ; No: a session without sequence cannot Resume safely.
mov [rbx + SHARD_OFF_SEQUENCE], r14d ; Restore the saved Gateway sequence.
or dword [rbx + SHARD_OFF_FLAGS], SHARD_FLAG_SEQ_VALID ; Re-publish sequence validity.
jmp .resume_done ; Continue reconnect with resumable Gateway state.
.no_resume:
mov dword [rbx + SHARD_OFF_SEQUENCE], 0xFFFFFFFF ; Sentinel means no usable sequence exists.
1. Load the retained Discord session pointer.
2. Reject Resume if no session object exists.
3. Verify the retained session is marked valid.
4. Verify the saved Gateway sequence was valid before reset.
5. Restore the saved sequence and its validity flag when both conditions pass.
6. Otherwise write the invalid-sequence sentinel and force a fresh Identify path.
fn restore_resume_state(
shard: &mut ShardContext,
saved_sequence: u32,
saved_flags: ShardFlags,
) {
let valid_session =
shard.session.as_ref().is_some_and(|session| session.valid);
let valid_sequence =
saved_flags.contains(ShardFlags::SEQ_VALID);
if valid_session && valid_sequence {
shard.sequence = saved_sequence;
shard.flags.insert(ShardFlags::SEQ_VALID);
} else {
shard.sequence = 0xFFFF_FFFF;
}
}
void restore_resume_state(struct shard_ctx *shard,
uint32_t saved_sequence,
uint32_t saved_flags) {
int valid_session = shard->session && shard->session->valid == 1;
int valid_sequence = (saved_flags & SHARD_FLAG_SEQ_VALID) != 0;
if (valid_session && valid_sequence) {
shard->sequence = saved_sequence;
shard->flags |= SHARD_FLAG_SEQ_VALID;
} else {
shard->sequence = 0xFFFFFFFFu;
}
}
def restore_resume_state(shard, saved_sequence, saved_flags):
valid_session = shard.session is not None and shard.session.valid
valid_sequence = bool(saved_flags & SEQ_VALID)
if valid_session and valid_sequence:
shard.sequence = saved_sequence
shard.flags |= SEQ_VALID
else:
shard.sequence = 0xFFFFFFFF
A session pointer alone is insufficient, and a saved sequence alone is insufficient. Resume state is restored only when the session object is valid and the pre-reset sequence carried SHARD_FLAG_SEQ_VALID.
A resumable shard keeps only the minimum Gateway state needed for Resume while all transport/crypto framing state is rebuilt from scratch.