TCP Reliability & Flow

TLS and Discord are only meaningful if the underlying TCP stream has reliable ordering and retry behavior.

Discord-PXE therefore treats TCP reliability as its own subsystem rather than assuming that "SYN succeeded" means the transport is solved.

Primary source:

• legacy_shard_tcp.inc
• legacy_shard_tcp_reliable.inc

Why a separate reliable workspace?

The top-level shard TCP page contains connection state, but retransmission tracking can require additional storage.

The implementation keeps a separate reliable queue pointer that can survive a connection-structure reset.

During a shard reconnect:

1. save the reliable-queue pointer
2. zero the TCP page
3. restore the queue pointer
4. zero the queue workspace
5. rebuild the shard's deterministic local port
6. return the connection state to IDLE

This avoids allocating and leaking a new reliability workspace every retry cycle.

Sequence ownership

Each shard owns its own:

local sequence
remote sequence
ACK progression
retry state
retransmission queue

Sharing any of these across shards would make one connection's retransmission behavior corrupt another connection.

Maximum payload

The sharded TCP send path uses a bounded payload size:

SHARD_TCP_MAX_PAYLOAD = 1200

The smaller bound simplifies frame construction and avoids assuming Ethernet-path fragmentation behavior for every higher-level write.

TLS can split its records across multiple TCP writes when necessary.

Checksum trust boundary

Because the runtime constructs TCP segments directly, checksum correctness is part of the application.

The checksum uses the IPv4 pseudo-header and TCP bytes, then one's-complement folds the result.

A checksum bug can present as:

• silent remote packet drops
• failed SYN/ACK progression
• TLS stalls that are actually transport loss
• apparent random shard instability

This is why debugging proceeds from TCP counters/state before blaming TLS.

Retry vs shard lifecycle

TCP-level retry behavior and shard-level reconnect behavior are related but not identical.

A single connection can retry within its current transport state.

If the transport is marked failed, the shard lifecycle moves the entire shard into BACKOFF and later rebuilds TCP/TLS/WebSocket/Discord state coherently.

That prevents the TLS layer from remaining "half alive" on top of a reset TCP connection.

See Scheduler & Lifecycle.

Reset behavior

When a shard connection is rebuilt, the lifecycle path intentionally resets:

• TCP connection page
• reliable retransmission queue contents
• TLS workspace
• WebSocket workspace
• Discord workspace
• RX/TX lengths
• heartbeat timing
• error state

A valid resumable Discord session may preserve its sequence separately, but TCP sequence state does not.

Why transport counters matter

The TCP context tracks receive and transmit totals plus bad/RST counts.

Those counters help classify failures:

no RX after SYN       -> routing / ARP / remote response problem
TCP UP, no TLS bytes  -> binding / TLS-start problem
RX grows, TLS fails   -> TLS parser / crypto / record issue
RST increases         -> connection rejected/reset

On bare metal, counters like these replace much of what tools such as ss, netstat, or packet-socket instrumentation would provide on an OS.

Shard-specific source ports

The deterministic port assignment is also useful diagnostically.

If two local shard contexts accidentally report the same local TCP port when they should not, the isolation contract is already broken before TLS or Gateway state is considered.

That is one reason the two-shard acceptance check validates distinct local TCP ports.

Reliability is below Discord semantics

The transport layer does not know whether a payload contains:

• TLS handshake bytes
• encrypted application data
• WebSocket frames
• Gateway JSON

It only owns byte-stream correctness.

Keeping those semantics separate means a transport fix should not require understanding Discord opcodes.

TCP reconnect reset and retransmission-state preservation

NASM original

Context before the block

Symbol / register	Meaning
RBX	Owning shard context.
R12	Per-shard TCP workspace.
R13	Separately allocated reliable retransmission workspace.
STCP_REL_PTR	Pointer stored inside the TCP workspace to that retransmission area.
SREL_PAGES	Number of pages reserved for reliable retransmission state.

mov r12, [rbx + SHARD_OFF_TCP_CTX]       ; R12 = this shard's TCP workspace pointer.
test r12, r12                            ; Was a TCP workspace ever allocated?
jz .tls                                  ; No: skip directly to resetting the TLS layer.

mov r13, [r12 + STCP_REL_PTR]            ; Preserve the reliable-queue allocation pointer.

mov rdi, r12                             ; RDI = start of 4 KiB TCP workspace.
xor eax, eax                             ; RAX = zero pattern for REP STOSQ.
mov ecx, 4096/8                          ; Clear 4096 bytes as 512 qwords.
rep stosq                                ; Zero the entire TCP workspace.

mov [r12 + STCP_REL_PTR], r13            ; Reattach the separately allocated reliable queue.

mov eax, [rbx + SHARD_OFF_ID]            ; EAX = global shard ID.
and eax, SHARD_TCP_PORT_MASK             ; Keep low 12 bits for deterministic port mapping.
add eax, SHARD_TCP_PORT_BASE             ; Translate into local source-port range.
mov [r12 + STCP_LOCAL_PORT], eax         ; Restore this shard's derived source port.

mov dword [r12 + STCP_STATE], SHARD_TCP_IDLE ; Fresh transport begins from IDLE.

test r13, r13                            ; Does the reusable reliable workspace exist?
jz .tls                                  ; No: continue with upper-layer reset.

mov rdi, r13                             ; RDI = reliable retransmission workspace.
xor eax, eax                             ; Zero fill value.
mov ecx, (SREL_PAGES * 4096) / 8         ; Number of qwords in reliable workspace.
rep stosq                                ; Clear retransmission state without freeing its pages.

Execution order

1. Load this shard's TCP workspace.
2. Save the separately allocated reliable-queue pointer.
3. Zero the 4 KiB TCP workspace.
4. Restore the reliable-queue pointer.
5. Recompute the shard's deterministic local source port.
6. Return TCP state to IDLE.
7. Zero the reliable retransmission workspace without freeing it.

Rust equivalent

fn reset_tcp_for_reconnect(shard: &mut ShardContext) {
    let Some(tcp) = shard.tcp.as_mut() else {
        return;
    };

    let reliable = tcp.reliable_queue.take();

    tcp.clear_page();
    tcp.reliable_queue = reliable;
    tcp.local_port = 50_000 + (shard.id & 0x0FFF);
    tcp.state = TcpState::Idle;

    if let Some(queue) = tcp.reliable_queue.as_mut() {
        queue.clear();
    }
}

C equivalent

void reset_tcp_for_reconnect(struct shard_ctx *shard) {
    struct tcp_ctx *tcp = shard->tcp;
    if (!tcp) return;

    void *reliable = tcp->reliable_ptr;
    memset(tcp, 0, 4096);

    tcp->reliable_ptr = reliable;
    tcp->local_port = 50000u + (shard->id & 0x0FFFu);
    tcp->state = SHARD_TCP_IDLE;

    if (reliable)
        memset(reliable, 0, SREL_PAGES * 4096u);
}

Python model

def reset_tcp_for_reconnect(shard):
    tcp = shard.tcp
    if tcp is None:
        return

    reliable = tcp.reliable_queue

    tcp.clear_page()
    tcp.reliable_queue = reliable
    tcp.local_port = 50_000 + (shard.id & 0x0FFF)
    tcp.state = "IDLE"

    if reliable is not None:
        reliable.clear()

Ownership invariant

The reliable workspace is intentionally not inside the 4 KiB TCP page. Failing to preserve STCP_REL_PTR before rep stosq would orphan that allocation and leak pages on each reconnect.

Success condition

After this block, the shard has a clean TCP state machine, the same reusable retransmission allocation, and its deterministic source port restored.

State after the block

Transport sequence/retry state is gone. The allocation topology remains intact, allowing TLS/WebSocket/Discord reset to continue without reallocating everything.

Source

• legacy_shard_tcp.inc
• legacy_shard_tcp_reliable.inc
• legacy_shard_lifecycle.inc