Diagnostics

Discord-PXE is a bare-metal project, so diagnostics have to exist before most of the services developers normally rely on.

There is no kernel log, no journalctl, no socket inspector, and no userspace debugger attached to a production PXE boot.

The runtime therefore treats serial output, shell commands, state counters, and staged acceptance tests as first-class engineering tools.

Primary sources:

• legacy_shell.inc
• legacy_shards.inc
• TESTING.md

Diagnostic philosophy

The useful question is not:

Did Discord-PXE work?

It is:

What was the highest layer that was definitely healthy before the failure?

Use this order:

1. PXE entry
2. stage-1 runtime download
3. x86-64 transition
4. NIC / IP
5. TCP
6. TLS
7. WebSocket
8. Gateway Hello
9. Identify / Resume
10. READY
11. heartbeat ACK
12. shard isolation
13. lease / multi-node ownership

Each successful layer narrows the next investigation.

For the exact stage-1 breadcrumb letters and every major serial prefix, see Startup Trace & Serial Markers.

Serial starts early

Serial is the recommended primary diagnostic console for physical Discord-PXE hardware.

The stage-1 loader initializes the standard legacy UARTs as 115200 baud, 8 data bits, no parity, 1 stop bit, no flow control. It brings up COM1 at 0x3F8 and COM2 at 0x2F8, enables the FIFO, and asserts DTR/RTS. The runtime later performs BIOS UART discovery as it transitions into the full shell/input path.

Serial initialization happens in the loader and runtime before the high-level Discord path exists.

That means serial can distinguish failures such as:

firmware API was never found
TFTP never completed
runtime started but long mode failed
network came up but TCP did not
TLS failed after TCP was healthy
Gateway failed after WebSocket upgrade

This is much more useful than one generic "boot failed" status.

Stage-1 breadcrumbs

The loader emits compact markers around important PXE operations.

These breadcrumbs are deliberately simple because the early environment is fragile.

A marker sequence can establish whether code reached:

• serial initialization
• PXE API discovery
• PXENV+ validation
• cached network lookup
• TFTP load
• runtime handoff

before richer logging is available.

Connecting to the interactive shell

Open the host serial terminal at:

115200 baud
8 data bits
no parity
1 stop bit
flow control: none

Keep the terminal connected through PXE boot. Early characters are loader breadcrumbs; after the 64-bit runtime initializes, the same console becomes interactive and eventually presents:

pxe>

On Windows, PuTTY or Tera Term are practical choices. Select the COM port corresponding to the physical serial interface or USB serial adapter.

RS-232 vs TTL UART

Match the adapter to the target's electrical interface. Traditional PC COM/DB9 ports use RS-232 levels. Bare motherboard/debug UART headers may use TTL levels. The protocol settings can be identical while the electrical signaling is not.

Interactive shell

The 64-bit runtime includes a small serial/keyboard shell.

The shell is intentionally simple so its parser is unlikely to become a debugging problem of its own.

General commands cover areas such as:

help
cpu
mem
memmap
pci
net
stats
input
ls
cat
fetch
hex
discord
pxe
config
reboot
seriallive

Sharding adds dedicated commands:

shards
shardstart
shardstop
shardplan
shardreset
shardtest2
shardteststatus
shardlease

Avoiding log corruption while typing

The shell exposes an "input busy" check.

Live telemetry can use that signal to avoid printing over the operator's partially typed command line.

The implementation checks:

• buffered shell line length
• input ring head/tail
• pending COM1 input
• pending keyboard controller input

This sounds small, but it matters in a serial-first debugging environment: unreadable interleaved output can destroy the evidence needed to understand a timing failure.

shards

The shard diagnostic command is intended to expose the local plan and per-slot state.

Useful fields include:

• global shard ID
• lifecycle state
• sequence validity
• heartbeat state
• Identify state
• TCP pointer/state
• TLS pointer/state
• WebSocket pointer
• Discord pointer
• reconnect count
• last error
• lease generation/deadline

The exact output can evolve, but the principle stays the same: the operator should be able to inspect one local slot without reverse-engineering raw memory.

shardtest2

The built-in two-shard acceptance command deliberately creates the smallest useful concurrency case.

It:

1. verifies configuration/token readiness
2. installs a two-shard local plan
3. shuts down the old singleton Gateway connection
4. quiesces singleton transport for the boot
5. allocates independent state for shard 0 and shard 1
6. starts both private TCP clients
7. applies Identify scheduling

This makes the result attributable to the new context-based path.

shardteststatus

The acceptance check intentionally verifies more than event-level success.

A complete pass requires both shards to be READY with authenticated TLS and heartbeat acknowledgements.

Code example of two-shard acceptance checks

NASM original

cmp dword [rel shard_active_count], 2
jne .not_ready

; structural isolation
mov rax, [r12 + SHARD_OFF_TCP_CTX]
test rax, rax
jz .not_ready
cmp rax, [r13 + SHARD_OFF_TCP_CTX]
je .isolation_fail

mov rax, [r12 + SHARD_OFF_TLS_CTX]
test rax, rax
jz .not_ready
cmp rax, [r13 + SHARD_OFF_TLS_CTX]
je .isolation_fail

; health
cmp dword [r14 + SHARD_OFF_STATE], SHARD_STATE_READY
jne .not_ready
test dword [r14 + SHARD_OFF_FLAGS], SHARD_FLAG_READY
jz .not_ready
cmp qword [r14 + SHARD_OFF_HEARTBEAT_ACKS], 0
je .not_ready
cmp dword [r14 + SHARD_OFF_LAST_ERROR], 0
jne .not_ready

This is the core of the real acceptance routine; the full function also checks local ports, WebSocket/Discord workspaces, IDs, totals, and authenticated TLS state.

Execution order

1. Require exactly two active local shard slots.
2. Resolve both shard contexts.
3. Verify the two shards do not share mutable TCP/TLS/WebSocket/Discord workspaces.
4. Verify their local TCP ports differ.
5. Verify IDs are 0 and 1 and both report a global total of 2.
6. Require both lifecycle states to be READY.
7. Require TLS to be authenticated/up.
8. Require at least one heartbeat ACK on each shard.
9. Require no shard-level error before reporting PASS.

Rust

fn shard_is_healthy(shard: &ShardContext) -> bool {
    shard.state == ShardState::Ready
        && shard.tls_authenticated
        && shard.heartbeat_acks > 0
        && shard.last_error == 0
}

fn two_shard_acceptance(a: &ShardContext, b: &ShardContext) -> bool {
    shard_is_healthy(a)
        && shard_is_healthy(b)
        && a.tcp_ctx != b.tcp_ctx
        && a.local_port != b.local_port
        && a.tls_ctx != b.tls_ctx
        && a.websocket_ctx != b.websocket_ctx
        && a.discord_ctx != b.discord_ctx
}

C

int shard_is_healthy(const struct shard_ctx *s) {
    return s->state == SHARD_STATE_READY
        && s->tls_authenticated
        && s->heartbeat_acks > 0
        && s->last_error == 0;
}

int two_shard_acceptance(const struct shard_ctx *a,
                         const struct shard_ctx *b) {
    return shard_is_healthy(a)
        && shard_is_healthy(b)
        && a->tcp_ctx != b->tcp_ctx
        && a->local_port != b->local_port
        && a->tls_ctx != b->tls_ctx
        && a->websocket_ctx != b->websocket_ctx
        && a->discord_ctx != b->discord_ctx;
}

Python

def shard_is_healthy(shard) -> bool:
    return (
        shard.state == "READY"
        and shard.tls_authenticated
        and shard.heartbeat_acks > 0
        and shard.last_error == 0
    )

def two_shard_acceptance(a, b) -> bool:
    return (
        shard_is_healthy(a)
        and shard_is_healthy(b)
        and a.tcp_ctx != b.tcp_ctx
        and a.local_port != b.local_port
        and a.tls_ctx != b.tls_ctx
        and a.websocket_ctx != b.websocket_ctx
        and a.discord_ctx != b.discord_ctx
    )

It also verifies separation of:

• TCP contexts
• local TCP ports
• TLS workspaces
• WebSocket workspaces
• Discord workspaces

READY alone is not proof of isolation

Two sessions can appear functionally alive while accidentally sharing state that fails only later under load or reconnect. Pointer/workspace checks make the acceptance test structural, not merely behavioral.

Reset testing

shardreset N rebuilds one local shard slot.

This is useful for verifying that:

• one shard can reconnect without restarting its neighbors
• reliable TCP storage is reused safely
• TLS/WebSocket state resets cleanly
• valid resume state is preserved only when appropriate
• heartbeat state does not leak across sessions

Lease testing

shardlease 30 arms local lease-expiry behavior.

When the lease expires, sharded transport stops.

This allows testing the safety property that a node eventually gives up ownership rather than remaining connected forever after control-plane loss.

Fatal vs transient failures

Diagnostics should distinguish:

temporary network failure
        ↓
BACKOFF / retry

invalid Gateway configuration
        ↓
FATAL / stop

An endless reconnect loop is itself a diagnostic failure because it hides the difference between recoverable and unrecoverable state.

Layer-specific symptoms

No PXE banner

Investigate firmware boot configuration, DHCP/PXE reply, boot filename, and TFTP reachability.

Stage 1 runs, runtime does not

Investigate runtime download length, protected memory overlap, PXE stack preservation, and final handoff.

Runtime shell works, network does not

Inspect NIC detection, IP/DHCP handoff, ARP/gateway state, and driver assumptions.

TCP is UP, TLS does not authenticate

Inspect TLS state, RX/TX byte counters, handshake transcript progress, certificate/authentication path, and record parsing.

TLS authenticates, WebSocket does not open

Inspect HTTP 101 response handling, nonce/accept verification, and encrypted application-data delivery.

WebSocket opens, Gateway never READY

Inspect Hello parsing, heartbeat interval, Identify scheduling, token/config state, and Gateway close codes.

READY occurs, heartbeat ACK never arrives

Inspect event-loop progression, heartbeat deadline scheduling, outgoing Gateway frames, and ACK flag transitions.

One shard repeatedly fails while another remains healthy

That is evidence the failure is likely shard-local rather than machine-global. Compare the two slot dumps.

Counters over guesses

Useful counters include:

• TCP RX/TX totals
• TCP retries
• RST count
• TLS RX/TX bytes
• handshake generation
• heartbeat sent/ACK/miss counters
• reconnect count
• last error

Counters are valuable because they show whether a state machine is actually progressing.

Hardware evidence matters

A successful NASM build proves syntax and link/layout assumptions.

A successful VM boot proves one firmware/hardware emulation environment.

Neither proves the same path on an old physical NIC/PXE ROM.

The project therefore treats hardware acceptance as its own layer of compatibility evidence.

Minimum useful bug report

commit:
physical hardware or VM:
CPU:
NIC:
PXE firmware / ROM:
PXE/TFTP server:
singleton or sharded mode:
global shard count:
local shard plan:
last successful layer:
shards output:
shardteststatus output:
serial log around failure:

Shell input-busy arbitration

The runtime avoids printing live telemetry over an active shell command.

NASM original

Context before the block

Symbol / port	Meaning
shell_len	Number of characters already typed into the active shell line.
input_head/input_tail	Producer/consumer indices for the runtime input ring.
input_uart1	Base I/O port for the selected UART.
UART + 5	16550-compatible Line Status Register. Bit 0 means RX data is ready.
0x64	PS/2 keyboard-controller status port. Bit 0 means output data is waiting.

legacy_shell_input_busy64:
    cmp dword [rel shell_len], 0   ; Has the operator already typed part of a command?
    jne .busy                      ; Yes: suppress asynchronous telemetry.

    mov eax, [rel input_head]      ; EAX = producer index of the input ring.
    cmp eax, [rel input_tail]      ; Compare against consumer index.
    jne .busy                      ; Pending buffered input means the operator is active.

    movzx edx, word [rel input_uart1] ; EDX = UART base I/O port, or zero if unavailable.
    test dx, dx                    ; Is a UART configured?
    jz .check_kbd                  ; No: skip directly to keyboard-controller check.

    add dx, 5                      ; DX = UART Line Status Register.
    in al, dx                      ; Read hardware UART status.
    test al, 1                     ; Bit 0 = received data ready.
    jnz .busy                      ; A byte is waiting even if the ring has not consumed it yet.

.check_kbd:
    in al, 0x64                    ; Read PS/2 controller status.
    test al, 1                     ; Bit 0 = output buffer contains a pending key byte.
    jnz .busy                      ; Avoid printing over that first unconsumed keystroke.

    xor eax, eax                   ; EAX = 0, shell is idle.
    ret                            ; Return false.

.busy:
    mov eax, 1                     ; EAX = 1, shell/input path is active.
    ret                            ; Return true.

Execution order

1. Check whether the shell line already contains typed characters.
2. Check whether the software input ring contains unread bytes.
3. Check the UART Line Status Register for pending RX data.
4. Check the PS/2 controller status for pending keyboard data.
5. Return busy immediately if any source has input waiting.
6. Return idle only when every input source is clear.

Rust equivalent

fn shell_input_busy(
    shell: &ShellState,
    input: &InputRing,
    uart: &mut Uart,
    keyboard: &mut KeyboardController,
) -> bool {
    if shell.length != 0 {
        return true;
    }

    if input.head != input.tail {
        return true;
    }

    if uart.is_configured() && (uart.line_status() & 0x01) != 0 {
        return true;
    }

    (keyboard.status() & 0x01) != 0
}

C equivalent

int shell_input_busy(void) {
    if (shell_len != 0) return 1;
    if (input_head != input_tail) return 1;
    if (input_uart1 && (uart_line_status(input_uart1) & 0x01)) return 1;
    if (inb(0x64) & 0x01) return 1;
    return 0;
}

Python model

def shell_input_busy(shell, input_ring, uart, keyboard) -> bool:
    if shell.length:
        return True
    if input_ring.head != input_ring.tail:
        return True
    if uart.configured and (uart.line_status() & 0x01):
        return True
    if keyboard.status() & 0x01:
        return True
    return False

Success condition

Telemetry callers receive EAX=0 only when the line buffer, software input ring, UART receive register, and keyboard-controller output buffer are all idle.

Why this tiny routine matters

On a normal OS, terminal rendering and kernel input queues hide this problem. Here, Discord-PXE owns both asynchronous diagnostics and the interactive shell, so preserving a readable command line is part of the runtime's observability contract.

Source

• legacy_shell.inc
• legacy_shards.inc
• legacy_shard_lifecycle.inc
• TESTING.md