Skip to main content

Troubleshooting

Debug Discord-PXE by the last transition that definitely succeeded.

Do not start at the final error line. The useful question is:

What was the highest healthy layer before progress stopped?

Failure ladder​

PXE firmware
  -> stage1
  -> runtime download
  -> long mode
  -> native NIC
  -> IP / ARP
  -> DNS
  -> TCP
  -> TLS
  -> WebSocket
  -> Gateway Hello
  -> Identify / Resume
  -> READY
  -> heartbeat ACK
  -> shard isolation
  -> lease / multi-node ownership

PXE never reaches the loader​

Check:

  • legacy PXE/CSM enabled
  • DHCP/PXE response
  • boot filename
  • TFTP root
  • PXE service reachability
  • network boot order

If the target never begins stage-1 execution, nothing above the loader matters yet.

Loader starts but runtime does not​

Inspect:

  • TFTP runtime length
  • stage-1 serial breadcrumbs
  • PXENV+ validation
  • cached DHCP retrieval
  • firmware stack preservation
  • runtime handoff

A runtime image that overlaps firmware-sensitive memory can fail in ways that look unrelated to TFTP.

Runtime starts but long mode fails​

Inspect:

  • x86-64 capability
  • page-table construction
  • PAE
  • CR3
  • EFER.LME
  • GDT
  • far jump into the 64-bit code segment

See Boot Chain.

Long mode works but networking fails​

Inspect:

  • PCI/NIC detection
  • Realtek backend
  • DHCP handoff values
  • router/DNS propagation
  • ARP
  • native TX/RX state

If DNS never sends a query, do not debug TLS.

DNS resolves but TCP does not connect​

Inspect:

  • remote IP
  • route/gateway
  • ARP destination MAC
  • TCP SYN state
  • local source port
  • retries/RST counters

Use shards for per-slot state when testing the shard path.

TCP connects but TLS does not authenticate​

Classify the last TLS marker.

No ClientHello​

Inspect:

  • RNG readiness
  • TLS initialization
  • TX availability
  • native NIC TX contention

Certificate chain fails​

Inspect:

  • TLS record reassembly
  • DER parser
  • issuer trust
  • RSA public-key extraction
  • current server chain

Client key flight sent, no authenticated session​

Inspect:

  • X25519 shared secret
  • transcript hash
  • TLS PRF
  • AES-GCM keys/IVs
  • record sequence numbers
  • Finished verify_data
SMP-only TLS failures

If BSP mode is stable but AP shard execution causes widespread TLS failures, suspect shared crypto scratch or shared TX ownership before assuming every Discord certificate/session is bad.

TLS authenticates but WebSocket does not open​

Once you have:

[TLS] authenticated tls established

stop changing X.509/X25519 without evidence.

Inspect:

  • HTTP Upgrade request
  • HTTP 101 response
  • Sec-WebSocket-Accept
  • encrypted application-record delivery
  • WebSocket frame parsing

WebSocket opens but Gateway never reaches READY​

Inspect:

  • Gateway Hello
  • heartbeat interval
  • Identify scheduling
  • session-start availability
  • token/config state
  • close codes

READY occurs but heartbeat ACKs stop​

Inspect:

  • heartbeat deadline
  • outbound Gateway frames
  • awaiting-ACK flag
  • miss counter
  • event-loop progression
  • reconnect transition

READY is not sufficient proof of a healthy long-lived shard.

Invalid-session storm​

Repeated invalid sessions are usually evidence of:

  • incorrect Identify scheduling
  • exhausted session-start budget
  • broken Resume/Identify choice

Local shell plans intentionally force Identify concurrency to 1 because runtime /gateway/bot discovery is not yet available.

One shard fails, neighbors stay healthy​

That strongly suggests shard-local state.

Compare:

  • TCP context pointer
  • local port
  • TLS state
  • WebSocket state
  • Discord state
  • sequence
  • heartbeat counters
  • reconnect count
  • last error

Many shards fail together​

Look for shared resources first:

  • native TX descriptor
  • crypto scratch
  • DNS/global resolver
  • connection admission
  • Identify scheduling
  • common routing failure

A synchronized failure across many shard IDs is less likely to be one bad private context.

Useful shell commands​

shards
shardteststatus
shardreset <slot>
shardstop
shardlease <seconds>

Minimum useful bug report​

commit:
hardware / VM:
CPU:
NIC:
PXE firmware / ROM:
PXE/TFTP server:
serial settings:
singleton or sharded mode:
global shard count:
local shard plan:
last successful layer:
first failing marker:
shards output:
shardteststatus output:
serial log around failure:
Keep the full serial session

Low-level failures are often diagnosed by the transition immediately before the final error. A single copied error line usually removes the useful context.

For deep protocol-specific diagnosis, see TLS & WebSocket, Diagnostics, and TCP Reliability.