Skip to main content

Design Decisions

This page collects the architecture choices that are easiest to misread when looking at individual assembly files in isolation.

These are not cosmetic preferences. They define how Discord-PXE preserves known-good hardware behavior while expanding from one Gateway connection into a multi-shard, potentially multi-node system.

Keep the hardware-proven singleton baseline​

The original runtime is preserved on the baseline branch.

Reason: the sharding work changes ownership of state throughout TCP, TLS, WebSocket, and Discord layers.

Without a preserved reference, a failure after refactoring would not answer whether:

  • the protocol implementation regressed
  • the hardware path regressed
  • or the new context conversion is wrong

The baseline is therefore a regression oracle.

Refactor one layer at a time​

The documented order is:

singleton behavior
      ↓
TCP context
      ↓
TLS context
      ↓
WebSocket context
      ↓
Gateway context
      ↓
2 live shards
      ↓
progressive scaling

Reason: transport layers depend on one another.

Replacing all global state in one change would produce a much larger debugging search space.

Prove two before scaling to 128​

The local pool supports up to 128 slots.

That is an architectural capacity target, not an acceptance shortcut.

Reason: the transition from one context to two contexts is where accidental global reuse becomes visible.

If two shards cannot prove independent:

  • source ports
  • TCP state
  • TLS workspaces
  • WebSocket workspaces
  • Discord workspaces
  • heartbeats
  • reconnect state

then increasing the count only multiplies the failure.

Concurrency bugs get easier before they get bigger

The smallest case that contains concurrency is often the best case for proving isolation.

Separate global shard count from local capacity​

A bot may have more global shards than one physical PXE node can or should own.

The runtime therefore keeps separate values for:

global total shards
local range start
local range count

Reason: physical capacity and Discord's logical shard topology are different problems.

This also allows multiple nodes to participate in one global plan.

Use a small top-level descriptor plus lazy workspaces​

Each local shard descriptor is 256 bytes, but large protocol workspaces are separately allocated.

Reason:

  • TLS state is much larger than TCP control state
  • inactive slots should remain cheap
  • resets differ by protocol layer
  • pointer identity is useful for isolation tests
  • allocation strategy can evolve without changing every descriptor offset

See Shard Context & Memory Layout.

Share machine-level IPv4 state, isolate connection-level TCP state​

All shards on one PXE machine share the NIC, machine IP, and route/gateway environment.

Each shard owns its own TCP source port, sequence space, callback state, counters, and reliability storage.

Reason: duplicating the entire IP stack per shard would waste memory and complicate hardware ownership, while sharing TCP connection state would be incorrect.

Keep TLS private per shard​

Each active shard gets a large private TLS workspace.

Reason: TLS record sequences, randoms, transcript, key material, and handshake buffers are connection-specific.

A shared TLS workspace would be unsafe even if access were serialized because multiple connections need state to persist concurrently.

Bind upper layers through explicit context callbacks​

Shard TCP owns an RX callback that can target the correct shard TLS context.

Reason: context identity should travel through the stack instead of being rediscovered through globals.

This makes data flow:

shard → TCP → TLS → WebSocket → Gateway

rather than:

packet → global parser → guess current shard

Keep lifecycle recovery at shard scope​

A failed TCP/TLS context puts only that shard into backoff.

Reason: one connection should not reboot the entire PXE machine or destroy unrelated Gateway sessions.

The lifecycle layer resets only the failing shard's transport/protocol memory.

Preserve resume state selectively​

A transport reconnect can preserve Discord sequence/session state only when both are valid.

Reason: lower-layer failure does not always invalidate the logical Gateway session, but blindly reusing stale session state is also incorrect.

The reset logic therefore distinguishes resumable Gateway state from transport state that must always be rebuilt.

Stagger initial heartbeats​

Shard IDs are used to spread the first heartbeat across the advertised heartbeat interval.

Reason: a node starting many shards at once should not create an artificial synchronized burst.

The normal recurring heartbeat cadence still follows the Gateway interval.

Coordinate Identify globally​

Local runtime scheduling understands Identify buckets, but multi-node operation adds a controller-level claim.

Reason: Discord's max_concurrency belongs to the bot's global shard set.

If every physical node independently believes it may use bucket 0, the combined system can violate Discord's limit.

Use lease-based shard ownership​

Controller assignments expire.

Reason: permanent ownership is unsafe under partitions.

If a node disappears, another node eventually needs to take ownership. The old node must also stop believing it owns the shard when its lease is no longer valid.

Leases make ownership a time-bounded claim.

Use deterministic contiguous controller ranges​

The controller sorts nodes and assigns capacity-weighted contiguous ranges.

Reason:

  • bare-metal local plans become start + count
  • output is predictable
  • changes are easy to inspect
  • node ownership is human-readable

A sophisticated distributed scheduler is not required for the current milestone.

Increment plan generation only on assignment change​

Repeated check-ins do not imply a new logical plan.

Reason: a generation should describe ownership topology, not HTTP request count.

This gives nodes a meaningful way to identify whether their assignment actually changed.

Keep the controller outside the bare-metal runtime​

The controller is Go/HTTP/JSON, while the PXE runtime is assembly-heavy and hardware-facing.

Reason: they solve different problems.

The bare-metal node owns:

  • transport
  • Discord state
  • local scheduling
  • hardware diagnostics

The controller owns:

  • node membership
  • leases
  • assignment policy
  • global Identify coordination
  • routing lookup

Combining them would make the bare-metal binary larger and the control plane harder to change.

Fail fatal Gateway configuration instead of retrying forever​

Some close codes mean the shard configuration is invalid.

Reason: retry is appropriate for a transient network failure, not for a condition that cannot become valid without operator/configuration change.

A FATAL state keeps the problem visible.

Keep the PoC account credential inert​

The repository contains a local-only proof-of-concept credential boundary, but the live Discord transport remains bot-token based.

Reason: development of event/listener structure does not require automating a normal user account against Discord.

The outbound PoC adapter remains disabled.

Treat diagnostics as architecture​

Serial breadcrumbs, shard counters, acceptance commands, and state dumps are not optional developer conveniences.

Reason: in bare-metal systems, observability has to be designed into the runtime.

A subsystem that cannot explain which layer failed is substantially harder to validate on real hardware.

Prefer explicit hardware milestones over vague "works"​

Discord-PXE documents concrete milestones:

singleton hardware-proven
2 isolated live shards
4
8
16
32
64
128
multi-node leases
resume / handoff
authenticated control plane

Reason: a project with firmware, networking, crypto, and distributed ownership has too many independent dimensions for one binary success flag.

Source​