Roflin
2893e7f6fb
Compat TODO item 1. S0_MR now sets the MACRAW MF bit, so the W5100 delivers
frames addressed to SHAR or broadcast only -- what the real BBA does, since
libogc2 configures NCRB = AB|CA|2_PACKETS_PER_INT (PR=0, PM=0, AB=1).
NCRB.PR (bit 0) clears MF if the GC asks for promiscuous, sampled when socket 0
is OPENed, so a later PR change applies on the next NCRA reset.
This was not a performance issue, it was a HANG. Swiss sat on "Initialising
Network" forever. libogc2's bba_interrupt() loops
while (status) { clear IR; start_rx; status = IR & IMR; }
which only exits when IR & IMR reaches zero. Unfiltered, every frame on the
segment set RI, re-asserting faster than the loop could clear it, so it never
exited. Unplugging the ethernet cable let Swiss run all the way through and
then fail DHCP for want of a server -- a one-minute experiment that identified
it after a lot of theorising about descriptors and interrupts.
Filtering against SHAR turned out to be safe for this path after all. The
earlier worry was that MF filters on SHAR, which we write only at init, so a
driver WRITING PAR0-5 would leave it stale and MF would drop everything.
libogc2 READS the MAC out of PAR0-5 instead, so SHAR -- written from reg.par in
that same init -- cannot go stale underneath it. Broadcast still passes MF and
a DHCP OFFER unicast to our MAC passes too.
Left open and recorded: refreshing SHAR on a PAR0-5 write plus a socket
re-open, for a driver that does write it (the official SDK driver is unread).
The W5500 back-end is unchanged, being the unused alternate path; it would need
the same Sn_MR treatment if revived.
T1 expects S0_MR = MACRAW|MF in the init sequence, T1b checks promisc high
re-inits with MF clear. Mutation-tested: never setting MF fails both.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1708 lines
93 KiB
Markdown
1708 lines
93 KiB
Markdown
# CLAUDE.md
|
||
|
||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||
|
||
## Project: GC BBA FPGA Replacement
|
||
|
||
Replace the GameCube Broadband Adapter (DOL-015 / MX98730EC) with an iCEbreaker
|
||
FPGA (Lattice iCE40UP5K) written in Amaranth HDL. The FPGA emulates the BBA
|
||
register interface over the GameCube EXI bus and bridges to a WIZnet ethernet
|
||
chip for real 100BASE-TX ethernet — default **W5100** (indirect parallel bus,
|
||
reaches the EXI throughput ceiling) or **W5500** (SPI Pmod, simpler wiring but
|
||
~12 Mbit/s). GC software (Swiss homebrew) sees an identical BBA. See "W5100 vs
|
||
W5500 ethernet back-end".
|
||
|
||
---
|
||
|
||
## Development Environment
|
||
|
||
**Preferred:** Use the devcontainer (`.devcontainer/`) which includes Python 3.12,
|
||
`nextpnr-ice40`, and `fpga-icestorm` pre-installed.
|
||
|
||
**Windows host + WSL2 devcontainer — USB flashing setup:**
|
||
1. Install `usbipd-win` (https://github.com/dorssel/usbipd-win/releases)
|
||
2. Run `.devcontainer/attach-icebreaker.ps1` as Administrator before opening the devcontainer
|
||
3. The devcontainer runs `--privileged` AND bind-mounts `/dev/bus/usb` so libusb
|
||
(`iceprog`, `pyftdi`) can see the FT2232H; the `vscode` user flashes via
|
||
passwordless `sudo` (the USB nodes are root-owned). If you change
|
||
`.devcontainer/`, **rebuild the container** for it to take effect.
|
||
4. Flashing the re-bba-rb board (bitstream + FT2232H EEPROM) is documented as the
|
||
`flash-re-bba-rb` skill — see `.claude/skills/flash-re-bba-rb/SKILL.md`.
|
||
|
||
**Local venv (outside devcontainer):**
|
||
```bash
|
||
python -m venv .venv
|
||
source .venv/bin/activate # Windows: .venv\Scripts\activate
|
||
pip install -r requirements.txt
|
||
```
|
||
|
||
Yosys is bundled in `amaranth-yosys`; `nextpnr-ice40` and `iceprog` must be
|
||
installed separately (via apt on Linux, or via the devcontainer).
|
||
|
||
---
|
||
|
||
## Commands
|
||
|
||
**Build and flash the iCEbreaker (must run from workspace root):**
|
||
```bash
|
||
python rebbarb/rebbarb.py
|
||
```
|
||
Runs synthesis (yosys), place-and-route (nextpnr-ice40), and flashes via `iceprog`.
|
||
Set `ICEPROG=/path/to/iceprog` env var to override the binary location.
|
||
Note: `rebbarb/rebbarb.py` builds a 36 MHz LED blink demo. The BBA
|
||
implementation (`exi_bba/`) clocks the SPI bit engine from the GC's EXI clock
|
||
pin itself and everything else from `sync` @ 24 MHz (internal HFOSC); there is
|
||
no PLL.
|
||
Synthesize/flash the real design with `python -m exi_bba.synth [--flash]`.
|
||
|
||
**Run a simulation:**
|
||
```bash
|
||
# New-API testbench style (preferred for new code):
|
||
python rebbarb/toggle_button.py # writes ToggleButton.vcd
|
||
python rebbarb/pulse_button.py # writes PulseButton.vcd
|
||
|
||
# Old-API process style (reference only, do not replicate in new code):
|
||
python examples/amaranth_cdc.py # CDC primitives demo
|
||
python examples/async_fifo.py # AsyncFIFO behaviour
|
||
python examples/icebreaker_fifo.py # iCEbreaker-specific FIFO (Verilog dump)
|
||
```
|
||
Open VCD output with `gtkwave`. Simulations are the primary testing mechanism —
|
||
there is no separate test runner.
|
||
|
||
**Clock sources:** there is no PLL and no external crystal any more. `sync`
|
||
comes from the internal `SB_HFOSC` ÷2 = 24 MHz; the EXI bit engine is clocked
|
||
by the GC's own EXI clock pin. Pin 35 (the old 12 MHz input) and the PLL hard
|
||
macro are both free.
|
||
|
||
---
|
||
|
||
## Current Implementation State
|
||
|
||
The `exi_bba/` module tree is **fully implemented** with simulation testbenches.
|
||
All modules elaborate without errors and pass their unit tests, including
|
||
`bba_top` at the real 27 MHz EXI rate (`EXI_HALF=1`).
|
||
|
||
✅ **FULL TEST SUITE PASSES ON A REAL CONSOLE (2026-10-09)** — every suite in
|
||
`gc_bba_test`: Identity (device ID `04 02 02 00`, HIPR, NWAYS), Registers,
|
||
Reset/control, EXI integrity (speed sweep `ok=[1,1,1,1,1,1]`, per-speed dump
|
||
clean at all six indices, raw reg dump `0x17` in slot 0), Ethernet, and
|
||
SPRAM/DMA. The EXI front end runs at the real 27 MHz hardware rate.
|
||
|
||
(The ID tests simply PASS or FAIL now, in any order, at any point in a run —
|
||
see "The device ID is answerable at any time" below. The old SKIP machinery,
|
||
`iface_alive()` and `g_ncra_reset_done` are all gone.)
|
||
|
||
**Timing closes on EVERY seed (re-measured 2026-10-10).** `python -m
|
||
exi_bba.synth --seeds 8`:
|
||
|
||
| domain | target | result |
|
||
|---|---|---|
|
||
| `clk` (sync, HFOSC) | 24 MHz | 28.1–30.8 MHz — **PASS on all 8 seeds** |
|
||
| EXI pin domain | 27 MHz | 57.7–66.8 MHz — **PASS on all 8 seeds** |
|
||
|
||
Best seed **6**: `clk` 30.83 MHz, EXI 57.74 MHz. Flash `build/seed<N>/top.bin`.
|
||
|
||
Measured after making the SPRAM response combinational (the `+1` descriptor
|
||
fix), which put SPRAM's output → byte mux → `tx_fifo` write data on `sync`'s
|
||
combinational path — the binding domain. It cost nothing measurable: `clk` was
|
||
28.5–30.7 before. The EXI figures replace a stale 41.6–49.7 in this table,
|
||
which predated the full-cycle-early MISO launch.
|
||
|
||
⚠️ **fmax is NOT the only constraint — MISO clock-to-pad matters more.** MISO
|
||
is source-synchronous: it leaves on the FALLING edge and the GameCube samples
|
||
it on the next RISING edge, so the path must fit a **half period (18.5 ns at
|
||
27 MHz)** minus global-buffer insertion, pad, connector and the GC's own
|
||
setup. A build measuring ~10.5 ns of that *internally* read every byte one bit
|
||
late at 27 MHz while reading correctly at 16 MHz — with every seed reporting
|
||
≥1.29× fmax margin. `synth.py` therefore parses this delay and ranks seeds by
|
||
it when it exists.
|
||
|
||
**MISO is now driven from an `SB_IO` output register** (`synth.py`), so
|
||
clock-to-pad is the IO tile's own delay and never enters the fabric — the
|
||
sweep prints `IOB (none)` because no `negedge → <async>` path exists any more.
|
||
Getting there also meant removing the EXI_J4 pad merge and the
|
||
`Mux(cs_n, 1, …)` LUT from the data path (see those notes). The EXI fmax
|
||
numbers dropped (62 → 49 MHz) because the posedge→negedge setup path into the
|
||
IOB register is now counted against that domain at a half-period constraint;
|
||
41.6 MHz worst case is still 1.5× over the 27 MHz target.
|
||
|
||
⚠️ **MISO is TRI-STATED when CS is high** (held high by the pad's PULLUP),
|
||
not actively driven. EXI MISO is a SHARED bus — SP1 is on channel 0 with
|
||
memory-card slot A — so driving it while deselected was wrong.
|
||
|
||
This replaces the long-standing "closes on a minority of seeds" problem, which
|
||
was entirely an artefact of the PLL-fed 54 MHz `capture` domain. Removing that
|
||
domain (the bit engine is now clocked by the EXI pin — see "Key Architecture
|
||
Decisions") took the pass rate from **2 of 8 to 8 of 8** and made the old
|
||
capture-domain tuning levers moot: the Gray-code CDC loop that was the binding
|
||
path, the pin co-location experiment, and the proposed ping-pong TX CDC are all
|
||
obsolete. Do not reintroduce them.
|
||
|
||
⚠️ **The binding constraint has FLIPPED.** `clk` (sync, 1.10–1.34× margin) is
|
||
now tighter than the EXI domain (1.69–2.39×). Rank seeds by **worst relative
|
||
margin**, not by the EXI number — `synth.py` does this. Added logic now
|
||
threatens `sync`, not the EXI front-end.
|
||
|
||
⚠️ **Still sweep.** `clk` at 26.41 MHz on seed 5 is only 10% over target, so a
|
||
seed can still land uncomfortably close even though all 8 pass today.
|
||
|
||
### `exi_bba/` module status
|
||
|
||
| Module | File | Tests pass |
|
||
|---|---|---|
|
||
| `BBATop` | `exi_bba/bba_top.py` | ✅ EXI integration + full W5100→SPRAM→GC RX loop, at `EXI_HALF=1/2/8`; synth PASS |
|
||
| `ExiCapture` | `exi_bba/exi_capture.py` | ✅ rx/tx byte-stream + over-push/flush |
|
||
| `SPIMode3Slave` | `exi_bba/spi_mode3_slave.py` | ✅ 7 checks (rx/tx edges, tx_load, MISO idle) — **mutation-tested**: flipping either clock edge fails it |
|
||
| `BBARegisterFile` | `exi_bba/bba_register_file.py` | ✅ 7 tests (real 4-byte header, proactive push + DMA stream) |
|
||
| `SPRAMArbiter` | `exi_bba/spram_arbiter.py` | ✅ 3 tests |
|
||
| `RXFrameAssembler` | `exi_bba/rx_frame_assembler.py` | ✅ 3 tests |
|
||
| `TXFrameDrain` | `exi_bba/tx_frame_drain.py` | ✅ 2 tests |
|
||
| `W5100ParallelMaster` | `exi_bba/w5100_parallel_master.py` | ✅ MACRAW init/TX/RX (T1–T5) + socket-N UDP send/receive (U1–U4) vs bus model, incl. ring wrap — **default eth back-end** |
|
||
| `W5500SPIMaster` | `exi_bba/w5500_spi_master.py` | ✅ init/TX/RX vs SPI-slave model (alt back-end) |
|
||
| `StatusPanel` | `exi_bba/status_panel.py` | ✅ 6 tests (heartbeat, stretched activity LEDs, debounced buttons, freeze) |
|
||
| `UARTConsole` | `exi_bba/uart_console.py` | ✅ 7 tests (event log + 'r' reinit) — event-logger console (`--console` build) |
|
||
| `UARTShell` | `exi_bba/uart_shell.py` | ✅ 8 tests (help/unicast/broadcast/reply-print/timeout/badip/backspace) — interactive UDP bring-up shell (**default build**) |
|
||
| `EEPROMModel` | `exi_bba/eeprom_model.py` | ✅ 4 tests |
|
||
|
||
**Bring-up status panel (optional):** `BBATop(status_panel=True)` adds a
|
||
`StatusPanel` driving onboard iCEbreaker LEDs + button (dedicated pins, so it
|
||
coexists with EXI + W5100). `synth.py` enables it: **LEDG=heartbeat**,
|
||
**LEDR=EXI activity** (the GC is talking), **RGB red=rx / green=tx / ready**
|
||
(via `SB_RGBA_DRV` on pins 39/40/41), **BTN_N=manual re-init**. All 5 panel
|
||
LEDs are now mapped on the iCEbreaker. For this build's timing see the seed
|
||
table above — it does NOT close on every seed.
|
||
|
||
On the **re-bba-rb board** the third RGB element (`led[4]`, "ready", pin 41) is
|
||
**D13 = YELLOW** (LCSC C2287), not blue — `synth.py`'s comment still calls it
|
||
blue, which describes the iCEbreaker dev board, not this PCB. D11=RED (rx) and
|
||
D12=GREEN (tx) are correct on both.
|
||
|
||
**Ethernet back-end is selectable:** `BBATop(eth="w5100")` (default — indirect
|
||
parallel bus, reaches the ~27 Mbit/s EXI ceiling) or `BBATop(eth="w5500")` (SPI,
|
||
~12 Mbit/s). Both masters expose the identical tx/rx/init/par streaming
|
||
interface; only the physical pins differ. See "W5100 vs W5500" below.
|
||
|
||
### Run all module testbenches (from workspace root)
|
||
```bash
|
||
python -m exi_bba.spi_mode3_slave
|
||
python -m exi_bba.exi_capture
|
||
python -m exi_bba.bba_register_file
|
||
python -m exi_bba.spram_arbiter
|
||
python -m exi_bba.rx_frame_assembler
|
||
python -m exi_bba.tx_frame_drain
|
||
python -m exi_bba.w5100_parallel_master # T1-5 MACRAW + U1-4 socket-N UDP tx/rx
|
||
python -m exi_bba.w5500_spi_master
|
||
python -m exi_bba.status_panel # 6 tests: heartbeat/activity/buttons
|
||
python -m exi_bba.uart_console # 7 tests: event log + 'r' reinit
|
||
python -m exi_bba.uart_shell # 8 tests: shell cmds + reply/timeout
|
||
python -m exi_bba.eeprom_model
|
||
python -m exi_bba.bba_top # end-to-end EXI integration test (W5100 RX loop)
|
||
EXI_HALF=1 python -m exi_bba.bba_top # ...at the REAL 27 MHz rate — always do this
|
||
|
||
```
|
||
|
||
### UART shell + UDP bring-up test (default flash build)
|
||
|
||
`BBATop(uart_shell=True)` adds an **interactive UART command shell** that shares
|
||
the FT2232H channel-B UART pins with (and replaces) the event-log console, and
|
||
drives a **UDP send/receive test on a second W5100 socket** (default socket 3)
|
||
alongside the BBA's socket-0 MACRAW path. `synth.py` builds it **by default**
|
||
(pass `--console` for the old event-logger instead). EXI/BBA keeps priority: the
|
||
UDP test is the lowest-priority branch in the W5100 master's bus arbiter, so a
|
||
GC frame always preempts a queued send/poll.
|
||
|
||
Shell (115200 8N1, `rebbarb> ` prompt):
|
||
- `help` — list commands
|
||
- `udp unicast <ip> [msg]` — send a UDP datagram to `<ip>:dst_port` (the W5100
|
||
ARPs the target), then wait (bounded) for a reply
|
||
- `udp broadcast [msg]` — same to 255.255.255.255 (limited broadcast; no ARP)
|
||
- after a send, the shell prints `sent`, waits up to `reply_timeout_cycles`
|
||
(~1 s) for a UDP reply on the socket, and prints `rx <payload>` or `timeout`
|
||
(so it never hangs). Example: `udp broadcast Hello World` → a responder
|
||
unicasts back → `rx <reply>` on the console.
|
||
|
||
Config is build-time via `synth.py` env vars (the board's own IP identity —
|
||
match your LAN): `UDP_SOCKET` (1–3, default 3), `UDP_SRC_IP`, `UDP_SUBNET`,
|
||
`UDP_GATEWAY`, `UDP_DST_IP`, `UDP_SRC_PORT`, `UDP_DST_PORT`. Receive on the PC
|
||
with `nc -ul <port>` or `sudo tcpdump -n udp port <port>`.
|
||
|
||
**MACRAW + UDP coexistence is datasheet-confirmed** (W5100S §4.6: *"MACRAW Mode
|
||
SOCKET 0 does not receive any Data Packet for other SOCKET"*; feature list:
|
||
*"Support 4 independent SOCKETs simultaneously"*). All 4 sockets get a 2 KB
|
||
RX + 2 KB TX buffer at reset (`RMSR/TMSR=0x55`), so socket-3 frames are stored
|
||
in the chip independently — no extra FPGA buffer needed. Register addresses are
|
||
still datasheet-from-memory — **confirm at hardware bring-up**.
|
||
|
||
### ✅ Self-update over the BBA works on hardware (2026-10-09)
|
||
|
||
Press **Z** in `gc_bba_test` and run `sudo ./scripts/send-dol.py <iface>
|
||
<file.dol>` on a Linux host on the same segment. The build is received over the
|
||
adapter, written to the SD card, **read back and verified**, rotated in (the
|
||
previous build is kept as `<name>.prev.dol`), and then the app boots
|
||
**`/boot.dol`** — a known-good Swiss. Nothing received is ever launched
|
||
directly, so a broken test build costs nothing. Proven end to end on console:
|
||
300076 bytes in 294 blocks, verify OK, Swiss boots, and new builds are now
|
||
delivered this way instead of by SD card.
|
||
|
||
**Measured 2026-10-09: 300908 bytes in 6.4 s (0.38 Mbit/s), `txto=0`.**
|
||
|
||
Getting there was all round-trip cost, not bandwidth — stop-and-wait means one
|
||
block per round trip. Three things mattered, in order of effect:
|
||
|
||
| change | result |
|
||
|---|---|
|
||
| host `--timeout` 0.3 → **0.05 s** | 40.1 s → 6.4 s |
|
||
| `send_ack` TI poll 200 → **8** | 200 EXI transactions ≈ 12 ms, spent only when TI was slow — i.e. exactly when the host was closest to timing out |
|
||
| block 256 → **1024** | 1166 blocks → 294 |
|
||
|
||
⚠️ **The timeout result is counter-intuitive and worth remembering:** raising it
|
||
0.1 → 0.3 barely changed the retry count (110 → 95) but tripled the cost of
|
||
each, so it made the transfer 2.4× SLOWER. Short is right here, because the GC
|
||
re-ACKs a duplicate block immediately — a premature retry costs one timeout
|
||
and nothing else.
|
||
|
||
⚠️ `txto=0` means every ACK was handed to the W5100 with TI set, so the drain
|
||
and the EXI side are clean. **TI is not proof of transmission**, so a residual
|
||
retry rate points past the hand-off (the chip, or the wire). Whether the
|
||
remaining misses are lost or merely late is NOT established — the retry count
|
||
is roughly timeout-independent, which suggests loss, but the 6.4 s result is
|
||
hard to reconcile with ~95 genuine losses. Get the retry count alongside
|
||
`txto` before theorising.
|
||
|
||
Details and the recovery ladder are in `gc_bba_test/README.md`. Key points:
|
||
- **Raw ethertype `0x88B5`, not IP** — no ARP, no IP stack, no libogc network
|
||
driver competing for the EXI channel. It reuses only `bba_tx_bytes()` and the
|
||
SPRAM ring, so each update also soak-tests the BBA data path.
|
||
- **It cannot be driven from the devcontainer.** Raw ethernet is link-local, so
|
||
the sender must run on a host with the physical NIC (the container has a
|
||
veth and the frames go nowhere).
|
||
- **Stop-and-wait is load-bearing** — see the RBFI note; one frame in flight
|
||
means the ring is never contended.
|
||
- `bba_tx_bytes()` streams in **8-byte** chunks, one EXI transaction each.
|
||
Writing byte-by-byte cost a full EXI_Lock/Select/Deselect/Unlock per byte —
|
||
~60 transactions per ACK — and that latency, not any transmit failure, is
|
||
what made ~⅓ of blocks retry (`txto=0` on hardware: TI always set).
|
||
|
||
⚠️ **The chunk size is bounded by `tx_bytes`, which is 16 deep.** Within one
|
||
transaction the bytes arrive back-to-back at EXI speed, far faster than
|
||
`TXFrameDrain` starts consuming, and `BBARegisterFile` silently drops bytes
|
||
when `w_rdy` is low because SPI cannot back-pressure. A 64-byte chunk
|
||
overran the FIFO, the frame went out truncated and the host saw no valid ACK
|
||
at all. Byte-by-byte only worked because the per-byte EXI overhead happened
|
||
to pace the drain. 8 is half the FIFO depth, so it cannot overflow even if
|
||
the drain consumes nothing for a whole transaction. **Do not raise it
|
||
without deepening `tx_bytes` or giving the TX path real back-pressure.**
|
||
|
||
### ✅ FIXED 2026-10-10 — ring descriptor at page offset +1 (tx FIFO underrun)
|
||
|
||
**The fix is one deleted register.** `SPRAMArbiter` latched the SPRAM word
|
||
into `rsp_buf` and raised `exi_rsp_valid` a cycle later; it now drives both
|
||
COMBINATIONALLY in the cycle SPRAM's output is valid. `BBARegisterFile` was
|
||
already taking the response the cycle it arrived, so the buffer bought nothing
|
||
and cost 42 ns inside a deadline that was short by about that much.
|
||
`bba_top` **T9 is now 6/6 at `EXI_HALF=1`, `2` and `8`**, and nothing else in
|
||
the design changed — the register file is untouched.
|
||
|
||
⚠️ **Not yet confirmed on hardware.** `netboot`'s reader still probes both
|
||
offsets; leave that in until a console run shows `+1=0`.
|
||
|
||
Original symptom: on console the SPRAM ring descriptor was at page+0 for ~23%
|
||
of frames and page+1 for ~77% (`desc@+0=100 +1=343` in one run), **varying
|
||
frame to frame**.
|
||
|
||
What has been ruled out:
|
||
- **Not ring overflow** — `lapped=0` with RBFI active.
|
||
- **Not the two dropped-write handshake bugs** in `RXFrameAssembler` (fixed,
|
||
mutation-tested; T2 would catch a recurrence).
|
||
- **Not write-port contention** — `eth_wr_ready` is asserted combinationally
|
||
whenever `eth_wr_valid` is high (ETH has absolute priority in
|
||
`SPRAMArbiter`), so the write port is never stalled. T5 models a stall
|
||
anyway and the layout is unchanged.
|
||
- **Not the late-SPRAM-response leak** — guarding on `outstanding != 0` and
|
||
clearing `sp_pending` in `HEADER0` fixed a real latent bug but left the
|
||
ratio unchanged.
|
||
|
||
✅ **REPRODUCED IN SIMULATION 2026-10-09** — `bba_top` **T9** performs EXI DMA
|
||
reads from the ring **while** the ethernet side writes frames into it, which no
|
||
earlier test did (T6 waits for the frame to finish first, so reads and writes
|
||
never overlapped).
|
||
|
||
```
|
||
EXI_HALF=8 T9 concurrent DMA reads: 6/6 byte-exact
|
||
EXI_HALF=2 T9 concurrent DMA reads: 6/6 byte-exact
|
||
EXI_HALF=1 T9 concurrent DMA reads: 1/6 byte-exact <-- the hardware rate
|
||
```
|
||
|
||
1-of-6 good is the same order as the console ratio (`desc@+0=100 +1=343`,
|
||
23% good), and the corruption is exactly the hardware symptom — **one extra byte inserted
|
||
at the front**, shifting everything:
|
||
|
||
```
|
||
expected: 00 00 00 0C DE AD BE EF
|
||
got: 00 00 00 00 0C DE AD BE
|
||
```
|
||
|
||
An INSERTED byte means an extra push into the tx FIFO, not a corrupted one, so
|
||
the fault is a count mismatch somewhere in the request/response path — not the
|
||
SPRAM data. It only appears at 27 MHz because the read must finish inside the
|
||
protocol's turnaround window while ETH (which has absolute arbiter priority) is
|
||
preempting it.
|
||
|
||
T9 is now the regression test for the fix: it must stay 6/6 at all three
|
||
rates. Do not relax it.
|
||
|
||
**MECHANISM, established by reading the frame body instead of the descriptor.**
|
||
Reading from 0x104 (first byte a known `0xDE`) rather than 0x100 (first byte
|
||
`0x00`, indistinguishable from a stale byte) shows what actually happens:
|
||
|
||
```
|
||
good: DE AD BE EF 01 02
|
||
bad: DE DE AD BE EF 01 <-- the FIRST BYTE IS DUPLICATED
|
||
```
|
||
|
||
Not an inserted stale byte — **a tx FIFO UNDERRUN.** When the FIFO runs dry for
|
||
one byte-time the bit engine reloads the stale FIFO head and sends the previous
|
||
byte again, shifting everything after it.
|
||
|
||
The register file must sustain one byte per **296 ns** at 27 MHz, and the round
|
||
trip is request → `spram_req` SyncFIFO → arbiter (where ETH writes have
|
||
absolute priority and preempt it) → SPRAM (1 cycle) → response → push → tx
|
||
`AsyncFIFO` CDC (2–3 EXI cycles, 74–111 ns). That is marginal even idle and
|
||
over budget under a frame write. Only ONE request may be in flight, so the
|
||
latency is never hidden.
|
||
|
||
⚠️ **FOUR FIXES TRIED AND FALSIFIED — do not repeat them:**
|
||
|
||
| attempt | result |
|
||
|---|---|
|
||
| **Fair (round-robin) arbitration** instead of strict ETH priority | still 1/6. Bounds the wait to ~2 cycles but does not close the gap, and it breaks `spram_arbiter` T3. |
|
||
| **Deepen `tx_fifo` 2 → 8** for cushion | **0/6**, with leading zeros. The engine drains leftovers with ONE POP PER HEADER BYTE, so a deeper FIFO leaks stale bytes into the data phase. `tx_depth=2` is deliberate — keep it shallow. |
|
||
| **Allow 2 outstanding requests** to pipeline the latency | **0/6 at every rate**, including `HALF=8` which previously passed. With no response FIFO, responses must be consumed the cycle they are valid; two in flight loses bytes. The existing comment was right. |
|
||
| **Return both SPRAM bytes per request** (16-bit response + `exi_rsp_odd`, two pushes per round trip) | **WORSE: 6/6 / 3/6 / 1/6** at `HALF=8/2/1` against the baseline's 6/6 / 6/6 / 1/6 — it broke `HALF=2`, left the `DE DE` duplication at `HALF=1`, and introduced a new bug where the FIRST read after an idle period returns all zeros. Reverted. |
|
||
|
||
Every one of those tried to carry MORE DATA or WAIT LESS per round trip. None
|
||
of them shortened the round trip, which is what the deadline is made of.
|
||
|
||
**What the measurement showed, and why it ended the guessing.** A counter in
|
||
`ExiCapture` recording `tx_load & ~tx_fifo.r_rdy` per byte boundary, with a
|
||
shift-register history of which pop underran:
|
||
|
||
```
|
||
[0] DE AD BE EF 01 02 ur=1 pops=10 hist=0010000000 <- good
|
||
[1] DE DE AD BE EF 01 ur=2 pops=10 hist=0011000000 <- bad
|
||
```
|
||
|
||
Ten pops per read (4 header + 6 data). Good and bad reads BOTH underrun at
|
||
pop 3 — harmless, mid-header, nothing is listening. A bad read adds **pop 4**,
|
||
the pop that loads **data slot 0**. So the late byte is the FIRST data byte,
|
||
not a later one, which kills every "prefetch further ahead" idea: byte 0's
|
||
request cannot be issued before `RHDR_LO`, because that is where the low
|
||
address byte arrives.
|
||
|
||
That makes the deadline pure arithmetic, with no contention in it at all:
|
||
|
||
| stage | cost |
|
||
|---|---|
|
||
| rx `AsyncFIFO` CDC (address byte → FSM) | ~125 ns |
|
||
| arbiter accepts the request | 42 ns |
|
||
| SPRAM read | 42 ns |
|
||
| **response register (`rsp_buf`)** | **42 ns** |
|
||
| tx `AsyncFIFO` CDC | ~110 ns |
|
||
| **total** | **~361 ns vs a 296 ns byte time** |
|
||
|
||
Over budget by ~65 ns *idle*. That is why fair arbitration changed nothing —
|
||
ETH contention was never the cause, only what pushed a marginal path over on
|
||
some frames. Deleting `rsp_buf` removes 42 ns and the CDCs' phase jitter covers
|
||
the rest.
|
||
|
||
⚠️ **Two things were implemented, measured, and then REMOVED as unnecessary** —
|
||
do not add them back without evidence:
|
||
- **issuing byte 1's request during the pad byte** (priming the pipeline), and
|
||
- **two requests in flight with a 3-deep response hold** (`SP_LIMIT=2`).
|
||
|
||
Ablation: with the `rsp_buf` deletion in place, T9 is 6/6 at every rate with
|
||
BOTH removed, with either alone, and with the register file reverted
|
||
completely. They were real pipelining and they were not the constraint. They
|
||
also cost logic on `sync`, which is the binding timing domain.
|
||
|
||
⚠️ And a trap worth knowing if anyone does revisit multiple outstanding
|
||
requests: `outstanding` was incremented by the FSM and decremented by the
|
||
response handler as two separate `m.d.sync` assignments. Amaranth resolves
|
||
those by PRIORITY, not arithmetic — the later one wins outright and the
|
||
increment is silently discarded. With one request in flight they can never
|
||
coincide; with two they coincide constantly, the count drifts down, and
|
||
responses get dropped by the `outstanding != 0` guard. Symptom: DMA reads with
|
||
bytes MISSING rather than duplicated (`00 00 00 0C DE EF 01 ...`). Give such a
|
||
counter exactly one writer fed by comb intent signals.
|
||
|
||
### ⚠️ Simulation rate — always re-run at the hardware rate
|
||
|
||
`bba_top`'s testbench clocks EXI at `27 MHz / EXI_HALF`. The default
|
||
`EXI_HALF=8` gives **3.4 MHz** — eight times slower than a real GC — which is
|
||
fast to simulate but hides every latency bug. **Run `EXI_HALF=1 python -m
|
||
exi_bba.bba_top` before trusting any timing-related change.** The current
|
||
design passes at `EXI_HALF=1`, `2` and `8`.
|
||
|
||
Historically **every testbench drove MOSI many ticks before toggling the
|
||
clock**, so MOSI was stable at BOTH edges and the sampling-edge convention was
|
||
structurally undetectable — that is how a Mode 2/Mode 3 mix-up reached silicon
|
||
with a fully green suite. Both `bba_top` and `spi_mode3_slave` now model a
|
||
strict Mode 3 master (change MOSI on the falling edge, sample MISO in the
|
||
clock-low phase), and `spi_mode3_slave`'s testbench is **mutation-tested**:
|
||
flipping either edge in the RTL makes it fail.
|
||
|
||
### ✅ FIXED 2026-10-03 — edge convention, and MISO latency
|
||
|
||
Two bugs, both now resolved by the pin-clocked rewrite:
|
||
|
||
**1. Wrong sampling edge (was Mode 2).** These notes once said "Mode 3
|
||
(CPOL=1, CPHA=1)" and then described sampling MOSI on the **falling** edge.
|
||
Idle-high + sample-on-falling is **Mode 2**. Real Mode 3 samples on the second
|
||
edge — the **rising** one — and that is what the GameCube does: it DRIVES MOSI
|
||
on the falling edge, so sampling there latched the previous bit. Symptom:
|
||
every byte one bit late in both directions at every clock speed; the `0x80`
|
||
register-space header byte read as `0x00`, which decodes as a command-space
|
||
device-ID query, so NWAYS reads returned the device ID. Raw 8-slot dump after
|
||
an FPGA reset: `00 00 02 01 01 00 00 00` = `04 02 02 00 >> 1`.
|
||
|
||
**2. MISO ~2 bits late at 27 MHz.** With MOSI fixed, register reads were
|
||
correct up to 4 MHz and failed from 8 MHz (hardware sweep: `ok=[1,1,1,0,0,0]`);
|
||
at index 5 a NWAYS read returned `05 C0` = `0x17 >> 2`. The MISO path was
|
||
delayed ~3 capture ticks ≈ 55 ns — 2 ticks of input `FFSynchronizer` on EXI CLK
|
||
plus 1 to register the shift — against a 37 ns bit time.
|
||
|
||
**Lowering the EXI clock was never an option:** Swiss hardcodes
|
||
`EXI_SPEED_32MHZ` in `exi_select()` (`cube/patches/bba/bba.c`).
|
||
|
||
**The fix was structural, not compensatory:** clock the bit engine from the EXI
|
||
clock pin itself (pin 44 sits at die (6,0) = global buffer input net 6, so it
|
||
drives a real clock network with no board change). This removes oversampling
|
||
and the synchroniser entirely, making MISO clock-to-Q. The two alternatives
|
||
were rejected: output-bit lookahead would have *tuned* a constant that may not
|
||
hold across parts and temperature, and 3× oversampling needs 81 MHz, which this
|
||
part cannot close. `SPIMode3Slave` has no `mode2` parameter any more — there is
|
||
only one correct convention and it is enforced by mutation tests.
|
||
|
||
### ✅ MISO timing SOLVED 2026-10-09 — launch a full cycle early
|
||
|
||
**Confirmed on a console:** speed sweep `ok=[1,1,1,1,1,1]`, per-speed dump
|
||
`0:1700 … 5:1700`, raw reg dump `0x17` in slot 0. Register reads are correct
|
||
at **every** EXI speed index including 5 (27 MHz, the real hardware rate).
|
||
|
||
The engine drives MISO on the **RISING** edge carrying the bit the host will
|
||
sample at the NEXT rising edge, so the output path gets a **FULL period
|
||
(37 ns)** rather than the half period (18.5 ns) a falling-edge launch gives.
|
||
`tx_out` is clocked on the rising edge and loads at `rx_bit == 7`; there is no
|
||
`exiclk_n` domain any more — one clock edge, one phase reference.
|
||
|
||
**Why a half period was not enough, and why no RTL change could fix it.**
|
||
~9 ns was removed from the fabric output path in three steps — the EXI_J4 pad
|
||
merge, then the `Mux(cs_n, 1, …)` LUT, then the entire fabric route by moving
|
||
the output into an `SB_IO` register — and the hardware result stayed **byte
|
||
identical** every time (`0B 80`). Nine nanoseconds out of 18.5 cannot change
|
||
nothing if propagation were the whole story. What remains is irreducible:
|
||
input pad + **global clock-network insertion** + IOB clock-to-Q + output pad,
|
||
plus the host's own setup. So the fix was to double the budget, not shrink the
|
||
delay.
|
||
|
||
`PHASE_METER` is what settled it: `0:8080 1:8080 2:8080 3:8080 4:8080 5:C040`
|
||
— the bit engine's own byte phase, perfect to 16 MHz and one bit late at 27.
|
||
The 8-slot dump read `C0 40 40 40 C0 40 40 40`: a uniform one-bit lag with
|
||
extra highs at bits 0 and 32, where libogc splits the read into two 4-byte
|
||
transfers and MISO tri-states to its pull-up between them.
|
||
|
||
⚠️ **Testbench consequence.** A master must sample MISO in the **CLK-LOW
|
||
phase** (the value latched ON the rising edge). Sampling after raising the
|
||
clock sees the NEXT bit in a zero-delay simulation, so the suite would report
|
||
everything one bit early while hardware was correct. Both `bba_top` and
|
||
`spi_mode3_slave` now sample in the low phase.
|
||
|
||
⚠️ **This relies on the delay landing in a window:** MISO must change late
|
||
enough to clear the host's hold time on the previous bit, and within a full
|
||
period for its setup. The measured insertion delay (~12–20 ns against 37 ns)
|
||
sits comfortably inside. If a LOW speed index ever regresses while 27 MHz
|
||
works, that is the hold edge of the window — the launch is too early — not a
|
||
setup problem.
|
||
|
||
Side effect: EXI-domain fmax rose 41.6–49.7 → **62.1–65.4 MHz**, because the
|
||
setup path into the IOB register became posedge→posedge against a full period
|
||
instead of posedge→negedge against a half. Same doubling, seen internally.
|
||
|
||
### ✅ Device ID answered by the BIT ENGINE (fixed 2026-10-09)
|
||
|
||
`bba_devid` returned `00 04 02 02` on a console — the ID correct but **one
|
||
byte late** — and it failed in its REAL shape (16 µs `gap()`, exactly four
|
||
bytes clocked), so it was not a probe artefact. Register-space reads were
|
||
fine throughout; only command space was affected.
|
||
|
||
**Cause was latency, not phase.** A command-space header is two bytes, so the
|
||
reply must be in the tx FIFO by the **last rising edge of the header
|
||
transfer** — one byte time, 296 ns at 27 MHz. The round trip is rx `AsyncFIFO`
|
||
(~125 ns) + a sync cycle + tx `AsyncFIFO` (~110 ns) ≈ 280–320 ns: marginal by
|
||
construction. `HEADER0` already pushed in the same cycle it consumed byte 0,
|
||
so the FSM side was already as early as possible.
|
||
|
||
⚠️ **The inter-transfer gap cannot help, and this is the non-obvious part.**
|
||
The load happens on that last rising edge and the clock is PAUSED afterwards,
|
||
so no edge exists during the gap for anything to update. That is why
|
||
`bba_devid`'s 16 µs gap made no difference, and why "give it more idle time"
|
||
is not a fix. **Pre-pushing at CS-fall does not work either:** the engine's
|
||
unconditional per-byte pops would eat it, and the header length is not known
|
||
until byte 0 has been decoded — which is the latency in question.
|
||
|
||
**Fix: take the round trip out of the path.** The device ID is a hardwired
|
||
constant and belongs to the physical interface, not the register file, and
|
||
`byte 0 == 0x00` identifies the query exactly (bit 7 clear = command space,
|
||
bit 6 clear = read, reg 0). `SPIMode3Slave` recognises it and answers
|
||
`04 02 02 00` from the EXI clock domain with **no CDC at all** — and since
|
||
2026-10-10 it needs nothing from `sync` either: `id_arm` is tied high (the
|
||
`id_phase` bit and its `FFSynchronizer` are gone). The only remaining use of
|
||
the port is the `PHASE_METER` build, which disarms the fast path so the
|
||
constant `0x80` is not replaced by ID bytes.
|
||
|
||
The register file's ID path was **removed**, not kept as a fallback: left in,
|
||
it pushes bytes a 2-byte header has too few pops to drain — exactly how an
|
||
earlier version made the *next* enumeration come back a slot late.
|
||
|
||
Tests: `spi_mode3_slave` **T7** (reply is `04 02 02 00` with the tx FIFO
|
||
deliberately feeding `0x00`, so only the fast path can answer) and **T8** (it
|
||
is GATED — with `id_arm` low, `0x5A` comes through from the FIFO; without this
|
||
the fast path would hijack address 0 forever and the GC could never read NCRA
|
||
back after its own reset). `bba_register_file` **T1** asserts the FSM pushes
|
||
NOTHING for an ID query.
|
||
|
||
⚠️ **Simulation cannot reproduce this bug.** `bba_top` passed the device ID at
|
||
`EXI_HALF=1` *before* the fix, because the sync domain's latency relative to
|
||
the EXI clock in simulation does not match silicon closely enough to miss a
|
||
296 ns window. T7/T8 prove the fast path works; they do not prove it is
|
||
necessary. Do not "simplify" the ID back into the register file because the
|
||
suite stays green — it will, and hardware will not.
|
||
|
||
### ⚠️ TX bit phase MUST come from the rising-edge counter (fixed 2026-10-03)
|
||
|
||
**Never give the TX shift register its own free-running falling-edge counter.**
|
||
`SPIMode3Slave` takes its byte phase from `rx_bit`, the RISING-edge counter —
|
||
the same edge the host counts bits on. A separate `tx_bit` counting falling
|
||
edges only stays in step while the host emits exactly as many falls as rises,
|
||
and it does not: a transfer that begins on a RISING edge (CS asserted while
|
||
CLK is already low, or the clock gating between two back-to-back host
|
||
transfers) delivers a rise with no matching fall. `tx_bit` then slips one bit
|
||
and never recovers. RX never notices, because the rising count always equals
|
||
the host's bit count — so the header decodes perfectly and only MISO is wrong.
|
||
|
||
**Symptom on console:** every MISO byte shifted right by 2 bits, *identically
|
||
at every EXI speed index*. A NWAYS read returned `05 C0` (= `0x17` with two
|
||
leading zero bits) and the device ID `01 00 80 80`. The speed sweep read
|
||
`ok=[0,0,0,0,0,0]`.
|
||
|
||
**How it was found, after two wrong diagnoses.** It is not propagation delay:
|
||
index 0 is 1 MHz, a 1000 ns bit time, and no output path fails there —
|
||
speed-independence is the tell. It is not the output edge either: the old
|
||
rising-edge build and the new falling-edge build returned *byte-identical*
|
||
`05 C0`, and an edge flip would have moved it. What settled it was a
|
||
measurement, not more theory — see PHASE_METER below.
|
||
|
||
**`PHASE_METER=1 python -m exi_bba.synth`** builds a DIAGNOSTIC,
|
||
NON-FUNCTIONAL bitstream in which the bit engine transmits a constant `0x80`
|
||
for every byte instead of the FIFO head. The register file and both byte FIFOs
|
||
drop out of the MISO path, so whatever the host reads back is purely the
|
||
engine's byte phase: `0x80` aligned, `0x40` one bit late, `0x20` two bits late.
|
||
On hardware it read `0x20` in all 8 slots, which localised the fault to the bit
|
||
engine and ruled out everything upstream in one run. Re-flash a normal build
|
||
afterwards.
|
||
|
||
**Regression test:** `spi_mode3_slave` T5 asserts CS while CLK is LOW, so the
|
||
transaction begins on a rising edge and carries one fewer falling edge. The
|
||
old counter fails it by exactly one bit (`A5 3C` → `52 9E`). Every earlier
|
||
testbench asserted CS with CLK high, so falls and rises always matched and the
|
||
slip was *structurally invisible* — the same failure of imagination as the
|
||
Mode 2/3 mix-up, which was invisible to a master that parked MOSI early. When
|
||
adding SPI tests, vary the edge the transaction starts on.
|
||
|
||
### EXI on the J4 debug header — REMOVED 2026-10-03
|
||
|
||
`EXI_J4=1` used to merge a second set of EXI pads (the unpopulated J4 debug
|
||
header, FPGA pins 9/10/11/12/6) onto the SP1 pads with an AND, so a bench host
|
||
(a Raspberry Pi) could drive EXI without the edge connector. **The whole path
|
||
is gone** — `exi_bba/exi_j4.py`, the `exi_j4` resource, the AND merge and
|
||
`scripts/pi-exi-wirecheck.py`. Recover it from git history if a bench host is
|
||
ever needed again.
|
||
|
||
It was removed because it was actively costing MISO timing, and the GameCube
|
||
itself is now the test host:
|
||
- the AND put a **LUT in the clock path**, so the EXI clock pad could not use
|
||
its dedicated global-buffer route; and
|
||
- MISO fanned out to both pads, and the J4 pad at die (17,0) was the far end
|
||
of the **critical output path** — 4.79 ns of routing to a header with
|
||
nothing plugged into it.
|
||
|
||
Dropping it cut MISO clock-to-pad from ~10.2–11.5 ns to ~9.3–9.6 ns.
|
||
|
||
**HARDWARE BRING-UP 2026-09-30: the EXI capture path is PROVEN on silicon.**
|
||
A Raspberry Pi 4 bit-banging EXI into J4 (EXI_J4 build, seed 8) enumerated the
|
||
device and exercised the register file end-to-end:
|
||
|
||
| test | result |
|
||
|---|---|
|
||
| device ID @0x0000, 4 B | `04 02 02 00` — **21/21** |
|
||
| NWAYS @0x31 | `0x17` — 21/21 |
|
||
| HIPR @0x3A | `0x01` — 21/21 |
|
||
| PAR0–3 write `DE AD BE EF` + readback | **match**; restore to `00 09 BF 00` verified |
|
||
| IMR @0x08 write `0x86` + readback | `0x86` |
|
||
| NCRA @0x00 write `0x01` (reset) | self-cleared to `0x00` on the first poll |
|
||
|
||
**The full Swiss init sequence runs on hardware (2026-09-30)**, driven from the
|
||
Pi over J4 with the board on 12 V — device ID `04 02 02 00`, NCRA reset
|
||
self-clearing, PAR0–5, MAR0–7, ANALOG, NWAYC, IMR `0x86` readback, NCRA SR,
|
||
and NWAYS `0x17`. See `hardware/re-bba-rb/exi_fullpath_rpi.py`.
|
||
|
||
✅ **FULL ROUND TRIP ON HARDWARE 2026-09-30.** The board sent 5 ARP requests
|
||
over EXI and received all 5 replies back: `op=2`, unicast to our own MAC
|
||
`00:09:bf:00:00:01`, sender 10.0.0.50 / target 10.0.0.99, read back out of the
|
||
SPRAM ring by EXI DMA with RWP advancing exactly one page per exchange. The PC
|
||
could only reply, and only address it to our MAC, if our request reached it —
|
||
so this closes the loop independently of any packet capture.
|
||
|
||
Getting there took four W5100 fixes, none of which simulation could catch
|
||
because the bus model reproduces neither the chip's reset timing nor its
|
||
interrupt latching. Each masked the next:
|
||
1. `tx_bytes` overflow — the drain waited for a length that only arrived at the
|
||
ST trigger, so frames truncated at exactly 16 bytes (see TWD fix below).
|
||
2. Post-reset settle was 1 ms; the W5100S needs 60.3 ms. Marginal, so it
|
||
*sometimes* worked — which is what made everything look flaky.
|
||
3. `w5100_rst_n` was declared and NEVER DRIVEN. The only reset was the MR
|
||
software reset, which a wedged chip cannot answer because it arrives over
|
||
the stuck bus — so a wedge survived NCRA reset, reflashing and even FPGA
|
||
reconfiguration, and needed a 12 V power cycle.
|
||
4. `RX_RSR_CHK` returned to IDLE on RSR==0 without clearing `Sn_IR`. SEND_OK
|
||
(bit 4) fires after every transmit and only RECV (bit 2) was ever cleared,
|
||
so INT_N latched low; since IDLE tests `~w5100_int_n` BEFORE `tx_valid`, the
|
||
FSM span and TX starved. Symptom: first frame transmits, all later ones
|
||
never set TI while RX keeps working.
|
||
|
||
⚠️ Note TI only means `TXFrameDrain` handed the frame to the W5100 — it does
|
||
NOT mean the chip put it on the wire. Do not treat TI as proof of transmission;
|
||
the round trip above is the real test.
|
||
|
||
✅ **BOTH DIRECTIONS PROVEN ON HARDWARE 2026-09-30 — the BBA data path works.**
|
||
|
||
*GC → wire* (after the TWD fix below): a full **60-byte ARP request**, captured
|
||
on the PC byte-for-byte correct — broadcast dst, `00:09:bf:00:00:01` source,
|
||
ethertype 0806, sender 10.0.0.99, target 10.0.0.50, padded to 60. TI sets.
|
||
|
||
*wire → GC*: a real **342-byte DHCP broadcast** from the PC was received by the
|
||
W5100 in MACRAW mode, assembled into the SPRAM ring by `RXFrameAssembler`, and
|
||
DMA-read back over EXI from 0x100. RI set. The ring descriptor read
|
||
`00 00 01 5A` = 346 = 342 + 4, and the frame's own IP total-length field (328)
|
||
+ 14 bytes of ethernet header = 342 — so the assembler's length is
|
||
independently confirmed correct, not just plausible.
|
||
|
||
Together: EXI ↔ BBARegisterFile ↔ {tx_bytes→TXFrameDrain, SPRAM ring} ↔
|
||
W5100ParallelMaster ↔ PHY, in both directions, on silicon.
|
||
|
||
✅ **GC→wire TX REACHES THE WIRE — the full path is proven.** A frame written
|
||
over EXI to TXDATA and triggered via the NCRA ST bits was captured on a PC:
|
||
`ff ff ff ff ff ff | 00 09 bf 00 00 01 | 08 06 | 00 01` — broadcast dst, our
|
||
PAR0–5 source MAC and the ARP ethertype, all byte-perfect. So
|
||
EXI → BBARegisterFile → tx_bytes → TXFrameDrain → W5100 → PHY works.
|
||
|
||
✅ **FIXED 2026-09-30: TX used to truncate at exactly 16 bytes** = the
|
||
`tx_bytes` FIFO depth. Past byte 16 the captured frame restarted from its own
|
||
beginning and mixed in stale bytes from earlier attempts. Cause:
|
||
`TXFrameDrain` sat in IDLE until it had a LENGTH, and the length only reached
|
||
`tx_ctrl` when the NCRA ST bits were written — i.e. *after* the GC had streamed
|
||
the whole frame. So the entire frame had to fit a 16-deep FIFO, and
|
||
`BBARegisterFile` silently drops bytes when `tx_bytes.w_rdy` is low (SPI cannot
|
||
back-pressure).
|
||
|
||
Fix: the length is pushed into `tx_ctrl` when **TWD_HI (0x35)** is written,
|
||
which the GC sets *before* streaming, so the drain consumes concurrently with
|
||
the EXI writes and the 16-deep FIFO becomes a rate-matching buffer instead of
|
||
whole-frame storage (`twd_armed` / `tx_queued` in `bba_register_file.py`; the
|
||
NCRA ST path is now only a fallback for when TWD_HI did not queue). Verified on
|
||
hardware — a full 60-byte ARP request captured byte-for-byte correct.
|
||
|
||
Simulation never caught the original bug: `TXFrameDrain`'s own tests drive the
|
||
FIFO signals directly, and the `BBATop` test only exercises the
|
||
W5100→SPRAM→GC **RX** loop. That gap is still open — there is no end-to-end
|
||
GC→wire TX test in simulation.
|
||
|
||
When probing TX, **NCRA-reset between every attempt**: a failed attempt leaves
|
||
the drain waiting on a stale length, and later writes feed that pending
|
||
request, which makes an unrelated frame length appear to succeed.
|
||
|
||
⚠️ **The W5100 can wedge and does NOT recover on FPGA reconfigure.** Symptom:
|
||
the UART shell's own socket-3 `udp unicast` echoes the command and never prints
|
||
`sent`, and any EXI TX then hangs waiting on the bus master. `iceprog -t` does
|
||
not clear it (RST_N floats to its pull-up while the FPGA is unconfigured). A
|
||
12 V power cycle is required. Check ETH_3V3 and GC_ON (U12 gates the W5100 rail
|
||
off GC_ON) before suspecting gateware.
|
||
|
||
### ✅ The device ID is answerable AT ANY TIME (one-shot removed 2026-10-10)
|
||
|
||
`id_phase` **used to be a one-shot**: set at reset and cleared permanently by
|
||
the GC's first NCRA write with the reset bit, after which a command-space read
|
||
of register 0 returned `0x00000000`. That is now gone — `id_arm` is tied high
|
||
and the register file has no `id_phase` output.
|
||
|
||
⚠️ **The one-shot broke Swiss outright, which is how it was caught.** Swiss
|
||
calls `EXI_GetID` from `bba_exists()` *and* again from `bba_requires_init()`,
|
||
long after init has run:
|
||
|
||
```c
|
||
if ((location & LOC_SERIAL_PORT_1) && EXI_GetID(EXI_CHANNEL_0, EXI_DEVICE_2, &id)) {
|
||
switch (id) { case EXI_MX98730EC_ID: ...
|
||
```
|
||
|
||
With the one-shot spent, that read returns `0x00000000`, no case matches, and
|
||
Swiss reports **no adapter at all** — not a broken one, absent. Running
|
||
`gc_bba_test` first was enough to spend it, so the adapter would "disappear"
|
||
after any test session.
|
||
|
||
**The old justification was wrong.** The comment claimed the fast path would
|
||
otherwise "hijack address 0 forever so the GC could never read NCRA back".
|
||
NCRA is **register space** (`0x80`/`0xC0` header); the fast path requires
|
||
byte 0 == `0x00`, i.e. **command space**. They were never ambiguous, and no
|
||
driver reads command register 0 for anything else (libogc2 reads
|
||
`0x01`/`0x03`/`0x0B`/`0x0F`, Swiss reads `0x03`). Real hardware answers
|
||
`EXI_GetID` whenever asked — that is how EXI enumerates devices.
|
||
|
||
Test: `bba_top` **T1b** reads the device ID *after* an NCRA reset, which T1
|
||
(before any reset) structurally cannot catch. Mutation-tested: disarming the
|
||
fast path returns `00 00 00 00` for both, exactly what Swiss saw.
|
||
|
||
Two bring-up gotchas, both cost real debugging time:
|
||
- **The master must sample MISO during the CLK-LOW phase.** The FPGA drives MISO
|
||
on the RISING edge (Mode 3), so sampling just after that edge races the
|
||
slave's own update and lands one bit position early — `04 02 02 00` comes back
|
||
as `08 ...`. Sample after taking CLK low, before raising it (this is exactly
|
||
what the simulation's `spi_byte` does).
|
||
- **Clock EXACTLY the requested byte count on a register read.** Over-clocking
|
||
(asking for 4 bytes then clocking 8–12) leaves stale prefetched bytes queued
|
||
and desynchronises following transactions — the symptom is a *deterministic*
|
||
every-Nth-read correctness cycle that looks like a signal-integrity fault but
|
||
is speed-independent. The GC driver never over-clocks a register read, so this
|
||
is a probe artefact, NOT a gateware bug; reads ≥ 0x100 deliberately stream
|
||
until CS (the DMA path). `hardware/re-bba-rb/exi_regcheck_rpi.py` clocks
|
||
correctly and is the reference host-side probe.
|
||
|
||
⚠️ **The two measurements below are HISTORICAL** — they predate the
|
||
2026-10-03 pin-clocked rewrite and refer to the `capture_clk` domain, which no
|
||
longer exists. Current numbers are in "Current Implementation State" above.
|
||
They are kept only because they document how added `sync`-domain cells moved
|
||
the pass rate, which still applies.
|
||
|
||
**Measured 2026-09-30 AFTER the TWD TX fix**, `EXI_J4=1` + socket-3 UDP,
|
||
`--seeds 8`: `clk` 28.1–30.2 MHz (PASS on all 8); `capture_clk` **2 of 8 PASS**
|
||
(seeds 4, 7), best **seed 7 = 60.03 MHz** (+6.0 MHz / 11% margin). The fix's
|
||
handful of extra exi-domain cells (`twd_armed`/`tx_queued`) dropped the pass
|
||
rate from 4/8 to 2/8 even though none of that logic is in the capture domain —
|
||
added cells raise congestion and capture is routing-bound. Best seed is
|
||
comfortably usable; just sweep and flash the winner. Earlier figures, before
|
||
the fix:
|
||
|
||
**Measured 2026-09-30 (pre-fix)**, `EXI_J4=1` + socket-3 UDP, `--seeds 8`:
|
||
`clk` 29.2–31.4 MHz (PASS on all 8); `capture_clk` **4 of 8 PASS** (seeds 2, 5,
|
||
6, 8), best **seed 8 = 63.09 MHz** (+9.1 MHz / 17% margin). Same pass rate as
|
||
the default build and a better best seed, so the merge costs no measurable
|
||
capture timing — but it is still seed-dependent, so **always sweep**.
|
||
|
||
### Pending work
|
||
- **Synthesis/timing**: ✅ — see the seed table above. Removing the PLL-fed
|
||
capture domain eliminated the constraint that had limited the design to a
|
||
minority of passing seeds.
|
||
- **W5500 init/TX/RX**: ✅ done — `W5500SPIMaster` has a real Mode-0 byte engine,
|
||
a generic register-transaction engine (header + wbuf/stream payload), the full
|
||
init sequence (MR reset, SHAR, S0_MR MACRAW, S0_CR OPEN, S0_IMR), MACRAW TX
|
||
(read TX_WR → stream frame to TX buffer → advance TX_WR → SEND) and MACRAW RX
|
||
(RSR → RD → 2-byte length → stream frame out → advance RD → RECV). All verified
|
||
on the wire by a responding W5500 SPI-slave model in the testbench.
|
||
- **PAR0–5 → W5500 SHAR**: ✅ done — `reg.par` wired to `w5500.par` in `BBATop`
|
||
(PAR0 packed in the low byte so it is the first SHAR octet).
|
||
- **NCRA SR bit**: ✅ done — `BBARegisterFile.ncra_sr` (= NCRA[3]) gates
|
||
`asm.rx_enabled` in `BBATop` (was hard-wired to 1).
|
||
- **W5500 SPI throughput**: SCK = sync÷2 = 12 MHz (~12 Mbit/s) — exceeds
|
||
real-world GC BBA TCP throughput (~6–10 Mbit/s) but is below the 27 Mbit/s raw
|
||
EXI ceiling. Pushing past 12 Mbit/s was investigated and found NOT achievable
|
||
on this UP5K (the W5500-operating logic is distributed ~40 MHz, not just the
|
||
bit-bang) — see the "Full-rate W5500 SPI" item below.
|
||
`W5500SPIMaster(clk_div=N)` divides SCK further if signal integrity needs it.
|
||
- **EXI DMA bulk reads**: ✅ done — SPRAM-region reads (addr ≥ 0x100) now STREAM
|
||
until CS deasserts instead of stopping at the header's 2-bit length, so they
|
||
serve both ≤4-byte immediate reads (Swiss) AND arbitrary-length DMA reads
|
||
(other GC software, and a future Swiss path for loading ROMs from a network
|
||
file store). Implementation:
|
||
- `SPIMode3Slave.cs_active` (synchronised CS level) → `ExiCapture` crosses it
|
||
to the exi domain (FFSynchronizer) → `BBARegisterFile.cs_active`.
|
||
- `BBARegisterFile` SPRAM_STREAM state: auto-increments the SPRAM address,
|
||
prefetches up to SP_LIMIT=4 reads in flight, pushes responses to tx_fifo;
|
||
SPRAM_END drains the in-flight pipeline + rx dummies on CS-rise.
|
||
- `ExiCapture` flushes tx_fifo on CS-fall to clear prefetch over-push so a
|
||
truncated DMA read can't leak stale bytes into the next transaction.
|
||
Tested: register-file streaming read (SPRAM model, 12 bytes), ExiCapture
|
||
over-push/flush, AND the full BBATop loop — a W5500 model delivers a frame →
|
||
W5500 master RX → RXFrameAssembler writes the SPRAM ring → GC reads RWP then
|
||
DMA-reads the descriptor+frame back (verified byte-for-byte).
|
||
Note: a DMA read header must keep length-1 within the 2-bit field; the GC
|
||
driver sets it ≤3 and clocks the real length via CS (the design streams
|
||
until CS regardless). (EXI DMA *writes* are not implemented; the GC's
|
||
DMA-write engine has a 1-bit-shift bug and Swiss avoids them — see
|
||
design-doc §"EXI DMA bug".)
|
||
- **S0_IR interrupt clear after RX**: ✅ done — `W5500SPIMaster` RX_CLR_IR state
|
||
writes Sn_IR[2]=1 after RECV so `INT_N` deasserts (else the FSM would re-enter
|
||
RX_CHECK forever on real hardware).
|
||
- **Full-rate W5500 SPI (27 Mbit/s) — INVESTIGATED, NOT achievable on UP5K**:
|
||
the W5500 SCK is sync÷2 = 12 MHz. Raising it needs the SPI engine on a ≥54 MHz
|
||
clock, but a standalone synth of `W5500SPIMaster` in the capture domain closes
|
||
only **40 MHz** — and the slack histogram shows the failure is *distributed*
|
||
(~140 endpoints fail 54, incl. the `wbuf`/header mux feeding the shift
|
||
register), NOT a single cuttable path. So the bottleneck is the **logic that
|
||
operates the SPI device** (transaction FSM, byte sourcing), not the bit-bang.
|
||
Consequences:
|
||
- The "split the bit engine to capture + per-byte CDC handshake" idea nets
|
||
only ~14 Mbit/s — the CDC round-trip ≈ the SPI byte time — not worth it.
|
||
- A capture-domain "streaming executor" would still contain that distributed
|
||
~40 MHz logic, so it wouldn't close 54 either.
|
||
- Hardware `SB_SPI` wouldn't help (it only offloads the bit-bang, which was
|
||
never the bottleneck) and is unsimulatable.
|
||
- There is no usable clock between 24 (HFOSC) and 54 (the one PLL, needed at
|
||
54 for the EXI front-end); PLL÷2 = 27 → SCK 13.5 MHz, a ~12% gain, not
|
||
worth the fabric divider.
|
||
Net: 12 Mbit/s is the practical W5500 ceiling on this part. It exceeds
|
||
real-world GC BBA TCP throughput and is fine for chunked ROM streaming.
|
||
Reaching 27 Mbit/s would need a faster FPGA or a much shallower W5500-operating
|
||
redesign (uncertain) — **OR a parallel-bus ethernet chip (see W5100 below)**,
|
||
which is the implemented solution for the ROM-streaming throughput target.
|
||
|
||
## W5100 vs W5500 ethernet back-end
|
||
|
||
The throughput insight: SPI serialises 8 bits/byte, so the W5500 byte rate is
|
||
(operating-logic clock)/16 — and that logic caps ~40 MHz on this UP5K → ~12
|
||
Mbit/s. A **parallel** bus moves a whole byte per access, so the *same* ~24 MHz
|
||
`sync` logic clears the 27 Mbit/s EXI ceiling (the real hard limit — the GC EXI
|
||
bus tops out there). So `W5100ParallelMaster` is the throughput path and is now
|
||
the `BBATop` default.
|
||
|
||
- **Interface:** W5100 **indirect parallel bus** (IDM). Only A[1:0] are wired
|
||
(board ties A[14:2]=0 so a power-up direct access at A=00 still hits MR):
|
||
`00`=MR, `01`=IDM_AR0(hi), `10`=IDM_AR1(lo), `11`=IDM_DR. A register/buffer
|
||
access = write IDM_AR (the 16-bit address) then read/write IDM_DR. With MR.AI
|
||
set, IDM_DR auto-increments → a multi-byte block is one address-set + a burst.
|
||
- **Bus engine:** drives A + D with `/CS` and `/RD`|`/WR` asserted for
|
||
`strobe_cycles` (default 3 ≈ 125 ns at 24 MHz, ≥ the W5100's ~80 ns access).
|
||
DATA[7:0] is bidirectional → an SB_IO tristate (`bus_data_o`/`oe`/`i`).
|
||
- **Pins (15):** A[1:0]=2, D[7:0]=8, /CS,/RD,/WR=3, /INT=1, /RST=1. With EXI (5)
|
||
+ clk (1) = **21 of ~34 usable SG48 I/O** — comfortable. See `synth.py`.
|
||
- **MR.AI requires init first:** unlike the W5500 (each SPI transaction is
|
||
self-framed), the W5100's multi-byte accesses depend on MR.AI, so the init
|
||
sequence (triggered by the GC's NCRA reset) MUST run before any TX/RX. The
|
||
BBATop test issues NCRA-reset before its RX loop for this reason; on hardware
|
||
the GC driver already does. (`BBATop(reset_cycles=N)` shrinks the MR settle
|
||
wait for sim.)
|
||
- **Ring wraparound is in fabric:** the W5100 does NOT auto-wrap the IDM address
|
||
at the socket-buffer boundary (the W5500 did), so the streamer re-sets IDM_AR
|
||
to the buffer base when the running address reaches the 2 KB boundary. Handled
|
||
in the SW/SR/RB paths (`xfer_wrap`/`xfer_wbase`/`xfer_wend`/`cur_addr`); both
|
||
TX and RX wrap cases are tested.
|
||
- **Register map differs from the W5500:** common regs at 0x0000 (MR, SHAR 0x09,
|
||
IMR 0x16, RMSR/TMSR 0x1A/0x1B), socket 0 at 0x0400 (S0_MR/CR/IR, TX_WR 0x424,
|
||
RX_RSR 0x426, RX_RD 0x428), TX buffer 0x4000, RX buffer 0x6000. MACRAW mode.
|
||
- **Status:** init/TX/RX (with wrap) verified vs a bus model; BBATop full
|
||
W5100→SPRAM→GC RX loop passes byte-for-byte; synth PASS. **Hardware bring-up
|
||
2026-09-20: a real UDP datagram was transmitted and captured on a PC** — the
|
||
ethernet back-end is proven end-to-end. Register addresses confirmed on silicon,
|
||
with three W5100S-specific gotchas the from-memory map got wrong:
|
||
- **NETLCKR (0x0071) must be unlocked with 0x3A** before GWR/SUBR/SHAR/SIPR are
|
||
writable (default 0x00 = locked). The original W5100 has no such register, so
|
||
without this every network-config write is silently dropped and the chip
|
||
transmits with a zero source MAC/IP.
|
||
- **The parallel/indirect (IDM) bus decodes only a 15-bit offset**, so the TX/RX
|
||
buffers are at the original **0x4000 / 0x6000** — NOT the 0x8000/0xC000 the
|
||
W5100S datasheet pseudo-code quotes. Using 0x8000 makes payload writes alias
|
||
onto 0x0000+ and overwrite SHAR/SIPR. Proven with an address-aliasing probe
|
||
(0xC000 and 0x4000 read back the same cell). Do not "fix" this again.
|
||
- **The W5100S needs 60.3 ms after RSTn** before it answers; polling sooner reads
|
||
all-zeros and looks like a dead chip.
|
||
- UDP **unicast** works (the chip ARPs the peer first, per Sn_MR FARP). A send to
|
||
255.255.255.255 sets SEND_OK but was not observed on the wire — prefer unicast
|
||
or a subnet-directed broadcast for bring-up tests.
|
||
|
||
### `rebbarb/` — LED blink demo (unchanged)
|
||
- `rebbarb.py` — blinks LEDs via a PLL (36 MHz), demonstrates `IceBreakerPlatform`
|
||
- `debouncer.py` — `Debouncer(cycles)` — synchronous debounce, configurable hold
|
||
- `toggle_button.py` — `ToggleButton` — edge-to-toggle state machine (wraps Debouncer)
|
||
- `pulse_button.py` — `PulseButton` — single-cycle pulse on rising edge (wraps Debouncer)
|
||
|
||
These components are reusable building blocks. The `Debouncer` and button wrappers
|
||
will be needed for any physical input in `exi_bba/`.
|
||
|
||
**Import note:** `rebbarb/` files use bare imports (`from debouncer import Debouncer`).
|
||
Run them as `python rebbarb/<file>.py` from the workspace root so Python adds
|
||
`rebbarb/` to `sys.path` automatically.
|
||
|
||
**Simulation at module level:** `toggle_button.py` and `pulse_button.py` run
|
||
their simulations unconditionally (no `__main__` guard) — importing either file
|
||
triggers a VCD write. New modules should guard simulation code with
|
||
`if __name__ == "__main__":`.
|
||
|
||
`examples/amaranth_cdc.py` contains handwritten `SyncFF` and `TogglePulseSync`
|
||
reference implementations — use `amaranth.lib.cdc` primitives (`FFSynchronizer`,
|
||
`PulseSynchronizer`) in production code instead.
|
||
|
||
`hardware/sp1_test_plug/` — KiCad project for a physical SP1 edge-connector test
|
||
plug (schematic, PCB, custom GameCube symbol library). Used to verify pad geometry
|
||
before ordering the interposer PCB; not part of the FPGA build.
|
||
|
||
`hardware/re-bba-rb/` — the actual adapter PCB (FT2232H + iCE40UP5K + W5100S,
|
||
powered from SP1 12 V via buck). Three companion documents, all mandatory
|
||
reading before touching this board:
|
||
- **`hardware/re-bba-rb/TODO.md`** — open action items (what still needs doing:
|
||
PCB placement of the 2026-07-18 fix parts, board PCF, bench checks, JLC
|
||
ordering options).
|
||
- **`hardware/re-bba-rb/REVIEW.md`** — the review **coverage record**: every
|
||
circuit that has been verified against its datasheet/reference design, with
|
||
verdicts — including things that LOOK wrong but are settled-correct (FT2232H
|
||
VPHY/VPLL on 3.3 V; D4/D5 floating VBUS + split-net flow-through; TPS562201
|
||
EN at 12 V; RSET 12.4 k; no TPS22810 CT cap; SP1 DO = GC→adapter = MOSI).
|
||
**Do not re-review or "fix" anything listed there without new evidence, and
|
||
keep REVIEW.md updated in the same commit whenever a reviewed circuit
|
||
changes or a new review is performed.**
|
||
- **`hardware/re-bba-rb/CPL_ROTATIONS.md`** — per-part assembly rotation
|
||
corrections applied on top of `kicad-cli`'s raw CPL export (KiCad's raw
|
||
rotation frequently doesn't match what the assembler expects, and a
|
||
generic community correction table was tried and found wrong for this
|
||
board — do not reapply it). **Regenerating `re-bba-rb-cpl.csv` "the normal
|
||
way" silently wipes these corrections** — always reapply the corrections
|
||
in this file after any CPL regen, and read its confidence notes before
|
||
trusting any specific part's rotation.
|
||
The authoritative board pin map (EXI, ETH bus, UART crossover, LED/DBG pins)
|
||
is in TODO.md and REVIEW.md; use it when writing the board PCF.
|
||
|
||
---
|
||
|
||
## Amaranth Simulator API
|
||
|
||
Two API generations are present in this repo:
|
||
|
||
| API | Where used | Status |
|
||
|---|---|---|
|
||
| `sim.add_testbench(async_fn)` + `await ctx.tick()` + `Period(MHz=n)` | `rebbarb/*.py` | **Use this for new code** |
|
||
| `sim.add_sync_process(gen_fn)` + `sim.run_until(t)` | `examples/` | Old — reference only |
|
||
|
||
New modules should use the testbench API (`add_testbench`, `sim.write_vcd(ctx)`
|
||
context manager). The old process API still works but is not idiomatic in current
|
||
Amaranth.
|
||
|
||
**Critical testbench timing rule:** `ctx.get(signal)` reads signal values AFTER
|
||
the clock edge (post-update registered values). Combinatorial signals that depend
|
||
on registered signals that were updated by the SAME tick will already reflect the
|
||
new registered values. For example: if `tx_sof = tx_bytes_r_rdy & is_first` and
|
||
`is_first` is cleared synchronously on the first byte, then reading `tx_sof` after
|
||
the first byte's tick always returns 0 — read BEFORE the tick instead.
|
||
|
||
**`ctx.set()` takes effect immediately** (combinatorial, not registered). Use it
|
||
AFTER `await ctx.tick()` to prepare inputs for the NEXT tick.
|
||
|
||
The full design specification lives in `docs/gc_bba_fpga_design.md`.
|
||
|
||
---
|
||
|
||
## Key Architecture Decisions
|
||
|
||
- **No network stack in the FPGA.** The GC CPU runs TCP/IP. The FPGA is a dumb
|
||
MAC bridge.
|
||
- **The EXI bit engine is clocked BY the GC's EXI clock pin** (2026-10-03
|
||
rewrite). There is no PLL and no oversampling.
|
||
- `exiclk` / `exiclk_n` — the EXI clock pin itself (~27 MHz), posedge and
|
||
negedge. Created inside `ExiCapture`; async-reset by CS. Hosts ONLY the
|
||
SPI bit engine. MOSI is sampled on the posedge, MISO driven on the negedge.
|
||
- `exififo_w` / `exififo_r` — the same pin, posedge, reset-less. The byte
|
||
FIFO sides. (Reset-less because resetting one side of an `AsyncFIFO`
|
||
desynchronises its gray pointers; and posedge because `AsyncFIFO`
|
||
instantiates a posedge-only reset CDC internally, so a negedge domain
|
||
cannot host a FIFO side.)
|
||
- `sync` — 24 MHz, internal `SB_HFOSC` ÷2. **Everything else**: register
|
||
file, SPRAM arbiter, RX/TX engines, ethernet master. The former separate
|
||
`exi` domain was merged into `sync` (they always shared the HFOSC net).
|
||
- **Why:** oversampling put BIT-level timing across a CDC boundary. The
|
||
engine's view of the clock lagged the pin by 2 `FFSynchronizer` stages plus
|
||
a register — ~55 ns against a 37 ns bit time — so MISO came out ~2 bits
|
||
late and the GC read `0x17` as `05 C0`. No amount of oversampling fixes it:
|
||
54 MHz gives only 2 ticks per bit and the part cannot close the 81 MHz that
|
||
3× would need. Clocking from the pin makes MISO clock-to-Q and leaves the
|
||
byte-level FIFOs as the only crossing, with a whole byte-time (~296 ns) of
|
||
slack instead of a fraction of a bit.
|
||
- **Timing consequence:** `capture_clk` is gone as a constraint. It was the
|
||
binding path for the whole design and the reason builds closed on only a
|
||
minority of seeds.
|
||
- **EXI clock reality:** the GC EXI clock tops out at ~27 MHz. libogc's
|
||
`EXI_SPEED32MHZ` is a nominal name — the real rate is 27 MHz. The old
|
||
"96 MHz = 3× 32 MHz EXI" target was doubly wrong and unreachable on UP5K
|
||
(which caps ~44 MHz for non-trivial logic).
|
||
- **TX/MISO across the split:** the register file PROACTIVELY pushes read
|
||
responses into the tx byte FIFO during the EXI clock-idle gap (the GC pauses
|
||
the clock between an EXI_Imm header-write and the data-read). The bit engine
|
||
drives MISO live from the FIFO head; see `ExiCapture` / `SPIMode3Slave`.
|
||
- **All CDC via `amaranth.lib.cdc`.** Never pass raw multi-bit signals across
|
||
domains. Use `FFSynchronizer` for slow single bits, `PulseSynchronizer` for
|
||
events, `AsyncFIFO` for data streams, `ResetSynchronizer` for resets.
|
||
- **Register file lives entirely in `exi` domain.** The `sync` domain only
|
||
communicates through AsyncFIFOs and PulseSynchronizers — never direct register
|
||
reads/writes.
|
||
|
||
---
|
||
|
||
## Critical Protocol Notes
|
||
|
||
### EXI / SPI Mode 3
|
||
- CLK idles **HIGH** (CPOL=1, CPHA=1).
|
||
- The master **changes MOSI on the falling edge**; both ends **sample on the
|
||
rising edge**. The device therefore **samples MOSI on the RISING edge and
|
||
drives MISO on the FALLING edge** (MISO must settle before the master
|
||
samples it on the rising edge).
|
||
- ⚠️ This file previously stated the opposite ("MOSI sampled on falling, MISO
|
||
driven on rising"), which is Mode **2**, and the implementation followed the
|
||
doc. Driving MISO on the rising edge is one bit late by construction — that
|
||
was the `0x17` → `05 C0` hardware bug. `spi_mode3_slave.py`'s testbench now
|
||
mutation-tests both edges so neither can silently regress.
|
||
- CS is active **low**, delineates each transaction, and async-resets the
|
||
bit engine.
|
||
|
||
### EXI Transaction Header
|
||
|
||
**Implemented 2026-10-03** (`bba_register_file.py`), from the two drivers that
|
||
actually run — libogc (`libdb/uIP/bba.c`) and Swiss (`cube/patches/bba/bba.c`).
|
||
There are TWO address spaces, selected by bit 7 of the first byte, and **no
|
||
length field**:
|
||
|
||
```
|
||
COMMAND space (6-bit reg) 2-byte header, then a SEPARATE data phase
|
||
write: [0x40|reg, 0x00] then N bytes written (libogc writes 1)
|
||
read : [reg, 0x00] then N bytes read (device ID reads 4)
|
||
REGISTER space (16-bit reg) 4-byte header, then a SEPARATE data phase
|
||
read : [0x80, reg>>8, reg&0xFF, 0x00] then N bytes read (Swiss uses DMA)
|
||
write: [0xC0, reg>>8, reg&0xFF, 0x00] then N bytes written
|
||
```
|
||
|
||
⚠️ **COMMAND SPACE HAS TWO FORMS AND BOTH ARE REAL — settled 2026-10-10 by
|
||
reading all three drivers.** Register space is the same everywhere; command
|
||
space is not:
|
||
|
||
| contract | Swiss patch | libogc (stock) | libogc2 |
|
||
|---|---|---|---|
|
||
| RX descriptor | packed LE bitfield | packed LE bitfield | packed LE bitfield |
|
||
| register space | 4-byte header + data phase | same | same |
|
||
| **command space** | **ONE 4-byte transfer**, value/response in **byte 3** | 2-byte header + data phase, **byte 2** | 2-byte header + data phase, **byte 2** |
|
||
| TWD on TX | **never written** | never written | never written |
|
||
| TX trigger | NCRA **ST1** | NCRA ST1 (`0x04`) | NCRA ST1 |
|
||
|
||
Sources, so this can be re-checked rather than re-guessed:
|
||
- Swiss — `cube/patches/bba/bba.c`, `bba_cmd_out8`:
|
||
`exi_imm_read_write(0x40<<24 | (reg&0x3F)<<24 | val, 4)`, and `bba_cmd_in8`
|
||
returns the **low byte** of that 32-bit result.
|
||
- libogc2 — `lwip/arch/gc/netif/gcif.c`, `bba_cmd_outsnosel`:
|
||
`req = (reg<<8)|0x4000; EXI_ImmEx(&req, 2, WRITE); EXI_ImmEx(val, len, WRITE)`.
|
||
- stock libogc — the same, read out of `libbba.a` (`gcif.o`, `__bba_init`):
|
||
`EXI_Imm(len=2)` of `42 00`, then `EXI_ImmEx(len=1)` of the data.
|
||
|
||
**They are NOT two protocols — only two amounts of padding.** This is worth
|
||
stating plainly, because "the menu and a game speak different protocols to the
|
||
same chip" is rightly suspicious and is not what happens. Both send the
|
||
*identical* 2-byte header `[0x40|reg, 0x00]`. They differ only in how many
|
||
bytes follow, and the device is length-agnostic — the data phase runs until CS,
|
||
exactly as in register space. Swiss's 4-byte version is an **EXI efficiency
|
||
choice**, not a different protocol: one `exi_imm_read_write(…, 4)` is a single
|
||
EXI immediate operation (two register pokes), where libogc2 spends two separate
|
||
`EXI_ImmEx` calls. For code injected into a running game, halving the EXI
|
||
operations per register access is worth a padding byte. The real BBA tolerates
|
||
both because it never cared about the length in the first place.
|
||
|
||
Menu vs game is therefore not a protocol split, it is just which library is
|
||
linked: `cube/swiss/source/bba.c` contains **no BBA register access at all**
|
||
(its header exposes only network config, detection and display, over
|
||
`<network.h>`), so the menu is 100% libogc2; the patch driver is Swiss's own
|
||
code and spends fewer EXI operations.
|
||
|
||
**How one decoder serves both:** they share bytes 0 and 1, and differ only in
|
||
how many bytes follow. So the device takes the **LAST data byte before CS
|
||
rise** as the write value (Swiss sends `[00, val]`, libogc sends `[val]` —
|
||
last is `val` either way), and **pushes a read response TWICE** so byte 2 and
|
||
byte 3 both carry it. Swiss clocks 4 bytes and pops both copies; libogc clocks
|
||
3 and leaves one, which the next transaction's header pops drain — a 2-byte
|
||
header gives exactly one drain slot, so the leftover is bounded at one and
|
||
cannot accumulate.
|
||
|
||
⚠️ **There must be NO auto-increment of `cmd_reg` across the data phase.** It
|
||
would apply Swiss's padding `0x00` to the register and `val` to `reg+1`. This
|
||
is how the libogc-only version broke Swiss.
|
||
|
||
History worth keeping, because the failure mode repeated: this file first had
|
||
Swiss's 4-byte form only, so libogc's writes were dropped (the decoder waited
|
||
for a fourth byte and ate the data byte as header byte 2). It was then
|
||
"fixed" to libogc's form only, which would have dropped Swiss's writes. Only
|
||
reading all three drivers showed that neither is "the" driver. The device-ID
|
||
read masked the original bug throughout, because `SPIMode3Slave` answers it on
|
||
a fast path independent of this FSM.
|
||
|
||
Tests: `bba_register_file` **T10** (libogc form) and **T12** (Swiss form,
|
||
asserting bytes 2 AND 3), both mutation-tested — re-adding the auto-increment
|
||
fails T12, and pushing the response once fails T12's byte 3.
|
||
|
||
**This is ONE decoder, not two.** There is no driver detection, no mode flag
|
||
and no conditional on which driver is talking — grep the RTL with comments
|
||
stripped and the words "Swiss" and "libogc" do not appear. The unification is
|
||
that **Swiss's "4-byte transfer" is not a second protocol**: from the device's
|
||
side it is the same 2-byte header followed by a data phase that happens to be
|
||
2 bytes (`00, val`) rather than 1 (`val`). Command space was simply the last
|
||
place that still assumed a fixed transfer length — the same mistake register
|
||
space already avoids ("no length field, the data phase runs until CS"). Both
|
||
rules are therefore length-agnostic: last data byte wins, fill the first two
|
||
data slots.
|
||
|
||
⚠️ **The residual ambiguity, which is real but currently inert.** A 2-byte
|
||
data phase has two meanings that cannot be told apart on the wire:
|
||
|
||
```
|
||
[0x42, 0x00, 0x00, val] Swiss -> cmd reg 2 = val (pad + value)
|
||
[0x42, 0x00, b0, b1 ] libogc2 -> cmd reg 2 = b0, 3 = b1 (two registers)
|
||
```
|
||
|
||
libogc2 really does multi-byte command writes — `bba_cmd_outs(0x04,
|
||
&priv->devid, 2)` runs unconditionally in its init. "Last byte wins" is wrong
|
||
for those, and auto-increment is wrong for Swiss, so **no single rule satisfies
|
||
both** and the real device's behaviour here cannot be deduced from the drivers.
|
||
|
||
⚠️ **It is also unobservable in them**, which is why it has never mattered:
|
||
`priv->devid` is the hardcoded constant `0xD107`, is never read back from the
|
||
device, and `bba_cmd_out8(0x05, priv->acstart)` on the very next line writes
|
||
`0x05` separately anyway — so under auto-increment half of that write is
|
||
immediately overwritten. Neither driver ever reads `0x04`/`0x05`. Do not try to
|
||
settle this from source; it would need a real BBA on a logic analyser.
|
||
|
||
It does not matter for us because:
|
||
- the only command registers implemented are **0x02** (IMR) and **0x03** (IR),
|
||
and BOTH drivers access those a single byte at a time; and
|
||
- libogc2's multi-byte command accesses target `0x04`/`0x05` and
|
||
`0x08`/`0x09`, none of which are decoded — they fall through the `Switch`
|
||
and are ignored.
|
||
|
||
So the decoder is correct for every register it implements. **If `0x04`/`0x05`
|
||
are ever implemented, the fix is per-register width knowledge (single-byte
|
||
registers take the last byte; multi-byte registers auto-increment), NOT a
|
||
driver mode.** Do not "resolve" the ambiguity by sniffing the transfer length.
|
||
|
||
⚠️ **Do NOT implement the challenge/response handshake.** libogc2 reads a
|
||
4-byte challenge from command reg `0x08`, computes `bba_calc_response()` and
|
||
writes it to `0x09` — but only when the DEVICE raises status bit `0x10` in
|
||
command reg `0x03`. It is device-initiated, we never set that bit, so the path
|
||
never runs. Likewise `bba_cmd_out8(0x05, priv->acstart)` and the `0x0B` status
|
||
read only appear inside those handlers. Emulating any of it would be work for
|
||
a path that cannot be entered.
|
||
NCRA, IR, NWAYS, RWP, RRP, PAR… all live in the REGISTER space
|
||
(`bba_in8`/`bba_out8`/`bba_ins`/`bba_outs`). Only the device-ID query uses a
|
||
short header — the single case where the two formats coincide.
|
||
|
||
**What the old (guessed) format cost.** The previous decoder used
|
||
`[7]=write, [6:0]=addr[12:6]` / `[7:2]=addr[5:0], [1:0]=len-1`. Under it a
|
||
`NWAYS` read (`80 00 31 00`) decoded as *write 0x31 to NCRA* — and bit 0 of
|
||
0x31 is RESET, so every register read Swiss performed would have reset the
|
||
device. Enumeration still appeared to work because `00 00` decodes to address
|
||
0 either way, which is exactly why the error survived so long. Do not
|
||
reintroduce a length field.
|
||
|
||
**The trailing `0x00` byte is the device's prefetch window.** The address is
|
||
complete one full byte before the data phase, giving ~8 bit-times (≈296 ns at
|
||
27 MHz) to fetch the response. The engine pops the TX FIFO once per byte
|
||
boundary, so the response the register file pushes during this turnaround byte
|
||
lands in data slot 0 — see the TX comment in `spi_mode3_slave.py`.
|
||
|
||
⚠️ **Command space has only ONE drain slot.** Pops during the header drain
|
||
whatever the previous transaction left in the FIFO, but a 2-byte header gives
|
||
just one such slot, so a command-space response must push **exactly** its
|
||
bytes and stop. `DEVID` streaming zeros until CS left two stale bytes behind,
|
||
one too many to drain, and the *next* enumeration came back a slot late
|
||
(`04 02 02 00` → `00 04 02 02`). Register-space reads may over-push safely:
|
||
their 4-byte header gives three drain slots.
|
||
|
||
**Data-phase length is whatever the GC clocks** (it streams until CS), so the
|
||
register path streams like the SPRAM path. `bba_ins` reads with **EXI DMA**;
|
||
DMA *reads* are fine (the GC's DMA-*write* engine is the buggy one).
|
||
|
||
### Device ID Query
|
||
On power-on the GC writes `0x0000` (2 bytes) then reads 4 bytes.
|
||
Must return: `0x04 0x02 0x02 0x00`. **Hardware-confirmed 2026-10-03** on a
|
||
real console.
|
||
|
||
---
|
||
|
||
## Memory Map (abridged)
|
||
|
||
| Range | Region |
|
||
|---|---|
|
||
| 0x0000–0x0033 | MAC control registers (register file, exi domain) |
|
||
| 0x0048 | TXDATA — bulk TX data port (→ `tx_bytes` AsyncFIFO) |
|
||
| 0x0100–0x0FFF | RX ring buffer in SPRAM (15 × 256-byte pages, pages 1–15) |
|
||
| 0x0100–0x1FFF | any read ≥ 0x0100 streams from SPRAM (DMA path); the ring proper is pages 1–15 above |
|
||
|
||
---
|
||
|
||
## Key Registers
|
||
|
||
| Addr | Name | Notes |
|
||
|---|---|---|
|
||
| 0x00 | NCRA | [0]=RESET self-clears; pulses `ncra_rst` to sync domain |
|
||
| 0x08 | IMR | Interrupt mask |
|
||
| 0x09 | IR | Write-1-to-clear. [1]=RI, [2]=TI. INT_N asserts when IR & IMR ≠ 0 |
|
||
| 0x16–17 | RWP | RX write pointer — updated by sync domain via `rx_wptr` FIFO |
|
||
| 0x18–19 | RRP | RX read pointer — GC writes after consuming frames |
|
||
| 0x20–25 | PAR0–5 | MAC address; also forwarded to W5500 as SHAR |
|
||
| 0x31 | NWAYS | Hardcode **0x17** (100M full-duplex link up, autoneg complete) |
|
||
| 0x3A | HIPR | Hardcode **0x01** (BBA present) |
|
||
| 0x48 | TXDATA | GC streams TX frame bytes here |
|
||
|
||
---
|
||
|
||
## Module Breakdown
|
||
|
||
| Module | Domain | File |
|
||
|---|---|---|
|
||
| `BBATop` | all | `exi_bba/bba_top.py` |
|
||
| `ExiCapture` | creates exiclk/exiclk_n/exififo_* (+sync side) | `exi_bba/exi_capture.py` |
|
||
| `SPIMode3Slave` | exiclk / exiclk_n | `exi_bba/spi_mode3_slave.py` |
|
||
| `BBARegisterFile` | sync | `exi_bba/bba_register_file.py` |
|
||
| `SPRAMArbiter` | sync | `exi_bba/spram_arbiter.py` |
|
||
| `RXFrameAssembler` | sync | `exi_bba/rx_frame_assembler.py` |
|
||
| `TXFrameDrain` | sync | `exi_bba/tx_frame_drain.py` |
|
||
| `W5100ParallelMaster` | sync | `exi_bba/w5100_parallel_master.py` (default eth) |
|
||
| `W5500SPIMaster` | sync | `exi_bba/w5500_spi_master.py` (alt eth) |
|
||
| `EEPROMModel` | sync | `exi_bba/eeprom_model.py` |
|
||
|
||
`ExiCapture` **creates the four pin-driven clock domains** (so the byte FIFOs
|
||
can name them) and wraps `SPIMode3Slave` plus the rx/tx byte AsyncFIFOs.
|
||
`SPIMode3Slave` just uses `m.d.exiclk` / `m.d.exiclk_n`; it has no `domain`
|
||
parameter any more. `BBARegisterFile` consumes the rx byte stream and
|
||
proactively pushes read responses into the tx byte FIFO — it never sees the
|
||
per-bit SPI cadence.
|
||
|
||
**Everything except the bit engine is in one `sync` domain.** The old separate
|
||
`exi` domain was merged into `sync`, which deleted six FIFOs (SPRAM req/rsp,
|
||
RWP, RRP, and the TX byte/length pairs became `SyncFIFO`s) and three
|
||
`PulseSynchronizer`s (now plain wires). The SPRAM response FIFO was removed
|
||
outright — the register file handles responses directly, which is what finally
|
||
closed the DMA read offset at 27 MHz.
|
||
|
||
---
|
||
|
||
## CDC Signal Inventory
|
||
|
||
Only **two** crossings remain, both byte-level, both with a full byte-time
|
||
(~296 ns at 27 MHz) of slack:
|
||
|
||
| Signal | Direction | Primitive |
|
||
|---|---|---|
|
||
| RX byte stream | `exififo_w` → `sync` | `AsyncFIFO` 8-bit, depth=4 |
|
||
| TX byte stream | `sync` → `exififo_r` | `AsyncFIFO` 8-bit, depth=2 |
|
||
| cs_active (transaction in progress) | EXI pin → `sync` | `FFSynchronizer` (stages=2) |
|
||
|
||
Everything that used to cross `exi`↔`sync` is now a same-domain `SyncFIFO` or
|
||
a plain wire. Note what is deliberately NOT here: the EXI CLK/MOSI/CS pins are
|
||
no longer synchronised into a fast domain — CLK *is* the clock and CS *is* the
|
||
async reset. Synchronising them was the bug, not the safety measure.
|
||
|
||
⚠️ **The FIFO domains must stay reset-less.** `exififo_w`/`exififo_r` are not
|
||
reset by CS: resetting one side of an `AsyncFIFO` desynchronises its gray
|
||
pointers against the other side. They are also both **posedge**, because
|
||
`AsyncFIFO` instantiates a posedge-only reset CDC internally and cannot host a
|
||
negedge side; the engine pops in `exififo_r` and hands the byte to its negedge
|
||
output stage through a plain register (a half-period path, 18.5 ns at 27 MHz).
|
||
|
||
---
|
||
|
||
## W5500 Configuration (on NCRA reset)
|
||
|
||
The W5500 selects the register **block** via the BSB field of the control byte,
|
||
NOT via the address — so register addresses below are **block offsets**, not flat
|
||
0x4000-style addresses (see `_W5500_*` and `_CTRL_*` in `w5500_spi_master.py`).
|
||
```
|
||
1. Write MR = 0x80 (common block, offset 0x0000) software reset
|
||
2. Wait ~1 ms
|
||
3. Write SHAR = MAC (common block, offset 0x0009, 6 bytes from PAR0–5)
|
||
4. Write S0_MR = 0x04 (socket-0 reg block, offset 0x0000) MACRAW
|
||
5. Write S0_CR = 0x01 (socket-0 reg block, offset 0x0001) OPEN
|
||
6. Write S0_IMR = 0x05 (socket-0 reg block, offset 0x002C) RECV | SEND_OK
|
||
```
|
||
|
||
W5500 SPI is **Mode 0** (CPOL=0 CPHA=0); SCK = **12 MHz** (the 24 MHz `sync`
|
||
domain ÷ 2 via a toggle clock-enable). Connect W5500 `INT_N` to an FPGA input
|
||
for low-latency RX detection. (The W5500 is the alternate back-end; the W5100
|
||
parallel master is the default — see "W5100 vs W5500".)
|
||
|
||
---
|
||
|
||
## Physical Interface (SP1 Edge Connector)
|
||
|
||
- PCB must be **1.2 mm thick, ENIG finish**.
|
||
- Staggered (not mirrored) top/bottom contact rows — same geometry as PCI/ISA.
|
||
- Derive exact pad geometry from **SP1ETH KiCad project** (silverstee1/SP1ETH),
|
||
cross-referenced with ETH2SP1 (LaserBear). Do not rely on YAGCD alone.
|
||
- Add **100 µF bulk cap** on the interposer near FPGA power pins (3.3 V budget
|
||
is tight: iCEbreaker ~80 mA + W5500 ~150 mA ≈ 230 mA).
|
||
- **Pin 5 is 12 V — do not connect to FPGA I/O.** Test point or leave open.
|
||
- `EXTIN` (pin 1): tie to 3.3 V via 10 kΩ — required for GC device enumeration.
|
||
- All signal levels are 3.3 V. No level shifting needed.
|
||
|
||
---
|
||
|
||
## ⚠️ Open compatibility gaps — see `docs/bba_compat_todo.md`
|
||
|
||
Known differences between this implementation and what real GC software
|
||
expects, with the evidence for each, are tracked in
|
||
**`docs/bba_compat_todo.md`**. Summary of what is still OPEN:
|
||
|
||
1. ✅ **Hardware RX filtering** — `S0_MR` sets the MACRAW **MF** bit, so the
|
||
chip delivers own-MAC + broadcast only; `NCRB.PR` clears it for
|
||
promiscuous. Running unfiltered **hung Swiss** on "Initialising Network":
|
||
libogc2's `bba_interrupt()` loops until `IR & IMR` is zero, and every
|
||
frame on the segment re-set `RI` faster than it could clear them, so it
|
||
livelocked. Unplugging the cable let Swiss through, which identified it.
|
||
Still to do: refresh `SHAR` on a PAR0–5 write (libogc2 reads the MAC
|
||
rather than writing it, so this path is safe today).
|
||
2. ✅ **Ring geometry** — `BP`/`RHBP` are honoured, with garbage rejected and
|
||
`geom_honoured` exposed. A ring must still be larger than
|
||
`_MAX_FRAME_PAGES` (6) or the worst-case reservation drops every frame.
|
||
3. ✅ **`NCRB` (0x01)** — decoded, stored and exported as `reg.ncrb`.
|
||
4. ✅ **Hardware defaults** — every register libogc2 writes now stores and
|
||
reads back (`NWAYC` `0x84`, `MISC` `0x3C`, …). Status registers
|
||
(`LTPS`/`LRPS`/`TWP`/`TRP`/`TXFIFOCNT`) still read `0x00` deliberately.
|
||
|
||
Keep that file updated in the same commit whenever one is closed.
|
||
|
||
---
|
||
|
||
## ⚠️ Which driver actually talks to us — and the TWO interrupt levels
|
||
|
||
**Swiss has several ethernet back-ends, and `cube/patches/bba/` holds one file
|
||
per chip** — `bba.c` (real DOL-015 / MX98730EC), `enc28j60.c`, `w5500.c`,
|
||
`w6100.c`, `w6300.c`. Only `bba.c` is our chain; the others are different
|
||
adapters (Swiss also ships `eth2gc` and `gcnet` artwork). When reading Swiss,
|
||
check which file you are in.
|
||
|
||
**Swiss uses TWO different drivers, and we meet both:**
|
||
|
||
| context | driver | command-space form |
|
||
|---|---|---|
|
||
| Swiss's own menu/network (`if_configex`) | **libogc2** `lwip/arch/gc/netif/gcif.c` | 2-byte header + data |
|
||
| patches injected into a GAME | **Swiss** `cube/patches/bba/bba.c` | one 4-byte transfer |
|
||
|
||
`cube/swiss/source/bba.c` is only the app-side glue: it calls `if_configex()`
|
||
and then picks a `GetExiSpeed` by the interface name libogc2 registered
|
||
(`'e'` → MX98730EC, `'E'` → ENC28J60, `'W'` → W5500, `'w'` → W6X00).
|
||
Detection is by **EXI device ID**:
|
||
|
||
```c
|
||
if ((location & LOC_SERIAL_PORT_1) && EXI_GetID(EXI_CHANNEL_0, EXI_DEVICE_2, &id)) {
|
||
switch (id) { case EXI_W5500_ID: case EXI_MX98730EC_ID: case EXI_ENC28J60_ID:
|
||
```
|
||
|
||
so returning `04 02 02 00` is what makes Swiss treat us as a real BBA and hand
|
||
us to the libogc2 path.
|
||
|
||
### ✅ FIXED 2026-10-10 — the COMMAND-level interrupt (showstopper)
|
||
|
||
**The BBA has two interrupt levels and we implemented only one.** Both drivers'
|
||
handlers are identical in shape:
|
||
|
||
```c
|
||
/* libogc2 bba_event_handler, and Swiss's exi_coroutine */
|
||
status = bba_cmd_in8(0x03); /* COMMAND level */
|
||
bba_cmd_out8(0x02, BBA_CMD_IRMASKALL); /* mask while working */
|
||
if (status & 0x80) bba_interrupt(); /* ONLY then the MAC level */
|
||
...
|
||
bba_cmd_out8(0x03, status); /* write-1-clear */
|
||
bba_cmd_out8(0x02, BBA_CMD_IRMASKNONE);
|
||
```
|
||
|
||
and `bba_interrupt()` is the function that reads register-space `IR`/`IMR` and
|
||
calls `bba_start_rx()`. **Bit 7 of command register `0x03` means "a MAC
|
||
interrupt is pending".** `r_cmd_ir` was write-1-clear and nothing ever set it,
|
||
so a command read of `0x03` returned `0x00`, every branch of both handlers fell
|
||
through, and **neither driver would ever have processed a received frame.**
|
||
|
||
⚠️ **Our own tooling could not have caught this.** `gc_bba_test` and `netboot`
|
||
poll `RWP`/`IR` directly and never use the interrupt path, so the entire
|
||
command-level mechanism was untested and unused — green suite, dead interface.
|
||
|
||
Now: `cmd_ir[7]` is a **live level** equal to `(IR & IMR) != 0`, and
|
||
`INT_N = (cmd_ir & cmd_imr) != 0`.
|
||
- A level, not a latch, because both drivers clear the MAC `IR` inside
|
||
`bba_interrupt()` and only then write-1-clear command `0x03`. A latched bit
|
||
would be cleared by that write even if a new frame had arrived in between;
|
||
a level stays asserted exactly as long as the MAC condition does.
|
||
- `INT_N` honours the **command-level** mask, which is how both drivers stop
|
||
the line re-asserting while their handler runs (`IRMASKALL` = `0x00` on
|
||
entry, `IRMASKNONE` = `0xF8` on exit). Ignoring it is an interrupt storm.
|
||
- `r_cmd_imr` resets to `0xF8` (unmasked). The real power-on value is unknown;
|
||
both drivers set it explicitly first thing, so a permissive default costs
|
||
them nothing and keeps `INT_N` usable for software that never writes it.
|
||
|
||
Tests: `bba_register_file` **T13** (bit 7 clear when idle, set after an RI with
|
||
`IMR` enabling it, `INT_N` asserted) and **T13b** (`IRMASKALL` drops `INT_N`
|
||
but must NOT clear the pending status). Both mutation-tested.
|
||
|
||
### Command register map — implemented 2026-10-10
|
||
|
||
Enumerated from libogc2's `gcif.c` rather than guessed at random. Swiss's patch
|
||
driver touches only `0x02` and `0x03`; everything else here is libogc2.
|
||
|
||
| reg | access | who / what | our behaviour |
|
||
|---|---|---|---|
|
||
| `0x01` | read | revision ID → `priv->revid` | **`0x00`**, read-only |
|
||
| `0x02` | write | interrupt mask (`IRMASKALL` `0x00` / `IRMASKNONE` `0xF8`) | gates `INT_N` |
|
||
| `0x03` | read, write-1-clear | interrupt status | bit 7 = live MAC level |
|
||
| `0x04` | write, **2 bytes** | `priv->devid` = `0xD107` | stored (last byte) |
|
||
| `0x05` | write | `priv->acstart` = `0x4E` | stored |
|
||
| `0x08` | read, **4 bytes** | challenge | stored/readable |
|
||
| `0x09` | write, **4 bytes** | response | stored |
|
||
| `0x0B` | read | challenge/response status | stored/readable |
|
||
| `0x0F` | read (slow, 200 µs) | in `__bba_reset()`, result **DISCARDED** | stored/readable |
|
||
| `> 0x0F` | — | nothing | reads `0x00` |
|
||
|
||
**`0x01` = `0x00` is a real-hardware value, not a guess:** the BBA never
|
||
received a functional firmware revision that incremented it. ⚠️ Do NOT take
|
||
libogc2's `priv->revid = 0xf0` as the device value — that is its pre-init
|
||
placeholder, overwritten by whatever this read returns. `revid`, `devid` and
|
||
`acstart` feed exactly one thing, `bba_calc_response()`, and that path is
|
||
device-initiated (status bit `0x10`), so for us they are functionally inert.
|
||
|
||
Everything from `0x04` up is **plain read/write storage — a GUESS.** Both
|
||
drivers treat these as write-only and never read them back, so
|
||
read-what-you-wrote is the least surprising behaviour and lets our own tests
|
||
confirm the write landed. The alternative (ignore them, as before) is strictly
|
||
worse: software that writes them was silently dropped.
|
||
|
||
⚠️ `0x0F` deserves a note: `bba_cmd_in8_slow(0x0F)` sits in `__bba_reset()`
|
||
between two 10 ms delays, reads one byte with CS held for 200 µs, and
|
||
**discards the result**. A read whose value is thrown away but whose timing is
|
||
deliberate looks like a hardware reset strobe. We treat it as an ordinary
|
||
register; if a real-hardware reset ever turns out to depend on it, this is the
|
||
place to add the side effect. The NCRA reset follows immediately in that
|
||
sequence anyway, which is why ignoring it costs nothing today.
|
||
|
||
**Measured cost** (A/B on identical seeds, 2026-10-10): +104 `SB_DFFE`,
|
||
+141 `SB_LUT4`, and `clk` 27.04 → 26.39 MHz on seed 2 while seed 1 went
|
||
28.72 → 29.07, i.e. about −2.4% on the worst seed with the rest inside
|
||
placement noise. 8/8 seeds still close; best seed 3 at 29.74 MHz = 1.24×. The
|
||
storage is a full low nibble (`0x00`–`0x0F`) rather than only the registers
|
||
listed above, deliberately: a sparse decode would silently drop writes to
|
||
anything unlisted, which is the behaviour being fixed. If `clk` margin ever
|
||
needs reclaiming, this array is the obvious thing to narrow.
|
||
|
||
Test: `bba_register_file` **T14** covers the whole table — revision ID value
|
||
and read-only-ness, write/read-back on `0x04`/`0x05`/`0x09`/`0x0B`/`0x0F`, and
|
||
`0x20` still reading `0x00` so a stray high register cannot alias onto a real
|
||
one. Mutation-tested: making `0x01` plain storage fails it, and dropping the
|
||
`0x04+` writes (the old behaviour) fails it.
|
||
|
||
---
|
||
|
||
## TX path — libogc writes NO TWD (found 2026-10-10)
|
||
|
||
`__bba_link_tx` is **four transactions**:
|
||
|
||
| # | transaction |
|
||
|---|---|
|
||
| 0 | read reg `0x0031` (NWAYS) — check the link first |
|
||
| 1 | **one** register-space write to `0x0048`, the whole frame |
|
||
| 2 | read reg `0x0000` (NCRA) |
|
||
| 3 | write reg `0x0000` = `0x04` (NCRA ST) |
|
||
|
||
⚠️ **No driver writes `TWD_LO`/`TWD_HI` — not Swiss, not libogc, not
|
||
libogc2** (Swiss: `bba_output` is `bba_outs(BBA_WRTXFIFOD, data, size)` then
|
||
NCRA ST1; libogc2: `bba_outsregister(BBA_WRTXFIFOD)` then NCRA ST1). So the
|
||
device has to COUNT the bytes it was handed. The TWD path is OUR test app's
|
||
convention, not a driver's — which is why the "GC sets TWD before streaming"
|
||
note in the TWD fix was self-confirming: `t_tx` writes TWD because we wrote
|
||
`t_tx`. `BBARegisterFile` now keeps `txdata_cnt`,
|
||
incremented per byte accepted at `0x0048`, and the NCRA ST fallback queues
|
||
**that** rather than `tx_frame_len`. Previously it queued `tx_frame_len`,
|
||
which a driver that skips TWD never sets — so the frame would have gone out
|
||
with a stale or zero length. `bba_register_file` **T11** drives TX in libogc's
|
||
shape with no TWD and is mutation-tested: switching the ST path back to
|
||
`tx_frame_len` queues length 0. `gc_bba_test` has the hardware counterpart
|
||
(**TX without TWD**).
|
||
|
||
The TWD path stays, and is still preferred when present: writing the length
|
||
before streaming is what lets `TXFrameDrain` consume concurrently and keeps
|
||
`tx_bytes` a rate-matching buffer instead of whole-frame storage (see the TWD
|
||
fix note). Swiss writes TWD; libogc does not. Support both.
|
||
|
||
⚠️ **Swiss rounds the DMA length UP to 32 bytes** (`exi_dma_write` sets
|
||
`EXI[..][2] = OSRoundUp32B(len)`), so TXDATA receives up to 31 trailing bytes
|
||
beyond the frame. With a counted length the frame goes out that much longer.
|
||
Real hardware must behave the same way, since Swiss works on it — trailing
|
||
padding past the IP total length is ignored by receivers. Worth knowing before
|
||
treating a slightly-too-long TX frame as a bug. Note a 1514-byte frame rounds
|
||
to 1536, over the 1518 maximum.
|
||
|
||
⚠️ **libogc streams the payload with `EXI_Dma`** — the 32-byte-aligned bulk via
|
||
`EXI_Dma` after a `PPCDCacheStoreAsync`, then `EXI_ImmEx` for the remainder.
|
||
This file says elsewhere that EXI DMA *writes* are avoided because the GC's
|
||
DMA-write engine has a 1-bit-shift bug; that is true of **Swiss**, not of
|
||
libogc, which uses them routinely for TX. From the FPGA's side a DMA write is
|
||
just bytes clocked in, so nothing in the gateware changes — but it means the
|
||
DMA-write path WILL be exercised by libogc-based software, and if the shift bug
|
||
is real it will show up as corrupted TX frames that are not our fault and not
|
||
fixable here. Worth knowing before chasing it in the gateware.
|
||
|
||
---
|
||
|
||
## RX Ring Descriptor — the format GC software actually expects
|
||
|
||
✅ **Corrected 2026-10-10 from libogc's own driver.** The 4 bytes at each ring
|
||
page start are a **packed 32-bit little-endian word**, not a byte-wise layout:
|
||
|
||
| bits | field |
|
||
|---|---|
|
||
| 0–11 | `next_packet_ptr` — the PAGE the NEXT frame starts at |
|
||
| 12–23 | `packet_len` — total bytes, **including** these 4 |
|
||
| 24–31 | `status` — `0x80` and `0x04` are errors; `0x00` is good |
|
||
|
||
```
|
||
Byte 0: next_packet_ptr[7:0]
|
||
Byte 1: packet_len[3:0] << 4 | next_packet_ptr[11:8]
|
||
Byte 2: packet_len[11:4]
|
||
Byte 3: status
|
||
```
|
||
|
||
⚠️ **This was WRONG until 2026-10-10 and nothing we had built could have
|
||
caught it.** The assembler wrote `00 00 len[11:8] len[7:0]` and every reader we
|
||
had — `netboot`, `t_dma`, the testbenches — was written by us to match it, so
|
||
the whole stack was self-consistent and green while being incompatible with the
|
||
only driver that matters. The error was found by disassembling
|
||
`/opt/devkitpro/libogc/lib/cube/libbba.a` (`gcif.o`, `bba_interrupt`):
|
||
|
||
```
|
||
lwbrx r9,0,r9 # the 4 bytes as a LITTLE-ENDIAN word
|
||
rlwinm r4,r9,20,20,31 # packet_len = (word >> 12) & 0xFFF
|
||
addi r4,r4,-4 # payload = packet_len - 4
|
||
cmpwi r4,1540 # ... must be <= 1540
|
||
srwi r9,r9,24 # status = word >> 24
|
||
and. r9,r9,r18 # r18 = 0x..84 -> errors are 0x80 | 0x04
|
||
lwz r9,cur_descr # and the low 12 bits go STRAIGHT to RRP:
|
||
# byte 0 -> 0x0018 (RRP_LO)
|
||
rlwinm r9,r31,24,28,31 # bits 11:8 -> 0x0019 (RRP_HI)
|
||
```
|
||
|
||
Under the old format a 342-byte frame (`00 00 01 5A`) decodes as
|
||
`packet_len = 16` → an 12-byte frame, with `next_packet_ptr = 0` pointing at
|
||
the reserved page. The length semantics and addressing were already right
|
||
(descriptor at `page<<8`, body at `+4`, length includes the descriptor); only
|
||
the packing was wrong, and `next_packet_ptr` was never written at all.
|
||
|
||
**Lesson worth keeping:** a green suite proves self-consistency, not
|
||
compatibility. For anything an external driver parses, check it against that
|
||
driver — the binary is on disk and `objdump` settles in minutes what guessing
|
||
could not. This is the third time a *guessed* interface cost real hardware
|
||
debugging (after the EXI header format and the Mode 2/3 mix-up).
|
||
|
||
`rx_frame_assembler`'s testbench now decodes descriptors with
|
||
`libogc_decode()`, which reproduces the instruction sequence above, so the
|
||
tests check the driver's algorithm rather than our own packing.
|
||
|
||
⚠️ **The status byte must be written EXPLICITLY** (`WRITE_D3`). The placeholder
|
||
issues four writes but the first lands twice, so `desc_base+3` is never
|
||
written; that was harmless while the length write covered byte 3. Left
|
||
unwritten it hands the driver whatever was in that SPRAM cell, and a stale
|
||
`0x80` or `0x04` discards the frame as an error. Simulation flags it as an
|
||
unwritten cell; real SPRAM has no such tell.
|
||
|
||
---
|
||
|
||
## SPRAM Notes
|
||
|
||
- iCE40UP5K has 128 KB SPRAM (SB_SPRAM256KA, 16-bit wide).
|
||
- **1-cycle synchronous read latency** — result of read at cycle N is valid at N+1.
|
||
- Byte writes via `MASKWREN`: lower byte = `0b0011`, upper byte = `0b1100`.
|
||
- Address to SPRAM = byte_address >> 1.
|
||
- ETH writes take priority over EXI reads in the arbiter (safe by ring-buffer
|
||
invariant: GC only reads pages the ETH engine has already finished).
|
||
|
||
---
|
||
|
||
## GC Initialisation Sequence — read out of libogc, 2026-10-10
|
||
|
||
⚠️ **The sequence below replaces a GUESSED one.** The old version said the
|
||
driver *writes* PAR0–5, writes `MAR0–7 = 0xFF`, `ANALOG = 0xD6`, `NWAYC`,
|
||
`GCA` and `IMR = 0x86`. Disassembling libogc's `__bba_init`
|
||
(`/opt/devkitpro/libogc/lib/cube/libbba.a`, `gcif.o`) shows otherwise. This is
|
||
the actual transaction list, in order:
|
||
|
||
| # | transaction | note |
|
||
|---|---|---|
|
||
| 0 | **cmd** write reg 2 = `0x00` | interrupt mask (IRMASKALL) |
|
||
| 1 | reg `0x0060` = `0x00` | |
|
||
| 2 | **cmd** read reg `0x0F` | |
|
||
| 3 | reg `0x0000` = `0x01` | **NCRA reset** |
|
||
| 4 | reg `0x0000` = `0x00` | |
|
||
| 5 | **cmd** read reg `0x01` | |
|
||
| 6,7 | **cmd** writes reg `0x04`, `0x05` | |
|
||
| 8–15 | reg `0x0058`–`0x005F` = `80 00 03 83 32 FE 1F 1F` | PHY/analog block |
|
||
| 16,17 | reg `0x0001` = `0x52`, `0x0060` = `0x74` | |
|
||
| 18,19 | reg `0x0014` = `0x00`, `0x0015` = `0x06` | |
|
||
| 20 | reg `0x0050` = `0x80` | |
|
||
| 21–24 | reg `0x000C/0D` = `0x00`, `0x000A` = `0x01`, `0x000B` = `0x00` | |
|
||
| 25,26 | reg `0x001A/1B` = `0x000F` | **RHBP** — ring high boundary = page 15 |
|
||
| 27,28 | reg `0x0016/17` = `0x0001` | **writes RWP** |
|
||
| 29,30 | reg `0x0018/19` = `0x0001` | **writes RRP** |
|
||
| 31 | reg `0x0032` = `0x08` | |
|
||
| 32 | reg `0x0000` = `0x08` | **NCRA SR — start receive** |
|
||
| 33–36 | reg `0x0030` read, then writes `0x04`, `0x0C` | |
|
||
| 37 | reg `0x0020`, **6-byte READ** | **reads PAR0–5 — the MAC** |
|
||
| 38 | reg `0x0009` = `0xFF` | IR: clear all |
|
||
| 39 | reg `0x0008` = `0xDF` | **IMR — not `0x86`** |
|
||
| 40 | **cmd** write reg 2 = `0xF8` | IRMASKNONE (enable) |
|
||
|
||
Three corrections that matter:
|
||
- **The driver READS the MAC from PAR0–5; it does not write one.** So the
|
||
PAR0–5 reset value is what the console will use — keep a valid Nintendo OUI
|
||
there (`00:09:BF:00:00:01`). It is a single 6-byte register-space read.
|
||
- **`IMR` is `0xDF`**, and `IR` is cleared with `0xFF` first.
|
||
- **It writes `RWP` (0x0016/17) and `RHBP` (0x001A/1B)**, which the register
|
||
file does not accept. Harmless today — `RHBP` is `0x000F`, matching our
|
||
hardcoded 15-page ring, and the NCRA reset already puts `RWP` at page 1 —
|
||
but it is a real divergence: a driver that configured a SMALLER ring would
|
||
be ignored, and our assembler would write outside it.
|
||
- Registers `0x0058`–`0x005F`, `0x0060`, `0x0001`, `0x0014/15`, `0x0050`,
|
||
`0x000A`–`0x000D`, `0x0030`, `0x0032` are not decoded. Writes fall through
|
||
the `Switch` with no `Case`, so they are ignored rather than aliased —
|
||
which is the safe failure, but worth knowing when a driver expects one back.
|
||
|
||
`gc_bba_test` suite 7 (**libogc init**) replays this list on hardware and
|
||
checks the shapes: `cmd-space write` (2-byte header + data phase, read back),
|
||
`MAC readable` (one 6-byte PAR read, Nintendo OUI), and `init replay` (all 30
|
||
register writes in order, then NWAYS/HIPR/NCRA.SR must still be correct — the
|
||
old guessed header format turned a NWAYS read into an NCRA reset, so a
|
||
misdecoded write is a real risk, not a theoretical one).
|
||
|
||
---
|
||
|
||
## Implementation Notes & Gotchas
|
||
|
||
- **`NWAYS` must return `0x17` always.** GC polls it to confirm 100 Mbps link
|
||
before enabling RX. Do not attempt to reflect real W5500 link status.
|
||
- **`EEPROMModel` can be stubbed initially.** Many GC BBA drivers write their own
|
||
MAC to PAR0–5 rather than using the EEPROM. Pre-populate PAR0–5 reset state
|
||
with a valid Nintendo OUI MAC (`00:09:BF:xx:xx:xx`).
|
||
- **`tx_load` timing in `SPIMode3Slave`:** pulses at CS assertion (first byte)
|
||
and after each complete received byte. Upstream must register next TX byte
|
||
within one `exi` clock.
|
||
- **No PLL.** The bit engine is clocked by the EXI pin; `sync` is HFOSC÷2.
|
||
Do not reintroduce an oversampling capture domain — see the clocking note
|
||
under "Key Architecture Decisions" for why it cannot work on this part.
|
||
- **TX buffer selection (NCRA ST bits):** Ignore buffer select (ST1 vs ST0).
|
||
Treat any non-zero ST as a TX trigger.
|
||
- **If nextpnr fails capture-domain timing at 54 MHz:** the isolated bit engine
|
||
closes ~91 MHz, so 54 has margin; if a seed fails, sweep seeds
|
||
(`synth.py --seeds N`) or instruct users to configure Swiss to a lower EXI
|
||
clock index.
|