Measure the throughput limit: window x RTT, not the adapter
Swiss dumping a disc over the adapter measures 850 KB/s (6.97 Mbit/s, 25.8% of the 27 Mbit/s EXI ceiling). The capture decomposes it completely: 18395 data frames / 9200 ACKs = 2.0 frames per ACK 18.84 MB / 9200 = 2048 B per round trip 22.78 s / 9200 = 2.48 ms per round trip 2048 B / 2.48 ms = 826 KB/s ~= the measured 850 So the flow is window x RTT bound and nothing else. Payload per frame is exactly 1024 B (wire 1078 minus 54 of headers) against a 2920 B window, so two segments fit in flight. Zero retransmissions means the 15-page ring never overflowed; 0.2% stall time means nothing waits on the disc; our W5100 bus engine sits at ~17% utilisation. The adapter is not the constraint. Levers, and the first two are not ours: a larger lwIP TCP_SND_BUF (scales almost linearly while window-bound), and full-MSS segments -- Swiss writes 1024 B at a time, so the same window and RTT at 1460 B would give ~1150 KB/s, a 1.43x gain for free. Third is a shorter round trip, which is GC-side interrupt dispatch plus lwIP, plus EXI lock contention: libogc2's bba_event_handler defers when EXI_Lock fails, so RX queues behind TX on the single channel. ⚠️ This REVERSES an earlier recommendation in the TODO. RX interrupt coalescing (NCRB.RXINTC, RXINTT) was listed as a throughput lever; for a window-bound flow it makes things worse, because delaying the ACK's interrupt lengthens the round trip and throughput is bytes-per-round-trip / RTT. The measurement showed the guess was backwards. Noted in item 3 as "implement for fidelity, not for speed". analyse-pcap.py now derives frames-per-ACK, bytes-per-round-trip and the implied RTT itself, identifies a window-bound flow, quantifies the full-MSS gain, and warns against coalescing -- so the next person gets the conclusion rather than the raw numbers. Self-test G replays this exact shape. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -168,6 +168,54 @@ For the record, libogc2 writes `TLBP = 0x0000`, `BP = 0x0001`,
|
||||
`RHBP = 0x000F` — exactly the old hardcoded layout, which is why this never
|
||||
showed up in practice.
|
||||
|
||||
## Throughput: measured 850 KB/s, and it is NOT the adapter
|
||||
|
||||
**Measured 2026-10-10**, Swiss dumping a disc over the adapter, captured on
|
||||
the PC and analysed with `scripts/analyse-pcap.py`:
|
||||
|
||||
```
|
||||
GC -> PC 18395 frames, 19.83 MB, 850 KB/s (6.97 Mbit/s, 25.8% of the EXI ceiling)
|
||||
807 frames/s, mean frame 1078 B, retransmissions 0
|
||||
PC -> GC 9200 frames, mean 54 B (pure ACKs)
|
||||
GC advertised window 2920 B; gaps >5 ms total 56 ms (0.2% of 22.78 s)
|
||||
```
|
||||
|
||||
**The flow is window × RTT bound and nothing else:**
|
||||
|
||||
| derived | value |
|
||||
|---|---|
|
||||
| data frames per ACK | 2.0 |
|
||||
| payload per round trip | 2048 B |
|
||||
| implied RTT | 2.48 ms |
|
||||
| 2048 B / 2.48 ms | **826 KB/s** ≈ the measured 850 |
|
||||
| payload per frame | **1024 B** (wire 1078 − 54 of headers) |
|
||||
| window ÷ segment | 2920 / 1024 → **2 segments in flight** |
|
||||
|
||||
So the sender can have 2048 bytes outstanding and each round trip costs
|
||||
2.48 ms. **Zero retransmissions** means the 15-page ring never overflowed and
|
||||
we drop nothing; **0.2% stall time** means nothing is waiting on the disc.
|
||||
Our W5100 bus engine runs at roughly 17% utilisation at this rate. The adapter
|
||||
is not the constraint.
|
||||
|
||||
The levers, in order of effect, and **none of the first two are ours**:
|
||||
|
||||
1. **A larger TCP send buffer on the GC** (lwIP `TCP_SND_BUF`). Throughput
|
||||
scales with it almost linearly while the flow is window-bound.
|
||||
2. **Full-MSS segments.** Swiss writes 1024 bytes at a time, so a 2920 B
|
||||
window carries 2048 B of payload. At 1460 B segments the same window and
|
||||
RTT would give ~**1150 KB/s, a 1.43× gain** for free.
|
||||
3. **A shorter round trip** — throughput scales 1/RTT. The 2.48 ms is GC-side
|
||||
interrupt dispatch and lwIP, plus EXI lock contention: libogc2's
|
||||
`bba_event_handler` bails out if `EXI_Lock` fails and defers until the TX
|
||||
path unlocks, so RX handling queues behind TX on the single channel.
|
||||
|
||||
⚠️ **RX interrupt coalescing would make this WORSE, and that reverses an
|
||||
earlier recommendation in this file.** `NCRB.RXINTC` asks for 2 packets per
|
||||
interrupt and `RXINTT` adds a timer; both were listed as throughput levers.
|
||||
For a **window-bound** flow they are not: delaying the ACK's interrupt
|
||||
lengthens the round trip, and throughput is bytes-per-round-trip ÷ RTT. The
|
||||
measurement is what showed this — the guess was backwards.
|
||||
|
||||
## 3. `NCRB` (0x01) is not decoded at all
|
||||
|
||||
**Status: ✅ DECODED 2026-10-10** — stored, readable, and exported as
|
||||
@@ -186,6 +234,12 @@ Bits, from libogc2:
|
||||
We interrupt per frame where libogc2 asks for every 2 packets, which is benign
|
||||
(more interrupts than requested). `PR`/`PM`/`AB` feed item 1.
|
||||
|
||||
⚠️ **Do NOT implement `RXINTC`/`RXINTT` coalescing as a throughput
|
||||
optimisation.** Measurement showed the bulk flow is window × RTT bound (see
|
||||
"Throughput" above), so delaying an interrupt lengthens the round trip and
|
||||
*lowers* throughput. Implement it for fidelity if a driver depends on the
|
||||
behaviour, not for speed.
|
||||
|
||||
`bba_register_file` **T15** asserts the export is `0x52` with `PR=0`, `PM=0`,
|
||||
`AB=1` — i.e. the real BBA would deliver own-MAC + broadcast only — which is
|
||||
the fact item 1 has to act on.
|
||||
|
||||
+46
-5
@@ -157,13 +157,54 @@ def main():
|
||||
print(f" transactions + W5100 socket bookkeeping.")
|
||||
print()
|
||||
|
||||
# ── Is the bulk direction window x RTT bound? ────────────────────────
|
||||
#
|
||||
# THE decisive statistic, and the one that is easy to miss: if the sender
|
||||
# can only have `window` bytes unacknowledged, throughput is
|
||||
# window / RTT no matter how fast the wire or the adapter is. Derive the
|
||||
# round trip from the capture itself -- one ACK is one round trip -- so
|
||||
# this needs no ping and no assumptions.
|
||||
data_pkts = stats["from_gc"]["pkts"]
|
||||
acks = stats["to_gc"]["pkts"]
|
||||
payload = stats["from_gc"]["payload"]
|
||||
if acks and payload:
|
||||
per_rt = payload / acks
|
||||
rtt = dur / acks
|
||||
print(f"\nround-trip analysis (bulk direction GC -> PC)")
|
||||
print(f" {data_pkts/acks:.1f} data frames per ACK, "
|
||||
f"{per_rt:.0f} B per round trip, "
|
||||
f"implied RTT {rtt*1000:.2f} ms")
|
||||
print(f" {per_rt/rtt/1024:.0f} KB/s = bytes-per-round-trip / RTT"
|
||||
f" <- compare with the measured throughput above")
|
||||
if wins:
|
||||
w = max(wins)
|
||||
seg = payload / data_pkts
|
||||
print(f" GC advertised window {w} B; segments average "
|
||||
f"{seg:.0f} B, so {w//max(int(seg),1)} fit in flight")
|
||||
if per_rt <= w * 1.2:
|
||||
print(f" ⚠️ WINDOW x RTT BOUND: {per_rt:.0f} B in flight "
|
||||
f"against a {w} B window.")
|
||||
print(f" The adapter is not the limit -- the sender "
|
||||
f"cannot have more outstanding.")
|
||||
print(f" Levers, in order of effect:")
|
||||
print(f" 1. a larger TCP send buffer / window on the "
|
||||
f"GC (lwIP TCP_SND_BUF)")
|
||||
if seg < 1400:
|
||||
full = w - (w % 1460) or 1460
|
||||
print(f" 2. full-MSS segments: {seg:.0f} B "
|
||||
f"segments waste the window;")
|
||||
print(f" {full} B per round trip would give "
|
||||
f"{full/rtt/1024:.0f} KB/s ({full/per_rt:.2f}x)")
|
||||
print(f" 3. a shorter round trip -- throughput scales "
|
||||
f"1/RTT")
|
||||
print(f" ⚠️ RX interrupt COALESCING would make this "
|
||||
f"WORSE, not better:")
|
||||
print(f" delaying the ACK's interrupt lengthens the "
|
||||
f"round trip, and")
|
||||
print(f" a window-bound flow pays for that directly.")
|
||||
if wins:
|
||||
print(f"GC advertised TCP window: min {min(wins)} max {max(wins)} "
|
||||
print(f"\nGC advertised TCP window: min {min(wins)} max {max(wins)} "
|
||||
f"last {wins[-1]} B")
|
||||
if max(wins) < 8192:
|
||||
print(f" ⚠️ a {max(wins)} B window caps a single TCP stream at "
|
||||
f"window/RTT; at 1 ms RTT that is "
|
||||
f"{max(wins)/1e-3/1024:.0f} KB/s")
|
||||
if zero_win_spans:
|
||||
print(f"PC saw zero-window from the GC {len(zero_win_spans)} times, "
|
||||
f"total {sum(zero_win_spans)*1000:.0f} ms "
|
||||
|
||||
@@ -129,6 +129,35 @@ if "pcapng" not in (r.stdout + r.stderr):
|
||||
else:
|
||||
print("F pcapng refused with a conversion hint")
|
||||
|
||||
# Scenario G: window x RTT bound -- 2 undersized segments per ACK, which is
|
||||
# the real shape measured on hardware (1024 B segments, 2920 B window).
|
||||
# The tool must call it window-bound, and must warn that RX interrupt
|
||||
# COALESCING would make it worse rather than better.
|
||||
pkts, seq, t = [], 1, 6000.0
|
||||
for i in range(200):
|
||||
pkts.append((t, tcp_frame(GC, PC, MAC_GC, MAC_PC, seq, b"\0" * 1024, win=2920)))
|
||||
seq += 1024
|
||||
t += 0.0002
|
||||
pkts.append((t, tcp_frame(GC, PC, MAC_GC, MAC_PC, seq, b"\0" * 1024, win=2920)))
|
||||
seq += 1024
|
||||
t += 0.0002
|
||||
pkts.append((t, tcp_frame(PC, GC, MAC_PC, MAC_GC, 1 + i, b"", win=65535)))
|
||||
t += 0.0024 # ~2.5 ms per round trip
|
||||
pG = os.path.join(tmp, "g.pcap")
|
||||
write_pcap(pG, pkts)
|
||||
out = run(pG, "--gc-ip", "10.0.0.64")
|
||||
if "2.0 data frames per ACK" not in out:
|
||||
fails.append(f"G: frames-per-ACK not derived\n{out}")
|
||||
elif "WINDOW x RTT BOUND" not in out:
|
||||
fails.append(f"G: window-bound flow not identified\n{out}")
|
||||
elif "COALESCING would make this WORSE" not in out:
|
||||
fails.append(f"G: missing the coalescing counter-warning\n{out}")
|
||||
elif "full-MSS segments" not in out:
|
||||
fails.append(f"G: did not suggest full-MSS segments\n{out}")
|
||||
else:
|
||||
print("G 1024B segments + 2920B window -> window x RTT bound,")
|
||||
print(" full-MSS suggested, coalescing warned against")
|
||||
|
||||
print("\nFAILURES:\n " + "\n ".join(fails) if fails else
|
||||
"\nAll analyse-pcap self-tests passed.")
|
||||
sys.exit(1 if fails else 0)
|
||||
|
||||
Reference in New Issue
Block a user