analyse-pcap: count UDP, because ROM streaming is UDP/FSP not TCP
The 850 KB/s dump measurement does not bound ROM streaming, and the tool could not have measured streaming at all: it only counted TCP payload, so a streaming capture would have reported ~zero throughput. Swiss's ROM streaming runs in the game via the patch driver's own stack (cube/patches/bba/tcpip.c), which contains no TCP whatsoever -- UDP + FSP, plus ICMP and ARP. Reads are sized to the path MTU rather than 1024: _fsp.data_length = MIN(length, env->pmtu - (fsp->data - eth->data)); so ~1446 B of payload per datagram at a 1500 MTU, with QUEUE_SIZE 2 outstanding and udp->checksum = 0x0000 (no payload checksum). None of the menu path's TCP window, lwIP overhead or 1024-byte writes apply. The analyser now counts UDP payload and picks the bulk direction by which way the payload actually flows -- streaming is PC -> GC, the opposite of a disc dump -- with --bulk to_gc to steer the gap analysis. Self-test H replays the streaming shape (1446 B datagrams, 2 per request) and checks the bulk direction is identified and the payload counted. Recorded in the TODO that "can it stream?" is still OPEN, but open on the GC's side: the dump established the adapter has headroom (zero retransmissions, ~17% W5100 bus utilisation, 26% of the EXI ceiling). The hard ceiling is 3375 KB/s and the GC's disc delivers ~2000-3100 KB/s, so full disc rate has no margin -- but games read in bursts, not continuously at peak. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -216,6 +216,40 @@ For a **window-bound** flow they are not: delaying the ACK's interrupt
|
|||||||
lengthens the round trip, and throughput is bytes-per-round-trip ÷ RTT. The
|
lengthens the round trip, and throughput is bytes-per-round-trip ÷ RTT. The
|
||||||
measurement is what showed this — the guess was backwards.
|
measurement is what showed this — the guess was backwards.
|
||||||
|
|
||||||
|
### ⚠️ The 850 KB/s figure does NOT bound ROM streaming — different stack
|
||||||
|
|
||||||
|
The dump was Swiss's **menu** path: libogc2 + lwIP + TCP, 1024-byte
|
||||||
|
application writes, a 2920-byte window. **ROM streaming uses a completely
|
||||||
|
different code path** — Swiss's in-game patch driver and its own stack in
|
||||||
|
`cube/patches/bba/tcpip.c`, which has **no TCP in it at all**:
|
||||||
|
|
||||||
|
- **UDP + FSP** (File Service Protocol), with ICMP and ARP. No TCP, no lwIP,
|
||||||
|
no congestion control, no window.
|
||||||
|
- Reads are sized to the path MTU, not 1024:
|
||||||
|
`_fsp.data_length = MIN(length, env->pmtu - (fsp->data - eth->data))`
|
||||||
|
— about 1446 B of payload per datagram at a 1500 MTU.
|
||||||
|
- `QUEUE_SIZE 2` requests outstanding.
|
||||||
|
- `udp->checksum = 0x0000` — no payload checksum, which is real CPU saved
|
||||||
|
versus TCP.
|
||||||
|
- It runs in the game's context via the patch driver, which uses the leaner
|
||||||
|
4-byte command-space form (see the command-space note in CLAUDE.md).
|
||||||
|
|
||||||
|
So the TCP window × RTT limit is not the operative one there, and the measured
|
||||||
|
850 KB/s says nothing about it. What the dump DID establish, and which carries
|
||||||
|
over, is that the adapter has headroom: **zero retransmissions**, ~17% W5100
|
||||||
|
bus utilisation, 26% of the EXI ceiling.
|
||||||
|
|
||||||
|
**Not yet measured.** `analyse-pcap.py` now counts UDP payload and picks the
|
||||||
|
bulk direction automatically (streaming flows PC → GC, the opposite of a
|
||||||
|
dump), so `--bulk to_gc` on a streaming capture gives the real number. Until
|
||||||
|
that is run, "can it stream?" is open — but it is open on the GC's side, not
|
||||||
|
on the adapter's.
|
||||||
|
|
||||||
|
The hard ceiling remains **3375 KB/s** (27 MHz × 1 bit/clock), and the GC's
|
||||||
|
own disc delivers roughly 2000–3100 KB/s, so there is no headroom to spare if
|
||||||
|
a game wants full disc rate — but games read in bursts, not continuously at
|
||||||
|
peak.
|
||||||
|
|
||||||
## 3. `NCRB` (0x01) is not decoded at all
|
## 3. `NCRB` (0x01) is not decoded at all
|
||||||
|
|
||||||
**Status: ✅ DECODED 2026-10-10** — stored, readable, and exported as
|
**Status: ✅ DECODED 2026-10-10** — stored, readable, and exported as
|
||||||
|
|||||||
+36
-4
@@ -61,11 +61,20 @@ def main():
|
|||||||
ap.add_argument("pcap")
|
ap.add_argument("pcap")
|
||||||
ap.add_argument("--gc-ip", default="10.0.0.64")
|
ap.add_argument("--gc-ip", default="10.0.0.64")
|
||||||
ap.add_argument("--top-gaps", type=int, default=5)
|
ap.add_argument("--top-gaps", type=int, default=5)
|
||||||
|
ap.add_argument("--bulk", choices=("auto", "from_gc", "to_gc"),
|
||||||
|
default="auto",
|
||||||
|
help="which direction carries the payload; 'auto' guesses "
|
||||||
|
"GC->PC (a disc dump). ROM streaming is to_gc.")
|
||||||
a = ap.parse_args()
|
a = ap.parse_args()
|
||||||
gc = socket.inet_aton(a.gc_ip)
|
gc = socket.inet_aton(a.gc_ip)
|
||||||
|
|
||||||
stats = {d: dict(pkts=0, wire=0, payload=0, retx=0, sizes=[], zerowin=0)
|
stats = {d: dict(pkts=0, wire=0, payload=0, retx=0, sizes=[], zerowin=0)
|
||||||
for d in ("from_gc", "to_gc")}
|
for d in ("from_gc", "to_gc")}
|
||||||
|
# Which direction carries the bulk is not knowable until the end, so the
|
||||||
|
# gap analysis needs a hint up front. Two passes would be cleaner; one
|
||||||
|
# pass with a hint is enough, and the hint only affects which direction's
|
||||||
|
# inter-frame gaps get recorded.
|
||||||
|
bulk_hint = a.bulk if a.bulk != "auto" else "from_gc"
|
||||||
seen = {d: set() for d in ("from_gc", "to_gc")}
|
seen = {d: set() for d in ("from_gc", "to_gc")}
|
||||||
t0 = t1 = None
|
t0 = t1 = None
|
||||||
wins, gaps, last_ts = [], [], None
|
wins, gaps, last_ts = [], [], None
|
||||||
@@ -95,6 +104,22 @@ def main():
|
|||||||
s["pkts"] += 1
|
s["pkts"] += 1
|
||||||
s["wire"] += orig
|
s["wire"] += orig
|
||||||
s["sizes"].append(orig)
|
s["sizes"].append(orig)
|
||||||
|
if ip[9] == socket.IPPROTO_UDP:
|
||||||
|
# ROM streaming (Swiss's in-game patch driver) is UDP + FSP, with
|
||||||
|
# no TCP anywhere -- so without this the tool reports a streaming
|
||||||
|
# capture as zero throughput.
|
||||||
|
udp = ip[ihl:]
|
||||||
|
if len(udp) >= 8:
|
||||||
|
ulen = struct.unpack("!H", udp[4:6])[0]
|
||||||
|
s["payload"] += max(ulen - 8, 0)
|
||||||
|
s["udp"] = s.get("udp", 0) + 1
|
||||||
|
if d == "from_gc" or d == "to_gc":
|
||||||
|
pass
|
||||||
|
if d == bulk_hint:
|
||||||
|
if last_ts is not None:
|
||||||
|
gaps.append((ts - last_ts, ts))
|
||||||
|
last_ts = ts
|
||||||
|
continue
|
||||||
if ip[9] != socket.IPPROTO_TCP:
|
if ip[9] != socket.IPPROTO_TCP:
|
||||||
continue
|
continue
|
||||||
tcp = ip[ihl:]
|
tcp = ip[ihl:]
|
||||||
@@ -164,13 +189,20 @@ def main():
|
|||||||
# window / RTT no matter how fast the wire or the adapter is. Derive the
|
# window / RTT no matter how fast the wire or the adapter is. Derive the
|
||||||
# round trip from the capture itself -- one ACK is one round trip -- so
|
# round trip from the capture itself -- one ACK is one round trip -- so
|
||||||
# this needs no ping and no assumptions.
|
# this needs no ping and no assumptions.
|
||||||
data_pkts = stats["from_gc"]["pkts"]
|
# Bulk = whichever direction actually moved the payload.
|
||||||
acks = stats["to_gc"]["pkts"]
|
bulk = max(("from_gc", "to_gc"), key=lambda d: stats[d]["payload"])
|
||||||
payload = stats["from_gc"]["payload"]
|
back = "to_gc" if bulk == "from_gc" else "from_gc"
|
||||||
|
data_pkts = stats[bulk]["pkts"]
|
||||||
|
acks = stats[back]["pkts"]
|
||||||
|
payload = stats[bulk]["payload"]
|
||||||
|
if payload:
|
||||||
|
print(f"\nbulk direction: {'GC -> PC' if bulk == 'from_gc' else 'PC -> GC'}"
|
||||||
|
f" ({stats[bulk].get('udp', 0)} UDP, "
|
||||||
|
f"{data_pkts - stats[bulk].get('udp', 0)} TCP frames)")
|
||||||
if acks and payload:
|
if acks and payload:
|
||||||
per_rt = payload / acks
|
per_rt = payload / acks
|
||||||
rtt = dur / acks
|
rtt = dur / acks
|
||||||
print(f"\nround-trip analysis (bulk direction GC -> PC)")
|
print(f"round-trip analysis")
|
||||||
print(f" {data_pkts/acks:.1f} data frames per ACK, "
|
print(f" {data_pkts/acks:.1f} data frames per ACK, "
|
||||||
f"{per_rt:.0f} B per round trip, "
|
f"{per_rt:.0f} B per round trip, "
|
||||||
f"implied RTT {rtt*1000:.2f} ms")
|
f"implied RTT {rtt*1000:.2f} ms")
|
||||||
|
|||||||
@@ -158,6 +158,39 @@ else:
|
|||||||
print("G 1024B segments + 2920B window -> window x RTT bound,")
|
print("G 1024B segments + 2920B window -> window x RTT bound,")
|
||||||
print(" full-MSS suggested, coalescing warned against")
|
print(" full-MSS suggested, coalescing warned against")
|
||||||
|
|
||||||
|
def udp_frame(src, dst, smac, dmac, payload):
|
||||||
|
udp = struct.pack("!HHHH", 7717, 7717, 8 + len(payload), 0) + payload
|
||||||
|
ip = struct.pack("!BBHHHBBH4s4s", 0x45, 0, 20 + len(udp), 0, 0, 64,
|
||||||
|
socket.IPPROTO_UDP, 0, src, dst)
|
||||||
|
return dmac + smac + b"\x08\x00" + ip + udp
|
||||||
|
|
||||||
|
|
||||||
|
# Scenario H: ROM streaming shape -- UDP/FSP, bulk flowing PC -> GC, with
|
||||||
|
# small requests going the other way. Before UDP was counted this reported
|
||||||
|
# zero throughput, because Swiss's in-game stack has no TCP at all.
|
||||||
|
pkts, t = [], 7000.0
|
||||||
|
for i in range(300):
|
||||||
|
pkts.append((t, udp_frame(GC, PC, MAC_GC, MAC_PC, b"\0" * 20))) # request
|
||||||
|
t += 0.0001
|
||||||
|
for _ in range(2): # 2 queued
|
||||||
|
pkts.append((t, udp_frame(PC, GC, MAC_PC, MAC_GC, b"\0" * 1446)))
|
||||||
|
t += 0.0002
|
||||||
|
t += 0.0008
|
||||||
|
pH = os.path.join(tmp, "h.pcap")
|
||||||
|
write_pcap(pH, pkts)
|
||||||
|
out = run(pH, "--bulk", "to_gc")
|
||||||
|
if "bulk direction: PC -> GC" not in out:
|
||||||
|
fails.append(f"H: bulk direction not detected as PC->GC\n{out}")
|
||||||
|
elif "UDP" not in out:
|
||||||
|
fails.append(f"H: UDP frames not counted\n{out}")
|
||||||
|
else:
|
||||||
|
# 600 x 1446 B over ~0.39 s should be well above 1 MB/s
|
||||||
|
kbs = float(out.split("throughput")[2].split("KB/s")[0].strip())
|
||||||
|
if kbs < 500:
|
||||||
|
fails.append(f"H: UDP payload undercounted, {kbs} KB/s")
|
||||||
|
else:
|
||||||
|
print(f"H UDP/FSP streaming shape -> bulk PC->GC, {kbs:.0f} KB/s")
|
||||||
|
|
||||||
print("\nFAILURES:\n " + "\n ".join(fails) if fails else
|
print("\nFAILURES:\n " + "\n ".join(fails) if fails else
|
||||||
"\nAll analyse-pcap self-tests passed.")
|
"\nAll analyse-pcap self-tests passed.")
|
||||||
sys.exit(1 if fails else 0)
|
sys.exit(1 if fails else 0)
|
||||||
|
|||||||
Reference in New Issue
Block a user