Streaming measured 509 KB/s, BELOW the menu's 850 KB/s TCP dump, and the
arithmetic says exactly why.
I had this wrong: QUEUE_SIZE 2 is the number of queued GAME reads, not
concurrent network requests. fsp_read_queued() sends ONE fsp_get_file and
arms a 1 s alarm; fsp_pop_queue() issues the next only once the response
arrives. Streaming is therefore STRICTLY ONE REQUEST IN FLIGHT, so
throughput = bytes_per_reply / round_trip
The server clamped replies to FSP_SPACE (1024), giving 1024 B / ~2.01 ms =
509 KB/s -- the measured figure almost exactly. But the in-game client asks
for pmtu - 40 (1460 at a 1500 MTU) and accepts it; only fsplib has the 1024
limit, and fsplib never asks for more than that. So honouring the requested
size is safe for both clients, and at 1440 B (the 1500 MTU budget rounded
down to 32) the same round trip gives ~1.41x, about 700 KB/s.
Replies stay a multiple of 32 unless they are the file tail, because the
client computes data_size = MIN(requested, OSRoundDown32B(reply_len)) and a
non-aligned mid-file reply would be silently truncated.
Also instruments the round trip: with one request in flight the observed
interval IS the RTT, so the report now splits it into total and this server's
median service time and says SERVER-bound or console-bound. Without that,
"it is slow" has no owner. And the raw socket now binds ETH_P_IP rather than
ETH_P_ALL, so our own outgoing frames are no longer copied to userspace and
parsed in Python on the path we are trying to shorten.
T13 checks a 1460-byte request yields 1440 (32-aligned) while an fsplib-sized
1024 request still yields exactly 1024.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Swiss reported "failed to write file" then "Failed to setup the file (too
fragmented?)". The FSP device declares FEAT_READ|FEAT_WRITE|FEAT_BOOT_GCM|
FEAT_PATCHES, so booting a disc image from FSP is supported -- but Swiss
writes patch files to swiss/patches/ first, and deviceHandler_FSP_setupFile
returns failure when that write fails, which the UI reports as the fragment
message.
Implements CC_UP_LOAD (0x43), CC_INSTALL (0x44), CC_MAKE_DIR (0x49),
CC_DEL_FILE (0x45) and CC_DEL_DIR (0x46), all confined to --root. --writable
(aliased as --claim-owner) enables them and reports FSP_DIR_OWNER, so the two
can no longer be set inconsistently: previously --claim-owner told Swiss it
could write to a server with no write support, which is exactly the
combination that produced the error.
⚠️ CC_UP_LOAD CARRIES NO FILENAME. fsplib calls buildfilename() only on the
read branch, so the data field is RAW FILE BYTES at `pos` and CC_INSTALL names
the destination afterwards (temp-then-commit). Handling UP_LOAD after the
path/password parse rejects every chunk as a bad password, so it is handled
before it; the password is validated on INSTALL, which does carry the path.
`pos` must also be echoed, because fsp_transaction checks it for UP_LOAD.
Tests: MAKE_DIR creates the directory, two 1024-byte UP_LOADs plus INSTALL
commit byte-exact content, writes outside --root are refused, and GET_PRO
reports OWNER when writable. Mutation-tested: moving UP_LOAD back below the
password parse fails T11 with CC_ERR on every chunk.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Swiss hung on "setting up device" while the server logged perfectly reasonable
GET_PRO replies. fsplib was dropping every one and retrying -- visible in the
log as the same request repeating with the low 3 sequence bits counting up,
since fsp_transaction encodes the retry number there.
FSP has TWO checksum formulas and they differ by the packet length:
fsp_pkt_write (what a client SENDS):
checksum = sum(bytes, sum byte zeroed) + used
sum_byte = checksum + (checksum >> 8)
fsp_pkt_read (what a client ACCEPTS):
mysum = sum(bytes) - sum_byte <- no length term at all
ok if ((mysum + (mysum >> 8)) & 0xff) == sum_byte
cube/patches/bba/tcpip.c matches the WRITE form (`sum = size` to start) and
never verifies replies, so the in-game path tolerated our packets; the menu's
fsplib did not. Replies are now built the way fsp_pkt_read verifies.
Incoming requests are accepted under either formula, since a tolerated request
is far easier to diagnose than a rejected one.
⚠️ The self-test passed throughout the bug, and that is the part worth
remembering: it verified replies with the SERVER'S OWN checksum function, so
the two agreed with each other and disagreed with Swiss. Identical in shape to
the RX descriptor format, where our writer, our readers and our tests were all
self-consistent and all wrong. The test now transcribes fsp_pkt_read literally
as fsplib_verify() and checks every reply against that, and mutating the server
back to the with-length formula fails it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Swiss sent CC_GET_PRO 'swiss/patches' and the server replied "unsupported".
Reply shape, from fsplib's fsp_getpro:
* pos MUST equal FSP_PRO_BYTES (1), or fsplib fails with ENOMSG
* the flags byte is at buf[len] -- the first byte of the EXTRA region,
after the echoed name, NOT inside the data field
* a missing directory must be CC_ERR so fsplib reports ENOENT, because
Swiss probes "swiss/patches", then "swiss", then "" and relies on that
fallback chain
Those bits are not cosmetic. deviceHandler-FSP.c does:
if ((pro & (MKDIR|ADD|DEL|GET)) == (MKDIR|ADD|DEL) || (pro & OWNER)) {
features |= FEAT_PATCHES; emulable |= EMU_ETHERNET;
} else { ... clears both ... }
so a plain read-only answer makes Swiss DISABLE patch support on this device.
Default is therefore the honest GET|LIST (0x50) with patches disabled, and
--claim-owner reports OWNER to turn them back on -- flagged in the help as
unsafe for now, because this server has no UP_LOAD or MAKE_DIR, so Swiss
believing it can write here would make a patch write fail rather than degrade.
Try without it first.
T9 checks pos == 1 and that the flags land at buf[len] with GET|LIST and
without OWNER; T10 checks a missing directory gives CC_ERR so the fallback
chain works.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
sudo ./scripts/fsp-server.py --root /workspace --password alltheroms
Swiss streams games over FSP/UDP rather than TCP, so hosting an image needs an
FSP server. Serves a directory read-only: VERSION, STAT, GET_DIR, GET_FILE,
BYE, with CC_ERR for anything else. Reports the achieved read rate every two
seconds, which is the measurement this was really built for -- the 850 KB/s
figure came from the menu's lwIP/TCP path and says nothing about streaming.
The protocol was taken from Swiss's own two clients rather than from memory,
because they disagree about nothing but exercise different commands:
cube/swiss/source/devices/fsp/fsplib.c (menu: browse, stat, read)
cube/patches/bba/tcpip.c (in-game streaming reads)
Details that are easy to get wrong, all verified against that source:
* checksum: sum = total_size + every byte with the checksum byte zeroed,
then sum += sum >> 8, low 8 bits.
* password: the data field is "path\npassword\0" -- the '\n' REPLACES the
path's terminating NUL (fsplib buildfilename), it is not appended after it.
* GET_DIR: entries are time(4) size(4) type(1) name(NUL) padded to 4 bytes;
the client reads blocks until one is shorter than the first, so the whole
listing then an EMPTY reply is what terminates it. dirpos % 4 != 0 is
rejected outright by fsplib, hence the alignment padding.
* STAT: the same 9-byte time/size/type prefix; type 0 means ENOENT.
* GET_FILE: sequence AND position must be echoed -- the in-game client
discards any reply where either differs.
* ⚠️ a GET_FILE reply must be a multiple of 32 bytes unless it is the tail,
because the in-game client does
data_size = MIN(requested, OSRoundDown32B(reply_len))
so a non-32-aligned reply is silently rounded DOWN and the stream stalls.
test-fsp-server.py runs a real server on 127.0.0.1 and drives it with a client
that packs requests exactly as Swiss does: checksums verify both ways, STAT
returns the true size, GET_DIR parses with fsplib's own walk and the second
page is empty, GET_FILE returns byte-exact data at three offsets with seq and
pos echoed, a 1024-byte request comes back 32-aligned, a wrong password gives
CC_ERR, and ../../etc/passwd is refused.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Roflin