Roflin/rebbarb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

scripts: add a read-only FSP server so Swiss can stream a disc image

Browse Source 
sudo ./scripts/fsp-server.py --root /workspace --password alltheroms

Swiss streams games over FSP/UDP rather than TCP, so hosting an image needs an
FSP server.  Serves a directory read-only: VERSION, STAT, GET_DIR, GET_FILE,
BYE, with CC_ERR for anything else.  Reports the achieved read rate every two
seconds, which is the measurement this was really built for -- the 850 KB/s
figure came from the menu's lwIP/TCP path and says nothing about streaming.

The protocol was taken from Swiss's own two clients rather than from memory,
because they disagree about nothing but exercise different commands:
  cube/swiss/source/devices/fsp/fsplib.c  (menu: browse, stat, read)
  cube/patches/bba/tcpip.c                (in-game streaming reads)

Details that are easy to get wrong, all verified against that source:
  * checksum: sum = total_size + every byte with the checksum byte zeroed,
    then sum += sum >> 8, low 8 bits.
  * password: the data field is "path\npassword\0" -- the '\n' REPLACES the
    path's terminating NUL (fsplib buildfilename), it is not appended after it.
  * GET_DIR: entries are time(4) size(4) type(1) name(NUL) padded to 4 bytes;
    the client reads blocks until one is shorter than the first, so the whole
    listing then an EMPTY reply is what terminates it.  dirpos % 4 != 0 is
    rejected outright by fsplib, hence the alignment padding.
  * STAT: the same 9-byte time/size/type prefix; type 0 means ENOENT.
  * GET_FILE: sequence AND position must be echoed -- the in-game client
    discards any reply where either differs.
  * ⚠️ a GET_FILE reply must be a multiple of 32 bytes unless it is the tail,
    because the in-game client does
        data_size = MIN(requested, OSRoundDown32B(reply_len))
    so a non-32-aligned reply is silently rounded DOWN and the stream stalls.

test-fsp-server.py runs a real server on 127.0.0.1 and drives it with a client
that packs requests exactly as Swiss does: checksums verify both ways, STAT
returns the true size, GET_DIR parses with fsplib's own walk and the second
page is empty, GET_FILE returns byte-exact data at three offsets with seq and
pos echoed, a 1024-byte request comes back 32-aligned, a wrong password gives
CC_ERR, and ../../etc/passwd is refused.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Dennis Brentjes
2026-10-10 16:06:08 +02:00
parent 2da721d7ed
commit fd95643593
2 changed files with 380 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/fsp-server.py
Executable
+248
View File
@@ -0,0 +1,248 @@
#!/usr/bin/env python3
"""A read-only FSP (File Service Protocol) server, for ROM streaming to Swiss.
sudo ./scripts/fsp-server.py --root /workspace --password alltheroms
Swiss streams games over FSP/UDP, not TCP -- see cube/patches/bba/tcpip.c.
This serves a directory read-only so Swiss can browse it, stat a disc image
and stream it while a game runs. It also reports the achieved read rate,
which is the number we actually want to know.
Protocol taken from Swiss's own two clients, not from memory:
* cube/swiss/source/devices/fsp/fsplib.c (the menu: browse, stat, read)
* cube/patches/bba/tcpip.c (in-game streaming reads)
Wire format (v2, all multi-byte fields BIG-endian):
cmd(1) checksum(1) key(2) sequence(2) len(2) pos(4) then data[len] extra[xlen]
checksum: sum = total_size; sum += every byte (with the checksum byte
zeroed); sum += sum >> 8; take the low 8 bits.
password: the data field is "path\\npassword\\0" -- the '\\n' REPLACES the
path's terminating NUL (fsplib buildfilename).
dir entry: time(4) size(4) type(1) name(NUL-terminated), 4-byte aligned;
type 0x00 END, 0x01 FILE, 0x02 DIR, 0x2A SKIP.
stat reply: the same 9-byte time/size/type prefix.
⚠️ GET_FILE replies must be a MULTIPLE OF 32 BYTES unless they are the last
piece of the request: the in-game client does
data_size = MIN(requested, OSRoundDown32B(reply_len))
so a non-32-aligned reply is silently truncated down and the transfer stalls.
"""
import argparse
import os
import socket
import struct
import sys
import time
CC_VERSION, CC_INFO = 0x10, 0x11
CC_ERR, CC_GET_DIR, CC_GET_FILE = 0x40, 0x41, 0x42
CC_BYE, CC_STAT = 0x4A, 0x4D
RDTYPE_END, RDTYPE_FILE, RDTYPE_DIR, RDTYPE_SKIP = 0x00, 0x01, 0x02, 0x2A
HSIZE, FSP_SPACE = 12, 1024
SERVER_KEY = 0x1234
def checksum(pkt):
"""pkt must already have byte 1 (the checksum) zeroed."""
total = len(pkt) + sum(pkt)
total += total >> 8
return total & 0xFF
def pack(cmd, key, seq, pos, data=b"", extra=b""):
body = struct.pack("!BBHHHI", cmd, 0, key, seq, len(data), pos) + data + extra
return body[:1] + bytes([checksum(body)]) + body[2:]
def unpack(pkt):
if len(pkt) < HSIZE:
return None
cmd, csum, key, seq, ln, pos = struct.unpack("!BBHHHI", pkt[:HSIZE])
if len(pkt) < HSIZE + ln:
return None
data = pkt[HSIZE:HSIZE + ln]
extra = pkt[HSIZE + ln:]
zeroed = pkt[:1] + b"\0" + pkt[2:]
return dict(cmd=cmd, key=key, seq=seq, pos=pos, data=data, extra=extra,
csum_ok=(checksum(zeroed) == csum))
def split_path(data, password):
"""Return (path, password_ok). fsplib sends "path\\npassword\\0"."""
s = data.split(b"\0")[0].decode("latin-1", "replace")
if "\n" in s:
path, _, pw = s.partition("\n")
else:
path, pw = s, ""
return path, (password is None or pw == password)
def safe_join(root, path):
p = os.path.normpath(os.path.join(root, path.lstrip("/")))
if not (p == root or p.startswith(root + os.sep)):
return None
return p
def dir_block(real):
"""One FSP directory block: entries 4-byte aligned, then an END marker."""
out = bytearray()
try:
names = sorted(os.listdir(real))
except OSError:
names = []
for nm in names:
full = os.path.join(real, nm)
try:
st = os.stat(full)
except OSError:
continue
isdir = os.path.isdir(full)
ent = struct.pack("!IIB", int(st.st_mtime) & 0xFFFFFFFF,
st.st_size & 0xFFFFFFFF,
RDTYPE_DIR if isdir else RDTYPE_FILE)
ent += nm.encode("latin-1", "replace") + b"\0"
while len(ent) % 4:
ent += b"\0"
# Keep the whole listing inside one block: the client treats
# "fewer than 9 bytes left in this block" as SKIP.
if len(out) + len(ent) + 9 > FSP_SPACE:
break
out += ent
out += struct.pack("!IIB", 0, 0, RDTYPE_END)
while len(out) % 4:
out += b"\0"
return bytes(out)
def main():
ap = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--root", default=".", help="directory to serve")
ap.add_argument("--password", default=None,
help="require this password (Swiss sends it with every request)")
ap.add_argument("--bind", default="0.0.0.0")
ap.add_argument("--port", type=int, default=21)
ap.add_argument("--quiet-reads", action="store_true",
help="only log the rate, not every read")
a = ap.parse_args()
root = os.path.realpath(a.root)
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
try:
s.bind((a.bind, a.port))
except PermissionError:
sys.exit(f"port {a.port} needs root; run with sudo")
print(f"fsp-server on {a.bind}:{a.port} root {root}")
print(f" password {'set' if a.password else 'NONE'}")
print(f" point Swiss's FSP device at this host and port\n")
served = 0
t_first = None
t_report = time.time()
reads = 0
fh_path, fh = None, None
while True:
pkt, peer = s.recvfrom(FSP_SPACE + HSIZE + 64)
r = unpack(pkt)
if r is None:
print(f" short packet from {peer[0]} ({len(pkt)} B)")
continue
if not r["csum_ok"]:
print(f" ⚠️ bad checksum from {peer[0]}, cmd {r['cmd']:#04x}")
cmd, seq, pos = r["cmd"], r["seq"], r["pos"]
if cmd == CC_VERSION:
s.sendto(pack(CC_VERSION, SERVER_KEY, seq, 0,
b"fsp-server (rebbarb)\0"), peer)
print(f"[{peer[0]}] VERSION")
continue
if cmd == CC_BYE:
s.sendto(pack(CC_BYE, SERVER_KEY, seq, 0), peer)
print(f"[{peer[0]}] BYE")
continue
path, pw_ok = split_path(r["data"], a.password)
real = safe_join(root, path)
if not pw_ok:
print(f"[{peer[0]}] {cmd:#04x} {path!r} -> WRONG PASSWORD")
s.sendto(pack(CC_ERR, SERVER_KEY, seq, pos, b"bad password\0"), peer)
continue
if real is None:
s.sendto(pack(CC_ERR, SERVER_KEY, seq, pos, b"bad path\0"), peer)
continue
if cmd == CC_STAT:
if os.path.exists(real):
st = os.stat(real)
body = struct.pack("!IIB", int(st.st_mtime) & 0xFFFFFFFF,
st.st_size & 0xFFFFFFFF,
RDTYPE_DIR if os.path.isdir(real)
else RDTYPE_FILE)
print(f"[{peer[0]}] STAT {path!r} -> {st.st_size} B")
else:
body = struct.pack("!IIB", 0, 0, 0) # type 0 == ENOENT
print(f"[{peer[0]}] STAT {path!r} -> not found")
s.sendto(pack(CC_STAT, SERVER_KEY, seq, pos, body), peer)
continue
if cmd == CC_GET_DIR:
if not os.path.isdir(real):
s.sendto(pack(CC_ERR, SERVER_KEY, seq, pos, b"not a dir\0"), peer)
continue
blk = dir_block(real)
# The client reads blocks until one comes back shorter than the
# first; so the whole listing, then an empty reply.
out = blk if pos == 0 else b""
s.sendto(pack(CC_GET_DIR, SERVER_KEY, seq, pos, out), peer)
print(f"[{peer[0]}] GET_DIR {path!r} pos={pos} -> {len(out)} B")
continue
if cmd == CC_GET_FILE:
want = struct.unpack("!H", r["extra"][:2])[0] if len(r["extra"]) >= 2 else 1024
want = min(want, FSP_SPACE)
if fh_path != real:
if fh:
fh.close()
try:
fh = open(real, "rb")
fh_path = real
except OSError as e:
s.sendto(pack(CC_ERR, SERVER_KEY, seq, pos,
f"{e}\0".encode()), peer)
continue
fh.seek(pos)
data = fh.read(want)
s.sendto(pack(CC_GET_FILE, SERVER_KEY, seq, pos, data), peer)
served += len(data)
reads += 1
now = time.time()
if t_first is None:
t_first = now
print(f"[{peer[0]}] GET_FILE {path!r} streaming started")
if not a.quiet_reads and reads <= 3:
print(f"[{peer[0]}] GET_FILE {path!r} pos={pos} "
f"want={want} -> {len(data)} B")
if now - t_report >= 2.0:
el = max(now - t_first, 1e-9)
print(f" streaming {served/el/1024:7.0f} KB/s "
f"{served/2**20:8.1f} MiB in {el:6.1f}s "
f"{reads/el:5.0f} reads/s last pos {pos:#x}")
t_report = now
continue
print(f"[{peer[0]}] unsupported command {cmd:#04x} {path!r}")
s.sendto(pack(CC_ERR, SERVER_KEY, seq, pos, b"unsupported\0"), peer)
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
print("\nstopped")
scripts/test-fsp-server.py
Executable
+132
View File
@@ -0,0 +1,132 @@
#!/usr/bin/env python3
"""Self-test for fsp-server.py, using a client that packs requests the way
Swiss's fsplib/tcpip.c do (path\\npassword\\0, 2-byte xlen, v2 checksum)."""
import os, socket, struct, subprocess, sys, tempfile, time
HSIZE = 12
CC_VERSION, CC_ERR, CC_GET_DIR, CC_GET_FILE, CC_STAT = 0x10, 0x40, 0x41, 0x42, 0x4D
PW = "alltheroms"
fails = []
def csum(pkt):
t = len(pkt) + sum(pkt)
t += t >> 8
return t & 0xFF
def req(cmd, seq, pos, path, pw=PW, extra=b""):
data = path.encode() + (b"\n" + pw.encode() if pw else b"") + b"\0"
body = struct.pack("!BBHHHI", cmd, 0, 0x1234, seq, len(data), pos) + data + extra
return body[:1] + bytes([csum(body)]) + body[2:]
def parse(pkt):
cmd, cs, key, seq, ln, pos = struct.unpack("!BBHHHI", pkt[:HSIZE])
zeroed = pkt[:1] + b"\0" + pkt[2:]
return dict(cmd=cmd, seq=seq, pos=pos, data=pkt[HSIZE:HSIZE + ln],
csum_ok=csum(zeroed) == cs)
tmp = tempfile.mkdtemp()
PAYLOAD = bytes((i * 7 + 3) & 0xFF for i in range(8192))
open(os.path.join(tmp, "dvd.iso"), "wb").write(PAYLOAD)
os.mkdir(os.path.join(tmp, "subdir"))
srv = subprocess.Popen(
[sys.executable, "-I", "scripts/fsp-server.py", "--root", tmp,
"--password", PW, "--bind", "127.0.0.1", "--port", "2121", "--quiet-reads"],
stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT)
time.sleep(1.0)
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.settimeout(3.0)
A = ("127.0.0.1", 2121)
def rt(pkt):
s.sendto(pkt, A)
return parse(s.recv(2048))
try:
# T1 checksum both ways
r = rt(req(CC_VERSION, 1, 0, ""))
if r["cmd"] != CC_VERSION or not r["csum_ok"]:
fails.append(f"T1 VERSION: cmd {r['cmd']:#04x} csum_ok {r['csum_ok']}")
else:
print("T1 VERSION answered, reply checksum verifies")
# T2 STAT returns the real size
r = rt(req(CC_STAT, 2, 0, "/dvd.iso"))
t, sz, ty = struct.unpack("!IIB", r["data"][:9])
if sz != len(PAYLOAD) or ty != 0x01:
fails.append(f"T2 STAT size {sz} type {ty}, want {len(PAYLOAD)} / 1")
else:
print(f"T2 STAT -> size {sz}, type FILE")
# T3 GET_DIR lists the file and the subdir, and parses as fsplib does
r = rt(req(CC_GET_DIR, 3, 0, "/"))
d, i, found = r["data"], 0, {}
while i + 9 <= len(d):
ty = d[i + 8]
if ty == 0x00:
break
sz = struct.unpack("!I", d[i + 4:i + 8])[0]
end = d.index(b"\0", i + 9)
found[d[i + 9:end].decode()] = (ty, sz)
i = end + 1
i += (-i) % 4
if found.get("dvd.iso") != (0x01, len(PAYLOAD)) or found.get("subdir", (0,))[0] != 0x02:
fails.append(f"T3 GET_DIR entries wrong: {found}")
else:
print(f"T3 GET_DIR -> {found}")
if (len(r['data'])) % 4:
fails.append("T3 dir block not 4-byte aligned (fsplib rejects dirpos%4)")
# T4 second GET_DIR page must be EMPTY so the client stops looping
r2 = rt(req(CC_GET_DIR, 4, len(r["data"]), "/"))
if len(r2["data"]) != 0:
fails.append(f"T4 second dir page returned {len(r2['data'])} B, want 0")
else:
print("T4 second GET_DIR page empty -> client terminates")
# T5 GET_FILE: right bytes, and sequence+position ECHOED (the in-game
# client discards a reply whose seq or pos does not match)
for pos, want in ((0, 1024), (4096, 1024), (8000, 1024)):
r = rt(req(CC_GET_FILE, 10 + pos, pos, "/dvd.iso",
extra=struct.pack("!H", want)))
exp = PAYLOAD[pos:pos + want]
if r["data"] != exp:
fails.append(f"T5 GET_FILE pos={pos}: {len(r['data'])} B, "
f"want {len(exp)}, match={r['data']==exp}")
elif r["seq"] != 10 + pos or r["pos"] != pos:
fails.append(f"T5 seq/pos not echoed: seq {r['seq']} pos {r['pos']}")
print("T5 GET_FILE at 0 / 4096 / EOF-straddle: bytes exact, seq+pos echoed")
# T6 a 32-aligned request must give a 32-aligned reply, or the in-game
# client's OSRoundDown32B silently truncates and the transfer stalls
r = rt(req(CC_GET_FILE, 99, 0, "/dvd.iso", extra=struct.pack("!H", 1024)))
if len(r["data"]) % 32:
fails.append(f"T6 reply {len(r['data'])} B is not a multiple of 32")
else:
print(f"T6 1024 B request -> {len(r['data'])} B, 32-aligned")
# T7 wrong password must be refused
r = rt(req(CC_STAT, 7, 0, "/dvd.iso", pw="wrong"))
if r["cmd"] != CC_ERR:
fails.append(f"T7 wrong password accepted (cmd {r['cmd']:#04x})")
else:
print("T7 wrong password -> CC_ERR")
# T8 path traversal must not escape the root
r = rt(req(CC_STAT, 8, 0, "/../../etc/passwd"))
t, sz, ty = struct.unpack("!IIB", (r["data"] + b"\0" * 9)[:9])
if r["cmd"] != CC_ERR and ty != 0:
fails.append(f"T8 traversal escaped the root: type {ty} size {sz}")
else:
print("T8 ../../etc/passwd refused")
finally:
srv.terminate()
print("\nFAILURES:\n " + "\n ".join(fails) if fails else
"\nAll fsp-server self-tests passed.")
sys.exit(1 if fails else 0)