fsp-server: reply checksums must use fsplib's VERIFIER formula, not its writer

Swiss hung on "setting up device" while the server logged perfectly reasonable
GET_PRO replies.  fsplib was dropping every one and retrying -- visible in the
log as the same request repeating with the low 3 sequence bits counting up,
since fsp_transaction encodes the retry number there.

FSP has TWO checksum formulas and they differ by the packet length:

  fsp_pkt_write (what a client SENDS):
      checksum = sum(bytes, sum byte zeroed) + used
      sum_byte = checksum + (checksum >> 8)

  fsp_pkt_read (what a client ACCEPTS):
      mysum = sum(bytes) - sum_byte           <- no length term at all
      ok if ((mysum + (mysum >> 8)) & 0xff) == sum_byte

cube/patches/bba/tcpip.c matches the WRITE form (`sum = size` to start) and
never verifies replies, so the in-game path tolerated our packets; the menu's
fsplib did not.  Replies are now built the way fsp_pkt_read verifies.
Incoming requests are accepted under either formula, since a tolerated request
is far easier to diagnose than a rejected one.

⚠️ The self-test passed throughout the bug, and that is the part worth
remembering: it verified replies with the SERVER'S OWN checksum function, so
the two agreed with each other and disagreed with Swiss.  Identical in shape to
the RX descriptor format, where our writer, our readers and our tests were all
self-consistent and all wrong.  The test now transcribes fsp_pkt_read literally
as fsplib_verify() and checks every reply against that, and mutating the server
back to the with-length formula fails it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

scripts/fsp-server.py

@@ -54,8 +54,34 @@ HSIZE, FSP_SPACE = 12, 1024
 SERVER_KEY = 0x1234
 
 
-def checksum(pkt):
-    """pkt must already have byte 1 (the checksum) zeroed."""
+# ⚠️ FSP HAS TWO DIFFERENT CHECKSUM FORMULAS AND YOU MUST USE BOTH.
+#
+# fsplib's generator and its verifier disagree by the packet length:
+#
+#   fsp_pkt_write (what a CLIENT sends):
+#       checksum = sum(bytes, with the sum byte zeroed) + used
+#       sum_byte = checksum + (checksum >> 8)
+#
+#   fsp_pkt_read (what a CLIENT accepts):
+#       mysum = sum(bytes) - sum_byte          # i.e. no length term at all
+#       ok if ((mysum + (mysum >> 8)) & 0xff) == sum_byte
+#
+# cube/patches/bba/tcpip.c's fsp_checksum matches the WRITE form (it starts
+# `sum = size`), and the in-game client never verifies replies at all.
+#
+# So: replies must be built the way fsp_pkt_read VERIFIES -- without the
+# length -- or Swiss's menu client silently drops every one.  That is exactly
+# what happened: fsplib retried, made no progress, and Swiss hung on
+# "setting up device" while the server logged perfectly good-looking replies.
+def checksum_reply(pkt):
+    """Match fsp_pkt_read's verifier: no length term.  pkt has sum byte = 0."""
+    total = sum(pkt)
+    total += total >> 8
+    return total & 0xFF
+
+
+def checksum_request(pkt):
+    """Match fsp_pkt_write / tcpip.c: includes the length.  pkt has sum = 0."""
     total = len(pkt) + sum(pkt)
     total += total >> 8
     return total & 0xFF
@@ -63,7 +89,7 @@ def checksum(pkt):
 
 def pack(cmd, key, seq, pos, data=b"", extra=b""):
     body = struct.pack("!BBHHHI", cmd, 0, key, seq, len(data), pos) + data + extra
-    return body[:1] + bytes([checksum(body)]) + body[2:]
+    return body[:1] + bytes([checksum_reply(body)]) + body[2:]
 
 
 def unpack(pkt):
@@ -75,8 +101,12 @@ def unpack(pkt):
     data = pkt[HSIZE:HSIZE + ln]
     extra = pkt[HSIZE + ln:]
     zeroed = pkt[:1] + b"\0" + pkt[2:]
+    # Accept EITHER formula on the way in: fsplib and tcpip.c both send the
+    # with-length form, but being permissive costs nothing and a rejected
+    # request is much harder to diagnose than a tolerated one.
+    ok = csum in (checksum_request(zeroed), checksum_reply(zeroed))
     return dict(cmd=cmd, key=key, seq=seq, pos=pos, data=data, extra=extra,
-                csum_ok=(checksum(zeroed) == csum))
+                csum_ok=ok)
 
 
 def split_path(data, password):