Files
hermes e0ae7de150 fix(web): serialise ws_connections mutations under asyncio.Lock
Race between websocket_endpoint and broadcast() in backend/web.py caused
intermittent 'KeyError: <WebSocket object>' errors every ~7s in
brencher-brencher-backend logs (counted 354+ in the last hour), forcing
the browser UI to reconnect and dropping log/sse updates.

Repro window:
  websocket_endpoint:
    175: self.ws_connections[websocket] = {}
    await websocket.send_text(version_payload)
    await websocket.send_text(branches_payload)
    184: self.ws_connections[websocket]["branches"] = branches_payload  # KeyError

  broadcast() runs concurrently, sends to N sockets, awaits each, and
  pops disconnected ones. The await yields control back to
  websocket_endpoint which then tries to write a key that was already
  popped (or — conversely — broadcast iterates over a snapshot while a
  parallel endpoint adds a new key it later tries to mutate).

Fix:
- Add asyncio.Lock to WebApp.__init__ and acquire it for every mutation
  of ws_connections (broadcast iteration + cleanup, websocket_endpoint
  registration, WebSocketDisconnect cleanup).
- websocket_endpoint now registers the websocket with all three
  payloads (version/branches/environments) in a single dict assignment
  BEFORE the first send_text await, so there is no window where the key
  exists but the inner dict is empty.
- broadcast() snapshots ws_connections.keys() via list() before iterating,
  and runs send_text + dict write + cleanup under the same lock.

Tests (tests/test_web_ws_lock.py, 4 cases):
- test_websocket_endpoint_registers_atomically_under_lock: a single
  endpoint connection with a slow send_text; concurrent broadcasts do
  not cause KeyError or a missing-key dict.
- test_broadcast_and_concurrent_register_dont_crash: 20 endpoints + a
  broadcaster all run concurrently; ws_connections must drain cleanly
  and never raise.
- test_ws_lock_exists_and_is_asyncio_lock: guard against future
  accidental removal.
- test_broadcast_skips_already_sent_payload: the de-dup check still
  works after the lock change (regression guard).

Refs #42
2026-09-05 17:52:44 +00:00
..
2026-05-09 17:31:50 +02:00
2026-05-09 17:31:50 +02:00