Syntax error triggers crash in parallel mode

[JukkaL]

Type checking a trivial file t.py with a syntax error seems to trigger a crash if using --num-workers=2:

1 2

Here's what happens:

$ mypy --num-workers=2 --show-traceback t.py
t.py:1: error: Simple statements must be separated by newlines or semicolons  [syntax]
error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 2.0.0+dev.dada3385e78ca558d49e2f6d60f88643847e19c5
error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 2.0.0+dev.dada3385e78ca558d49e2f6d60f88643847e19c5
Traceback (most recent call last):
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/jukka/src/mypy/mypy/build_worker/__main__.py", line 6, in <module>
    console_entry()
  File "/Users/jukka/src/mypy/mypy/build_worker/worker.py", line 273, in console_entry
    main(sys.argv[1:])
  File "/Users/jukka/src/mypy/mypy/build_worker/worker.py", line 105, in main
    serve(server, ctx)
  File "/Users/jukka/src/mypy/mypy/build_worker/worker.py", line 133, in serve
    graph_data = GraphMessage.read(receive(server), manager)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jukka/src/mypy/mypy/build.py", line 5006, in read
    assert read_tag(buf) == GRAPH_MESSAGE
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError:
note: use --pdb to drop into pdb
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/jukka/src/mypy/mypy/build_worker/__main__.py", line 6, in <module>
    console_entry()
  File "/Users/jukka/src/mypy/mypy/build_worker/worker.py", line 273, in console_entry
    main(sys.argv[1:])
  File "/Users/jukka/src/mypy/mypy/build_worker/worker.py", line 105, in main
    serve(server, ctx)
  File "/Users/jukka/src/mypy/mypy/build_worker/worker.py", line 133, in serve
    graph_data = GraphMessage.read(receive(server), manager)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jukka/src/mypy/mypy/build.py", line 5006, in read
    assert read_tag(buf) == GRAPH_MESSAGE
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError:
note: use --pdb to drop into pdb
Found 1 error in 1 file (errors prevented further checking)

cc @ilevkivskyi

[ilevkivskyi]

Yeah, this one I discovered when I tried to reproduce the crash you reported in #21119. The fix is simple and it is on my TODO list.

However, whatever I do I cannot reproduce the crash you see on the other receive()call, I mean the one at

    assert read_tag(buf) == ACK_MESSAGE

If that one is specific to the "splitting" PR, could you please post some kind of repro there? (I mean a specific syntax error that would trigger it, plus maybe some relevant import structure.)

[JukkaL]

I'm still investigating the other crash. Here's what Claude Code has suggested (I haven't validated if this is correct yet):

❯ This is the traceback:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "mypy/build_worker/worker.py", line 105, in main
  File "mypy/build_worker/worker.py", line 177, in serve
  File "mypy/build.py", line 4788, in read
AssertionError:

⏺ The crash is at line 177 in the worker: AckMessage.read(receive(server)). The coordinator sent
something other than an AckMessage. Let me check.

Searched for 1 pattern (ctrl+o to expand)

⏺ The bug is clear. The crash is at worker.py:177 — AckMessage.read(receive(server)) — because
the worker received a shutdown SccRequestMessage instead of an AckMessage.

The scenario:

1. Worker A sends interface result successfully, waits for ack (line 177)
2. Worker B (or another SCC) encounters a blocking parse error → sends blocker to coordinator
3. Coordinator receives Worker B's blocker, raises CompileError
4. Coordinator's finally block sends SccRequestMessage(scc_id=None) (shutdown) to ALL workers
5. Worker A reads the shutdown message expecting an AckMessage → assert read_tag(buf) ==
   ACK_MESSAGE fails

The coordinator in wait_for_done_workers may process Worker B's blocker before sending the ack
to Worker A (depends on ready_to_read ordering), or the blocker may arrive from a different
wait_for_done iteration while Worker A is still waiting.

The fix: the worker should handle receiving a non-ack (shutdown) message at line 177
gracefully.

[ilevkivskyi]

Yes, this is ~correct, and this is what I was trying to reproduce, but so far I didn't find a deterministic way to trigger this. More context: this requires a blocker in semanal.py that happens exactly between two phases in another SCC, IoW this should be a rare condition, but it looks like there is some import structure and/or specific syntax error that makes it likely.

[JukkaL]

I will try to find a small reproducer, but it can be tricky. It happens pretty consistently in our big internal codebase though, so at least it will be easy to validate a potential fix.

[ilevkivskyi]

@JukkaL If you are fine with fix without test I actually already have one. I can combine it with the fix for this crash (because the fix is ~same).

[JukkaL]

Ok that seems fine.