ci: Improve benchmarking tests with CodSpeed

[Copilot]

Summary

Replace pytest-benchmark with pytest-codspeed for benchmarking, extend test coverage to full process lifecycle and run-only benchmarks, and parameterize across connector types.

Changes

• pyproject.toml: Swap pytest-benchmark → pytest-codspeed>=4.4.0
• tests/benchmark/test_benchmarking.py:
  • test_benchmark_process_lifecycle — benchmarks full init → run → destroy using @pytest.mark.benchmark
  • test_benchmark_process_run — benchmarks only process.run() using BenchmarkFixture.pedantic with setup/teardown callbacks and uvloop.run, excluding init and destroy from timing
  • Both parameterized over AsyncioConnector/LocalProcess, ZMQConnector/LocalProcess, and RayConnector/RayProcess
• .github/workflows/benchmarks.yaml: Replace manual checkout-both-branches-and-compare workflow with CodSpeedHQ/action@v4. Adds push to main and workflow_dispatch triggers for baseline tracking and backtesting. Uses OIDC auth (id-token: write). Adds RAY_ENABLE_UV_RUN_RUNTIME_ENV and PLUGBOARD_IO_READ_TIMEOUT env vars for Ray-based benchmarks.
• uv.lock: Updated to reflect dependency change

[Copilot]

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• metadata.google.internal
  • Triggering command: /home/REDACTED/work/plugboard/plugboard/.venv/bin/python /home/REDACTED/work/plugboard/plugboard/.venv/bin/python /home/REDACTED/work/plugboard/plugboard/.venv/lib/python3.12/site-packages/ray/dashboard/dashboard.py --host=127.0.0.1 --port=8265 --port-retries=50 --temp-dir=/tmp/ray --log-dir=/tmp/ray/session_2026-03-25_20-28-43_333217_4102/logs --session-dir=/tmp/ray/session_2026-03-25_20-28-43_333217_4102 --logging-rotate-bytes=536870912 --logging-rotate-backup-count=5 --gcs-address=127.0.0.1:33279 --cluster-id-hex=9ec9fda9fcb85e65d5dc26651d1db3efb8d117964fb0bb2db1d9c416 --node-ip-address=127.0.0.1 --stdout-filepath=/tmp/ray/session_2026-03-25_20-28-43_333217_4102/logs/dashboard.out --stderr-filepath=/tmp/ray/session_2026-03-25_20-28-43_333217_4102/logs/dashboard.err (dns block)
  • Triggering command: /home/REDACTED/work/plugboard/plugboard/.venv/bin/python /home/REDACTED/work/plugboard/plugboard/.venv/bin/python /home/REDACTED/work/plugboard/plugboard/.venv/lib/python3.12/site-packages/ray/dashboard/dashboard.py --host=127.0.0.1 --port=8265 --port-retries=50 --temp-dir=/tmp/ray --log-dir=/tmp/ray/session_2026-03-25_20-33-01_962890_5585/logs --session-dir=/tmp/ray/session_2026-03-25_20-33-01_962890_5585 --logging-rotate-bytes=536870912 --logging-rotate-backup-count=5 --gcs-address=127.0.0.1:37307 --cluster-id-hex=6c37b0969d30736700cb2e0a3ff6b3b023662ffa335ba1d9fb65db86 --node-ip-address=127.0.0.1 --stdout-filepath=/tmp/ray/session_2026-03-25_20-33-01_962890_5585/logs/dashboard.out --stderr-filepath=/tmp/ray/session_2026-03-25_20-33-01_962890_5585/logs/dashboard.err cal/bin/git hub.com/.extrahe/home/REDACTED/.cache/uv/builds-v0/.tmpycA9Bm/bin/python (dns block)
  • Triggering command: /home/REDACTED/work/plugboard/plugboard/.venv/bin/python3 /home/REDACTED/work/plugboard/plugboard/.venv/bin/python3 /home/REDACTED/work/plugboard/plugboard/.venv/lib/python3.12/site-packages/ray/dashboard/dashboard.py --host=127.0.0.1 --port=8265 --port-retries=50 --temp-dir=/tmp/ray --log-dir=/tmp/ray/session_2026-03-25_20-34-45_613600_6660/logs --session-dir=/tmp/ray/session_2026-03-25_20-34-45_613600_6660 --logging-rotate-bytes=536870912 --logging-rotate-backup-count=5 --gcs-address=127.0.0.1:34471 --cluster-id-hex=e65bcc378675174b4494fed2ef08f5c8492840459a2b9e0ece8361a3 --node-ip-address=127.0.0.1 --stdout-filepath=/tmp/ray/session_2026-03-25_20-34-45_613600_6660/logs/dashboard.out --stderr-filepath=/tmp/ray/session_2026-03-25_20-34-45_613600_6660/logs/dashboard.err python http.https://gitbash (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[codspeed-hq]

Congrats! CodSpeed is installed 🎉

🆕 3 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

• test_benchmark_process_lifecycle[ray]: 9 s
• test_benchmark_process_lifecycle[zmq]: 3.9 s
• test_benchmark_process_lifecycle[asyncio]: 2.6 s

---

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[toby-coleman]

Wait for CodSpeedHQ/pytest-codspeed#113 before merging this.

[toby-coleman]

@copilot Reinstate the run-only tests - take a look at c663755 to see what to add back in. Update the version of pytest-codespeed to >=4.4.0

[Copilot]

@copilot Reinstate the run-only tests - take a look at c663755 to see what to add back in. Update the version of pytest-codespeed to >=4.4.0

Reinstated the run-only tests based on c663755 and updated pytest-codspeed to >=4.4.0 in 85362c6. Also fixed the _run function signature (removed unused process parameter) and added a teardown callback for proper cleanup in f585742.