Skip to content

blog: how we stopped shipping performance regressions#438

Open
amavashev wants to merge 2 commits intomainfrom
blog/performance-regression-gate
Open

blog: how we stopped shipping performance regressions#438
amavashev wants to merge 2 commits intomainfrom
blog/performance-regression-gate

Conversation

@amavashev
Copy link
Copy Markdown
Contributor

Summary

  • New engineering blog post: How We Stopped Shipping Performance Regressions — the Cycles two-tier CI benchmark gate.
  • Covers: release-blocking at 25% vs a pinned baseline; nightly trend-warning at 30% vs a rolling 7-run median; 3-trial medians to absorb GH-runner noise; [benchmark-skip] escape hatch; benchmark-data branch pattern; parse-layer failing loudly.
  • Grounded in real source (parse-benchmarks.py, check-regression.py, release.yml, nightly-benchmark.yml, benchmarks/README.md) with verbatim code quotes.
  • Dated 2026-04-18.

Review cycle

  • Cycle 1: link verification, fact-check, SEO review in parallel. Broken forward-ref to unmerged TTL sweeper post stripped; p99 overclaim for Release/Event corrected; description trimmed for SEO.
  • Cycle 2+3: self-review for flow; rated 9/10 — cited, specific, no hype.
  • External reviewer: rated publishable. Five line-edits applied (tightened opener, softened empirical variance claim, softened operation-history claim, removed mild pattern-stealing overclaim, corrected "floor" to "bound").

Test plan

  • npm run build succeeds (no dead-link errors)
  • Post renders correctly in dev preview
  • Internal cross-links resolve (benchmark post, Lua/Redis post, property tests post)
  • GitHub source-code links resolve to main branch

Adds a post on the Cycles two-tier CI benchmark gate: release-blocking
at 25% against a pinned baseline, trend-warning at 30% against a
rolling 7-run median, with 3-trial medians to absorb GitHub-runner
noise. Covers the escape hatch, the data-branch pattern, and why the
parse layer fails loudly. Dated 2026-04-18.
- Soften "Docker build doesn't even run" to "release is blocked
  before the image is published" (substance unchanged, cleaner framing)
- Reframe 25% threshold line from "big enough / small enough" to
  "current operating threshold: above normal runner wobble"
- Trim the bottom-line landing to end on the "bound the release
  refuses to let drift" sentence
- Replace "boy who cried wolf" in the intro with "training the team
  to ignore it" — matches the rest of the tone
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant