Align snapshot regeneration workflow with VRT runtime#1358
Align snapshot regeneration workflow with VRT runtime#1358lukasoppermann merged 2 commits intomainfrom
Conversation
Agent-Logs-Url: https://github.com/primer/primitives/sessions/ad2259b8-a1f0-4312-9522-cab043f6e1af Co-authored-by: lukasoppermann <813754+lukasoppermann@users.noreply.github.com>
Agent-Logs-Url: https://github.com/primer/primitives/sessions/ad2259b8-a1f0-4312-9522-cab043f6e1af Co-authored-by: lukasoppermann <813754+lukasoppermann@users.noreply.github.com>
|
There was a problem hiding this comment.
Pull request overview
Aligns the snapshot regeneration workflow with the Visual Regression Testing (VRT) runtime to prevent baseline drift and reduce false diffs by running snapshots under the same environment and sharding model.
Changes:
- Updates snapshot regeneration to use 5 shards to match VRT.
- Mirrors VRT install/build steps (
npm ci && npm run storybook:install, thennpm run build). - Runs Playwright snapshot updates inside the same Playwright Docker image and uses the same host-bridge
STORYBOOK_URL.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/update_visual_snapshots.yml |
Brings snapshot regeneration sharding, dependency setup, build, and Playwright runtime/networking in line with the VRT workflow. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (1)
.github/workflows/update_visual_snapshots.yml:33
npm run storybook:installalready runsnpm ciinsidedocs/storybook(with--legacy-peer-deps). The additionalnpm ci --legacy-peer-depsin theBuild storybookstep is redundant, adds time, and can reintroduce drift vsvisual_regression_test.yml(which just runsnpm run build-storybook). Consider removing the extranpm cihere and relying onstorybook:installfor dependency setup.
- name: Build storybook
working-directory: docs/storybook
run: |
npm ci --legacy-peer-deps
npm run build-storybook
- Files reviewed: 1/1 changed files
- Comments generated: 1
| @@ -14,7 +14,7 @@ jobs: | |||
| runs-on: ubuntu-latest | |||
| strategy: | |||
There was a problem hiding this comment.
The snapshot update job's matrix strategy doesn't set fail-fast: false. With sharding enabled, the default fail-fast behavior can cancel remaining shards on the first failure, which makes snapshot regeneration less reliable and can leave the PR with only a partial set of updated snapshots. Consider matching visual_regression_test.yml by explicitly setting fail-fast: false under strategy:.
| strategy: | |
| strategy: | |
| fail-fast: false |
Summary
VRT failures were caused by environment drift: the snapshot update workflow generated baselines differently than the VRT verification workflow.
This change makes snapshot regeneration run with the same sharding, dependency setup, build scope, and Playwright runtime as VRT.
List of notable changes:
.github/workflows/update_visual_snapshots.ymlto use 5 shards ([1,2,3,4,5]) to matchvisual_regression_test.ymlnpm ci && npm run storybook:installnpm run buildmcr.microsoft.com/playwright:v1.56.1-jammywith host-bridgeSTORYBOOK_URL, matching VRT networking model172.17.0.1What should reviewers focus on?
.github/workflows/update_visual_snapshots.yml.github/workflows/visual_regression_test.ymlSteps to test:
update snapshotslabel.Snapshotsworkflow runs and commits regenerated snapshots.Visual Regression Testingand verify prior false diffs are resolved.Supporting resources (related issues, external links, etc):
Contributor checklist:
Reviewer checklist: