Agentic Visual Testing: Running in CI

With baselines committed, CI runs one verb: check. It re-captures every entry in the manifest, diffs each against its baseline, classifies what changed, and fails the build on a regression.

If you have not captured baselines yet, start with Setting Up.

The check command


blazediff-agent check --judge host --json

It starts the dev server when config.devServer is set, runs every entry through Playwright, diffs each capture, and emits a CheckReport:


{
  "summaryPath": ".blazediff/summary.md",
  "totalEntries": 23,
  "passed": 22,
  "failed": 0,
  "pendingJudgments": 1,
  "results": [
    {
      "id": "agent",
      "url": "/agent",
      "status": "needs-judgment",
      "verdict": {
        "label": "ambiguous",
        "headline": "5 regions: 4 content-change, 1 addition @ left (0.13%, low)",
        "action": "investigate"
      }
    }
  ]
}

results[] lists non-passing entries only. Full per-entry detail lives in .blazediff/summary.md and .blazediff/judgments/<id>/request.json.

Check-only in CI. When CI=1 or there is no TTY, only check runs. onboard, capture, rewrite and reset are blocked. Authoring belongs on a developer’s machine, where a baseline change gets reviewed like any other diff.

GitHub Actions


- run: pnpm install
- run: npx blazediff-agent browsers install
- run: npx blazediff-agent --cwd apps/website check --json
  env:
    # Only needed if any entry uses a login harness. One pair per persona.
    # In CI, set these as secrets rather than committing .blazediff/.env.
    BLAZEDIFF_AUTH_DEFAULT_EMAIL: ${{ secrets.BLAZEDIFF_AUTH_DEFAULT_EMAIL }}
    BLAZEDIFF_AUTH_DEFAULT_PASSWORD: ${{ secrets.BLAZEDIFF_AUTH_DEFAULT_PASSWORD }}

Pass -C, --cwd <abs-path> to target one app inside a monorepo. Each app keeps its own .blazediff/ directory, so there is nothing central to keep in sync.

Exit codes

Code	Meaning
`0`	Every entry passed
`1`	At least one regression, intentional change, noise, or pending judgment
non-zero, with JSON	Infrastructure failure: missing manifest, no Chromium, and so on

A route that times out is logged once in the result array and skipped. It never blocks the run.

When a check fails

Exit code 1 usually means a diff needs a verdict rather than a fix. Locally, your coding agent reads the judgment request and decides; intentional changes are accepted with rewrite.

Judging and harnesses: verdicts, driving the page before a screenshot, and masking flaky regions
Agent-judged screenshots in GitHub Actions: a full workflow, including judging inside the job
Every command and flag: @blazediff/agent reference