Image Difference Analysis

Interpret mode takes a raw pixel diff and tells you what changed, where, and how much. Instead of one number, you get a list of regions, each with a bounding box, a change type, a percentage, and a severity.

It is deterministic. No model, no weights, no network call. It lives in @blazediff/interpret-native, a Node binding over the blazediff-interpret crate, and sits above whatever located the change: a pixel diff, an SSIM map, or boxes you already have.

This is what makes agent review tractable. A coding agent asked to compare two full-page screenshots invents differences; an agent handed three cropped regions that were measured deterministically is answering a much easier question. See Agentic visual testing.

How it works

Pixel diff produces a binary change mask.
A morphological close bridges small gaps in that mask.
Connected components isolate the mask into fragments.
Fragments are assembled into regions: a noise census counts speck-sized fragments to measure how noisy the pair is, nearby bounding boxes are merged (an inpainted object shattered into patches, the words of one recolored text run) — but only when the box that would enclose them is mostly touched below the diff threshold, so two distinct edits with untouched background between them stay separate instead of collapsing into one region — and a noise floor that scales with the census drops the rest, so a clean UI render keeps its smallest real regions while a recompressed photo sheds hundreds of ringing fragments.
Evidence is extracted per region:
- Dual-image gradients and luminance correlation. Edges in both images plus spatial correlation, to detect whether structure was preserved.
- Color delta distribution. Mean, max and standard deviation of YIQ distance, which separates a uniform recolor from a patchy texture change.
- Chroma-plane movement. How the color mass moved: hue-rotation cosine, saturation on both sides, and the smoothness of the chroma delta field. A recolor moves chroma coherently; a replacement scatters it. This is what separates the two on photographic edits, where regenerated texture makes luminance correlation useless.
- Background distance. How far changed pixels sit from the local unchanged pixels, in each image separately.
A six-label rule cascade classifies each region.
A post-pass finds region pairs where the content that left one location is the content that appeared at another — by directly correlating the two image crops, scored best-first — and relabels both halves as a Shift.

Pick a fixture pair. The analysis runs in your browser, in a Web Worker, on the same Rust classifier compiled to wasm — nothing here is precomputed.

Image 1

Image 2

Usage

Every entry point returns the same shape: a summary string, a regions[] array with position, changeType and percentage on each entry, and an overall severity.

interpret-native


import { interpret } from "@blazediff/interpret-native";
 
const result = await interpret("fixtures/3a.png", "fixtures/3b.png");
 
console.log(result.summary);
for (const region of result.regions) {
  console.log(`${region.position}: ${region.changeType} (${region.percentage.toFixed(2)}%)`);
}

Pass a third argument to keep the diff visualization, and source to locate the regions with a similarity map instead of a pixel diff:


await interpret("3a.png", "3b.png", "diff.png");
await interpret("3a.png", "3b.png", undefined, { source: "ms-ssim" });

When something else already knows where to look — DOM rectangles from a layout pass, say — interpretRegions skips the search and classifies those boxes:


import { interpretRegions } from "@blazediff/interpret-native";
 
await interpretRegions("3a.png", "3b.png", [{ x: 0, y: 0, width: 64, height: 64 }]);

Identical images

Image 1

Image 2

When nothing changed, regions is empty and summary says so.

Change types

Type	Meaning
`Addition`	Content appeared. Blends with the background in the before image, distinct in the after image.
`Deletion`	Content was removed. Distinct before, blends with the background after.
`Shift`	Content moved. Two regions whose before-crop and after-crop hold the same content, paired by patch correlation.
`ColorChange`	A recolor. Either luminance structure is preserved under a color shift (UI recolors), or the chroma moved coherently over regenerated texture (photographic recolors).
`ContentChange`	A structural change. Structure replaced and the chroma scattered rather than rotated.
`RenderingNoise`	Sub-pixel artifacts. Filtered out of the output.

Accuracy

Measured against datasets with hand-labeled change regions. Full breakdown in crates/blazediff-interpret-verify/BENCHMARKS.md .

Dataset	What it tests	Classifier-only macro F1	End-to-end macro F1
`addition_deletion`	Clean object insert and remove on photographs	1.000	0.958
`shift`	Sub-region translations with pixel-perfect ground truth	1.000	0.799
`inpaintcoco`	Inpaint edits that mix recolor and texture replacement	0.718	0.488
`html_color_pairs`	Recolors on rendered Tailwind UI screenshots	1.000	0.874

Read the two columns as different questions. Classifier-only assumes the regions were found correctly and asks whether the label is right. End-to-end runs the full detector first, so it also pays for missed regions and spurious small ones.

What the numbers say, plainly:

On the three clean-ground-truth datasets — object insert/remove, moved blocks, and UI recolors — the classifier labels every known region correctly. On shift the patch-correlation matcher pairs all 388 moved-block events with no false pairs.
On real inpainted photographs it lands the right label roughly five times in seven. This is the honest ceiling for per-region pixel statistics: a diffusion inpaint regenerates the texture whether the semantic edit was a recolor or a replacement, so the chroma-coherence evidence carries the whole distinction.
End to end, the census-scaled noise floor is what makes the photographic datasets tractable at all — on inpaintcoco it cuts spurious detections from tens of thousands to a few hundred. The remaining html_color_pairs misses are recolors whose pixel delta never crosses the diff threshold, which end-to-end detection cannot see by construction.

Put this to work in a test loop: Agentic visual testing
Full result type and options: @blazediff/interpret-native reference