Skip to navigation
Test your docsUI proceduresScreenshots

Detect visual regressions in screenshots

Turn a screenshot into a regression test by comparing each capture against a saved reference image. When the two drift apart by more than a threshold you set, Doc Detective flags it for review. New to capturing screenshots? Start with Capture screenshots. The comparison runs fine in a headless or CI context.

Before you begin

Make sure you have the following:

  • Doc Detective installed.
  • A page or site to capture. The examples below capture a local page.
  • A directory where you can create a .spec.json file.
  • A base screenshot step to build on. See Capture screenshots.
  • A committed reference image at path (the first run seeds it).

The steps in this guide work the same in every authoring style. Write them in a standalone .spec.json, inline in a doc source file, or let Doc Detective detect them from prose.

Detect visual regressions

A screenshot becomes a regression test when you compare it against a saved reference image.

Set maxVariation to a value from 0 to 1 to enable comparison. It’s the maximum acceptable fractional difference between the new capture and the reference at path. A difference beyond the threshold returns WARNING rather than FAIL, because visual drift is a signal to review rather than a hard break:

{
  "description": "Compare against the committed reference image.",
  "screenshot": {
    "path": "./reference/home.png",
    "maxVariation": 0.05,
    "overwrite": "aboveVariation"
  }
}

overwrite controls when Doc Detective replaces the reference image with the new capture:

  • "false": never overwrite.
  • "true": always overwrite.
  • "aboveVariation" (default): overwrite only when the difference exceeds maxVariation.

When a comparison runs, the step exposes $$variation (the fractional difference) and $$aspectRatioMatch (whether the two images share an aspect ratio). Capture either with the step-level variables object. For the full list of outputs, see Visual regression testing and the screenshot schema.

Reference images in cloud storage

path can be an http(s):// URL that ends in .png, so a team can keep reference images in cloud storage like S3. Signed-URL query strings are fine. Doc Detective downloads the remote image once per run and uses it as a read-only reference. It writes the new capture to ./doc-detective-runs/<run-timestamp>/ for inspection, and ignores overwrite because it never mutates remote storage:

{
  "description": "Compare against a reference image in S3.",
  "screenshot": {
    "path": "https://my-bucket.s3.amazonaws.com/reference-images/home.png",
    "maxVariation": 0.05
  }
}

Doc Detective refuses to fetch a reference image from a private, loopback, or link-local address, including its IPv4-mapped IPv6 form. This guards against server-side request forgery (SSRF). It also refuses the 169.254.169.254 cloud-metadata endpoint. A blocked URL fails the step with a message that starts with Couldn't fetch remote reference image. To compare against a reference on an internal address, such as a local development server, set DOC_DETECTIVE_ALLOW_LOCAL_URLS=true to opt out. That disables the guard for every reference URL in the run, so use it only with specifications you trust.

Verify it works

  • A comparison that stays within maxVariation reports PASS.
  • A comparison that exceeds maxVariation reports WARNING, so the run continues while flagging the drift for review.

Troubleshooting

Visual comparison never runs

Cause: maxVariation isn’t set, so the step only captures an image without comparing it.

Solution: set maxVariation to a value from 0 to 1 to enable comparison against the reference at path.

A remote reference URL fails with “Refusing to fetch private/loopback address”

Cause: the SSRF guard blocks reference URLs that resolve to a private, loopback, or link-local address.

Solution: host the reference image on a public address, or set DOC_DETECTIVE_ALLOW_LOCAL_URLS=true to compare against an internal address such as a local development server.

What’s next