> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.doc-detective.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.doc-detective.com/_mcp/server. # Reporters and artifacts > Choose output formats, find the run folder, and gate CI on the results of a run. Every Doc Detective run produces output. That means a summary in your terminal, machine-readable results, and, when your tests capture them, screenshots and recordings. This guide covers choosing which reporters run, where each one writes its output, and how to fail a CI build. For the full list of reporter and output options, see the [`config`](/reference/schemas/config) reference. ## Before you begin * [Install Doc Detective](/docs/get-started/installation) and have a spec you can run. * Know your output directory. Reporters that write files use the `output` config option (or the `--output` flag); it defaults to the current working directory. ## Step 1: Choose your reporters Doc Detective ships six reporters: `terminal`, `json`, `html`, `runFolder`, `junit`, and `markdown`. By default, it runs `terminal`, `json`, and `runFolder`. To choose a different combination, set the `reporters` option. | Reporter | Output | | ----------- | ------------------------------------------------------------------------------------------------------------------ | | `terminal` | A colored pass/fail/warning/skipped summary printed to stdout. Writes no files. | | `json` | A flat JSON results file with the full step-by-step outcome. | | `html` | A self-contained HTML report you can open in a browser. | | `runFolder` | A per-run folder containing the results (JSON and HTML) alongside the screenshots and recordings the run captured. | | `junit` | A `junit.xml` report that CI platforms render in their test summary views. | | `markdown` | A `doc-detective-summary.md` run summary for a CI job summary or a merge request comment. | The `junit` and `markdown` reporters write a fixed filename in the output directory and overwrite it on each run, so a CI artifact path stays stable. If `output` names a file, they write beside that file. On GitLab, point the test summary at the `junit.xml` file: **`.gitlab-ci.yml`** ```yaml title=".gitlab-ci.yml" doc-tests: script: - doc-detective --input ./docs --reporters terminal junit --output ./reports artifacts: reports: junit: reports/junit.xml ``` On GitHub Actions, append the Markdown summary to the job summary: **`.github/workflows/docs.yml`** ```yaml title=".github/workflows/docs.yml" - run: doc-detective --input ./docs --reporters terminal markdown --output ./reports - if: always() run: cat reports/doc-detective-summary.md >> "$GITHUB_STEP_SUMMARY" ``` Select reporters on the command line: ```bash doc-detective --input api.spec.json --reporters terminal json html ``` Or set them in your config file so every run uses the same combination: **`.doc-detective.json`** ```json title=".doc-detective.json" { "reporters": ["terminal", "json", "html"] } ``` ## Step 2: Read the terminal summary The `terminal` reporter prints a summary grouped by specs, tests, contexts, and steps, with counts for passed, failed, warnings, and skipped. When everything passes and at least one test ran, it prints a success banner. When no tests run, whether from an empty input or an active filter, it prints a warning instead. An empty run therefore never looks like a green run. ## Step 3: Find the JSON results The `json` reporter writes the full results to a file: * If `output` ends in `.json`, Doc Detective writes to that path, adding a numeric suffix (`-0.json`, `-1.json`, …) to avoid overwriting an existing file. * Otherwise it writes `testResults-.json` in the output directory. The results file carries a `summary` object with the pass, fail, warning, and skipped counts. The per-spec, per-test, per-context, and per-step detail follows it. For an example of the results structure, see [Create your first test](/docs/get-started/create-your-first-test#outcome). To act on results programmatically, parse this file. See [Fail CI when tests fail](#step-6-fail-ci-when-tests-fail). ### Find slow specs with `durationMs` Every node in the results carries a `durationMs` field in whole milliseconds. That covers the run, each spec, each test, each context, and each step. Doc Detective populates it. You never set it yourself. ```json { "durationMs": 61840, "specs": [ { "specId": "api.spec.json", "durationMs": 48200, "tests": [{ "testId": "create-a-widget", "durationMs": 48200 }] } ] } ``` The meaning shifts slightly by level, and the difference matters when you run tests concurrently: | Level | `durationMs` is | | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Run | Elapsed time for the execution phase: the warm phase, every spec, and cleanup. It excludes test detection, resolution, and any just-in-time dependency install, so a cold runner spends longer than this value reports. | | Spec | The sum of its tests' durations. | | Test | The sum of its contexts' durations. | | Context | Elapsed time for the context, covering setup, every step, and cleanup. | | Step | Elapsed time for the step. | Spec and test durations are **sums of work**, not elapsed spans. Contexts from every test run through the same set of concurrent runners. An elapsed span for a spec would therefore count time while unrelated specs were running. Summing keeps the number stable no matter what you set [`concurrentRunners`](/reference/schemas/config) to. That's what you want when comparing specs to each other. The consequence: when `concurrentRunners` is greater than 1, the run's `durationMs` is *less* than the sum of its specs' durations, because specs overlapped. That's expected. Read the run value as elapsed wall-clock time, and the spec values as where the work went. To find the specs slowing your CI down, sort the specs by `durationMs`: ```bash jq '.specs | sort_by(-.durationMs) | .[:5] | .[] | {specId, durationMs}' testResults-*.json ``` Anything that runs more than once reports the **final attempt** in `durationMs`. That's the attempt the reported result describes. What that costs you depends on which kind of retry happened: * A step re-run by [`onFail` retry routing](/reference/schemas/routing) reports only its last attempt. The enclosing context's duration still covers every attempt and the waits between them, so the time isn't lost from the totals. It isn't separable, though. The difference between a context and the sum of its steps also includes ordinary setup and cleanup. Read it as an upper limit on what retrying cost, not a measurement of it. Use `attempts` to tell whether a step retried at all. * A context re-run by the [`retries`](/reference/schemas/config) policy reports only its last attempt. No context, test, or spec duration absorbs the earlier ones. The run-level `durationMs` is wall-clock, so it does include them. The `retries` count is your signal that it happened. A step that never ran reports `0`. That covers a step that was skipped, guarded off, or downstream of a failure. ### Read the warm-phase timings The results also carry a `warm` object that records what the run provisioned before executing tests. At the start of every run, Doc Detective front-loads setup the tests would otherwise pay one at a time. It runs those tasks concurrently. That setup covers browser and driver installs, simulator and emulator boots, and related downloads. Each entry lists the task, its outcome, and how long it took: ```json "warm": { "durationMs": 4211, "tasks": [ { "name": "browser-install:chrome", "kind": "browser-install", "outcome": "skipped", "durationMs": 2, "note": "'chrome' is already available" }, { "name": "session-probe", "kind": "session-probe", "outcome": "warmed", "durationMs": 4205, "note": "1 combination ok" } ] } ``` Outcomes are `warmed` (Doc Detective performed the work), `skipped` (nothing to do, or the environment handles it later), and `failed`. The warm phase is best-effort. A `failed` task logs a warning and never fails the run. The affected test context retries or skips with its normal behavior and reason. Use the timings to spot provisioning costs in slow CI runs. A long `warmed` task on every run usually means the [runtime cache](/docs/ci/docker-and-headless#step-3-warm-runtime-assets) isn't persisted between runs. ## Step 4: Find the HTML report The `html` reporter writes a self-contained HTML report you can open in a browser: * If `output` ends in `.html` or `.htm`, Doc Detective writes to that path, adding a numeric suffix (`-0.html`, `-1.html`, …) to avoid overwriting an existing file. * Otherwise it treats `output` as a directory and writes `testResults-.html` there (or `coverageResults-.html` for a coverage run). The report embeds the same results the `json` reporter produces. A single file is therefore enough to share a run with someone who doesn't have Doc Detective installed. ## Step 5: Collect the run folder and artifacts The `runFolder` reporter archives each run into its own folder so you can attach it to a CI job or diff one run against another: ``` /.doc-detective/runs// ├── testResults.json ├── testResults.html └── specs// └── tests// └── contexts// ├── screenshots/--.png └── recordings/.mp4 ``` The file names inside the folder are constant, because the run ID lives in the folder name. Two run folders therefore compare cleanly. Auto-captured screenshots and recordings land beside the results in the same folder. The whole folder becomes a single artifact to upload from CI for reviewers to inspect. ## Step 6: Fail CI when tests fail This is the most common mistake when wiring Doc Detective into a pipeline, so read it carefully. > **Warning** > > By default, the standalone `doc-detective` CLI does **not** exit with a non-zero code when tests fail. It exits non-zero only when it crashes or your config is invalid. A pipeline that relies on the CLI's exit code therefore reports success even when documentation tests failed. Opt in with `--exit-on-fail` to change that. To gate your build, choose one of these approaches: * **Set `--exit-on-fail` (least setup, works on any CI).** Pass the flag, or set `exitOnFail: true` in your config. The CLI then exits with code `1` whenever a spec fails, so the job fails like a broken unit test. `WARNING` results are non-fatal and never trigger a non-zero exit. It works the same everywhere, including GitLab CI, Jenkins, CircleCI, and a bare shell. ```bash doc-detective --input ./docs --exit-on-fail ``` * **Use the GitHub Action.** Set its `exit_on_fail` input to `true` so the job fails when tests fail. See the [GitHub Action guide](/docs/ci/github-action). * **Parse the JSON results.** Read the `summary` object from the `json` reporter's output and fail the step yourself when any `fail` count is greater than zero. Useful when you want custom logic beyond a pass/fail gate. ```bash doc-detective --input ./docs --reporters json --output results.json && \ node -e "process.exit(require('./results.json').summary.specs.fail > 0 ? 1 : 0)" ``` The `&&` runs the check only if Doc Detective itself exited cleanly. A crash or a broken config still fails the step, and the results parse never hides it. ## Verify it works After a run, confirm that: * The terminal shows the summary you expect. * The `json` reporter wrote a results file at the path you configured. * If you enabled `runFolder`, a `.doc-detective/runs//` folder exists with `testResults.json` and any captured screenshots or recordings. ## What's next * [Run Doc Detective in CI with the GitHub Action](/docs/ci/github-action). * [Run in Docker and headless](/docs/ci/docker-and-headless) for containerized pipelines. * [Troubleshoot a failing or flaky test](/docs/troubleshoot/overview) when the results aren't green.