Keep secrets out of logs, reports, and captures

Tests often need credentials: an API token in a request header, a password typed into a login form, a key in a .env file. Doc Detective uses the real values to run your steps. Everything it writes shows them as ***NAME*** instead: logs, reports, results files, and the GitHub Action’s issues.

spec       {"type": {"keys": "$PASSWORD", "selector": "#password"}}
report     "type": {"keys": "***PASSWORD***", "selector": "#password"}

Masking is on by default. Your steps, assertions, and routing conditions still compare real values, so a login test that checks what a user typed keeps working.

What Doc Detective masks

A value is masked when it came from one of these places:

• A $NAME that Doc Detective substituted, in a step, the config, or an OpenAPI example. That includes a variable a step set with variables.
• A variable loaded from your project’s .env file, a loadVariables file, or a credential file (apiKeyFile or apiTokenFile), even if no step uses it.
• A variable whose name contains TOKEN, SECRET, PASSWORD, API_KEY, PRIVATE_KEY, or CREDENTIAL, in any case, even if no step uses it.

Doc Detective never masks a value shorter than 4 characters, or the values true, false, and null. So a variable such as DEBUG=1 doesn’t erase every 1 in a report. Other variables in your environment, such as HOME or GITHUB_REF_NAME, aren’t masked unless a step uses them.

At the debug log level, the run lists the variables it masks, by name only:

Masking the values of 4 variables: API_TOKEN, BASE_URL, PASSWORD, SESSION.

When two variables hold the same value, the mask shows the first one’s name. That debug line and the run’s hint list both names.

Show a value that isn’t secret

Some values a run uses aren’t secret, such as a base URL. To show one, list its name in publicVariables. * matches any characters.

{
  "publicVariables": ["BASE_URL", "DOCS_*"]
}

masked     "command": "curl -H \"Authorization: Bearer ***API_TOKEN***\" ***BASE_URL***/me"
public     "command": "curl -H \"Authorization: Bearer ***API_TOKEN***\" https://api.example.com/me"

You can set publicVariables on the config, on a spec, or on a test, including an inline test in a doc. A name listed anywhere is public for the whole run. One variable has one value across a run, so once any test shows it, masking it elsewhere protects nothing. A spec or a test can’t make a name secret again.

{
  "specId": "login",
  "publicVariables": ["LOGIN_USER"],
  "tests": [
    {
      "steps": [
        { "goTo": "https://example.com/login" },
        { "type": { "keys": "$LOGIN_USER", "selector": "#user" } },
        { "type": { "keys": "$LOGIN_PASSWORD", "selector": "#password" } }
      ]
    }
  ]
}

On Windows, names match without regard to case, the way Windows environment variables do.

For a one-off run, set it through DOC_DETECTIVE_CONFIG. ["*"] turns masking off:

DOC_DETECTIVE_CONFIG='{"publicVariables":["BASE_URL","DOCS_*"]}' doc-detective
DOC_DETECTIVE_CONFIG='{"publicVariables":["*"]}' doc-detective

When a run masks a variable whose name doesn’t look secret, it suggests listing it:

💡 Hint: BASE_URL and APP_ENV show as ***NAME*** in this run's output. If they aren't secret, list them in publicVariables:
    "publicVariables": ["BASE_URL", "APP_ENV"]

A path or URL built from a masked value is masked in reports too. If a report can’t link a file whose path holds a variable’s value, make that variable public.

For every config option, see the config reference.

Forms Doc Detective catches

Doc Detective masks a value in each form that commonly reaches output:

Saved files

When runShell, runCode, httpRequest, or runBrowserScript saves its output with path, the file holds ***NAME*** in place of each secret value. A later run masks its new output the same way before it compares the two with maxVariation. So a saved file still matches when the secret behind it changes.

A file saved before version 5 holds the real value, so its first masked comparison can report a variation. Save it again with overwrite, or delete it, and the next run compares masked output.

saveCookie files are the exception. They hold the real cookie, because loadCookie reads them back. A cookie saved with variable holds the real value too. See What masking can’t catch.

Screenshots and recordings

Masking text can’t change pixels, so in a browser Doc Detective hides secrets on the page itself:

• Typed secrets. A type step, or the type option of a find step, can type a secret into a field. The field then shows dots for the rest of the page’s life. Later screenshots and recording frames show the dots too. The field still holds the real value, so the form works.
• Screenshots. Before each browser screenshot, Doc Detective blurs every element whose text or form value holds a secret. That covers screenshot steps and automatic screenshots. It takes the capture, then removes the blur. The page never receives your secrets. Doc Detective reads the page’s text and decides itself which elements to blur, so a page’s own scripts can’t collect them.

Because the blur changes the image, a reference screenshot taken before version 5 that showed a secret differs from new captures. Retake it.

These captures can’t hide a secret:

• page text that shows a secret while a recording runs, because only screenshots blur;
• text inside iframes or shadow DOM, text drawn on a canvas, and text from CSS pseudo-elements such as ::before;
• a secret split across sibling elements whose parent has no text of its own, such as <div><span>hun</span><b>ter2</b></div>;
• native app and device screenshots and recordings.

What masking can’t catch

• Transformed values. A value that a command hashes, splits, reverses, or encodes some other way isn’t masked.
• Variables no rule covers. runShell and runCode commands inherit the whole environment. A command can print a variable no step substituted. Doc Detective masks it only when a .env or loadVariables file set it, or when its name looks secret. A secret-looking name counts only for variables present when the run started, such as CI secrets, and for variables a step set with variables.
• Saved cookies. A saveCookie file holds real cookie values, because loadCookie reads them back. Keep these files out of version control. A cookie saved with variable isn’t masked until a later step substitutes it as $NAME, even when the variable’s name looks secret. Don’t print the variable or list the environment in runShell or runCode commands.
• Collisions. A secret that matches ordinary text, such as a short dictionary word, also masks that text.
• Steps you run from code. getRunner().runStep in the JavaScript API returns a step’s real result to your program, so mask anything you print from it yourself. runTests() returns masked results.