> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.doc-detective.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.doc-detective.com/_mcp/server. # checkLink > Check if a URL returns an acceptable status code, primarily via a GET request with a HEAD fallback for some 429 or 403 responses. The `checkLink` action checks if a URL returns an acceptable status code. It primarily issues a browser-mimicking `GET` request. If the response is still `429` or `403` after retries, it retries the URL once with a `HEAD` request as a fallback. This action is useful for verifying that a hyperlink or image URL is valid. You can specify the target URL directly as a string, or use an object for more options: * `url`: (Required in object format) The URL to check. Can be a full URL or a path. If a path is provided, an `origin` must be specified either in the step or in the configuration file. * `origin`: (Optional) Protocol and domain prepended to `url` when `url` is a path. If omitted and `url` is a path, the global `origin` from the configuration file is used. * `statusCodes`: (Optional) A single integer or an array of integers representing acceptable HTTP status codes, including non-2xx codes like `429` (rate limited), `403` (forbidden), or `500` (server error). If omitted, defaults to `[200, 301, 302, 307, 308]`. * `headers`: (Optional) Additional HTTP headers to include in the request. Merged on top of Doc Detective's default browser-mimicking headers. Accepts either a key/value object or a newline-separated string. Useful for sites that allowlist specific headers or bypass tokens. Defaults to `{}`. * `params`: (Optional) Query parameters to append to the resolved URL. Merged with config-level `originParams`. Values support environment variable substitution with `$VAR` syntax. Defaults to `{}`. **Outputs:** the action exposes `outputs.statusCode`, the HTTP status code returned by the request. To capture it for later steps, use the step-level `variables` object with the expression `$$statusCode`. > For comprehensive options, see the [`checkLink`](/reference/schemas/checklink) reference. ## Behavior Each `checkLink` step: 1. Issues a `GET` request with browser-mimicking default headers (`User-Agent`, `Accept`, `Accept-Language`, `Accept-Encoding`, `Upgrade-Insecure-Requests`, and the `Sec-Fetch-*` and `Sec-Ch-Ua*` Client Hints set) to reduce false 429 or 403 responses from bot-protection layers. 2. Retries up to three times (four attempts total) with exponential backoff (1 second, then 2 seconds, then 4 seconds) on `429` or `5xx` responses. If the server sends a `Retry-After` header, Doc Detective waits that duration (capped at 10 seconds) instead. 3. If the final response is still `429` or `403`, retries the URL once with a `HEAD` request. Some web application firewalls (WAFs) rate-limit HTML `GET` requests but allow `HEAD` requests. If the `HEAD` returns an accepted status code, the step passes. 4. Compares the final status code against `statusCodes` and reports PASS or FAIL. ## Examples Here are a few ways you might use the `checkLink` action: ### Check if a link is valid (string shorthand) ```json { "tests": [ { "steps": [ { "description": "Check if Google is up.", "checkLink": "https://www.google.com" } ] } ] } ``` ### Check if a link is valid (object format) ```json { "tests": [ { "steps": [ { "description": "Check if Google is up.", "checkLink": { "url": "https://www.google.com" } } ] } ] } ``` ### Check for a specific status code response ```json { "tests": [ { "steps": [ { "description": "Check if Google is up, expecting only 200.", "checkLink": { "url": "https://www.google.com", "statusCodes": 200 } } ] } ] } ``` ### Check for one of a set of status code responses ```json { "tests": [ { "steps": [ { "description": "Check if Google is up with extra status codes.", "checkLink": { "url": "https://www.google.com", "statusCodes": [200, 201, 202] } } ] } ] } ``` ### Accept non-2xx status codes Include non-2xx status codes in `statusCodes` to accept responses that would otherwise fail. This is useful for testing endpoints that return rate limiting (`429`), authentication errors (`403`), or other expected non-success responses. Doc Detective's retry-with-backoff and HEAD-fallback logic, described above in [Behavior](#behavior), only runs while the response code isn't yet accepted. As soon as a response matches `statusCodes`, Doc Detective stops immediately and passes. That includes a match on the very first `GET`, as in the example below. It doesn't retry or fall back to `HEAD` merely because the code happens to be `429` or `403`. ```json { "tests": [ { "steps": [ { "description": "Check API endpoint, accepting rate limit responses.", "checkLink": { "url": "https://api.example.com/endpoint", "statusCodes": [200, 429] } } ] } ] } ``` ### Check a link with a separate origin and URL path ```json { "tests": [ { "steps": [ { "description": "Check if Google search path is valid using a specific origin.", "checkLink": { "url": "/search", "origin": "https://www.google.com" } } ] } ] } ``` ### Pass custom headers (allowlist or bypass) Use the `headers` field to send additional HTTP headers on top of Doc Detective's browser-mimicking defaults. This is useful for sites that allowlist a shared secret or require a bypass token. ```json { "tests": [ { "steps": [ { "description": "Check a protected portal with a shared-secret header.", "checkLink": { "url": "https://portal.example.com/docs", "headers": { "X-Doc-Detective-Check": "shared-secret" } } } ] } ] } ``` `headers` also accepts a newline-separated string, which is convenient for multi-header strings loaded from environment variables: ```json { "checkLink": { "url": "https://api.example.com/endpoint", "headers": "X-Api-Key: abc123\nAuthorization: Bearer token" } } ``` ### Check a relative link using global origin Assuming a global `origin` of `https://www.google.com` is set in the configuration: ```json { "tests": [ { "steps": [ { "description": "Check if Google search path is valid using global origin.", "checkLink": { "url": "/search" } } ] } ] } ``` ## Query parameters You can append query parameters to a URL using the `params` option. This is useful for authentication tokens or tracking parameters. * **Step-level `params`:** An object specifying query parameters to append to the URL. Applies to all URLs, both relative and absolute. Values support environment variable substitution with `$VAR` syntax. * **Config-level `originParams`:** Default query parameters in your configuration file that apply to relative URLs resolved against `origin`. Step-level `params` merge on top, with step keys taking precedence on collision. When your URL contains a fragment such as `/page#section`, query parameters are inserted before the fragment, giving `/page?token=value#section`. > **Warning** > > Values in `params` and `originParams` are embedded directly in request URLs and appear in test results, logs, and reports. Avoid storing long-lived secrets in these fields. ### Check a link with query parameters ```json { "tests": [ { "steps": [ { "description": "Check API health with testing token.", "checkLink": { "url": "/api/health", "origin": "https://my-app.com", "params": { "__clerk_testing_token": "$CLERK_TESTING_TOKEN" } } } ] } ] } ``` ## Troubleshooting * [`checkLink` fails with 429 or 403 even though the URL works in a browser](#checklink-fails-with-429-or-403-even-though-the-url-works-in-a-browser) * [`checkLink` fails due to unrecognized certificates](#checklink-fails-due-to-unrecognized-certificates) ### `checkLink` fails with 429 or 403 even though the URL works in a browser Sites behind bot protection or a web application firewall (WAF) can return `403` or `429` to non-browser traffic even when the URL works in your browser. Examples include Cloudflare, Akamai, Amazon Web Services (AWS) WAF, Imperva, PerimeterX, and Vercel deployment protection. Doc Detective already sends a browser-like header set and retries transient failures. Some WAFs still fingerprint deeper signals that header spoofing can't influence, such as Transport Layer Security (TLS) client fingerprints or source IP reputation. If you hit this, you have three escape hatches: * **Widen `statusCodes`** to accept the WAF response code, for example `"statusCodes": [200, 301, 302, 307, 308, 403, 429]`. This is pragmatic but loses signal, because the check only confirms that the URL exists. * **Pass allowlist or bypass headers** through the `headers` field. Common options when you control the site: | Provider | Headers or cookies | | ------------------------------------------- | ------------------------------------------------------------------------------------- | | Cloudflare Access | `CF-Access-Client-Id` + `CF-Access-Client-Secret` (service-token pair) | | Cloudflare Bot Management | `Cookie: cf_clearance=; __cf_bm=` (obtained from a browser session) | | AWS CloudFront or Application Load Balancer | Custom allowlist header, for example `X-Origin-Verify: ` | | AWS API Gateway | `x-api-key: ` | | Vercel deployment protection | `x-vercel-protection-bypass: ` (optionally `x-vercel-set-bypass-cookie: true`) | | Vercel password protection | `Cookie: _vercel_jwt=` | | Generic WAF rule | Any custom header and shared secret you allowlist on the WAF | * **Use a browser-driven step** (`goTo`) instead of `checkLink` when the URL truly requires JavaScript or a full browser fingerprint. If you own the site, add a WAF rule that allowlists a custom header and shared secret, such as `X-Doc-Detective-Check`. Prefer that over trying to mimic a browser. ### `checkLink` fails due to unrecognized certificates The `checkLink` action might fail for a valid URL that loads without redirects. An internal or custom certificate the testing machine doesn't recognize is a common cause. #### Example Consider the following test configuration. It checks the validity of `https://self-signed.badssl.com/`, a website using a self-signed certificate: **`bad-certificate.json`** ```json title="bad-certificate.json" { "tests": [ { "steps": [ { "description": "Check site with a self-signed certificate", "checkLink": { "url": "https://self-signed.badssl.com/", "statusCodes": [200, 201, 202, 301] } } ] } ] } ``` To run the test, use the following command: #### NPX ```bash npx doc-detective -i bad-certificate.json ``` #### Docker ```bash docker run -v .:/app docdetective/docdetective -i /app/bad-certificate.json ``` > **Note** > > The `-v` argument mounts your current directory (`.`) to `/app` in the container. This command executes the test, but it fails, returning the following response: ```json { "result": "FAIL", "resultDescription": "Invalid or unresolvable URL: https://self-signed.badssl.com/" } ``` This occurs because the self-signed certificate isn't recognized by the testing machine. This behavior is expected in `axios`, but you can bypass it in Doc Detective by setting an environment variable. #### Solution To fix this issue, follow these steps: 1. Create a `.env` file with the following content: **`ignore-certificate-problems.env`** ```text title="ignore-certificate-problems.env" NODE_TLS_REJECT_UNAUTHORIZED=0 ``` 2. Modify your test configuration to include a `loadVariables` step (Note: `setVariables` is deprecated, use `loadVariables`): **`bad-certificate.json`** ```json title="bad-certificate.json" {5-8} { "tests": [ { "steps": [ { "loadVariables": "ignore-certificate-problems.env" }, { "description": "Check self-signed.badssl.com", "checkLink": { "url": "https://self-signed.badssl.com/", "statusCodes": [200, 201, 202, 301] } } ] } ] } ``` #### Expected result After applying these changes, the test should pass: ```json { "result": "PASS", "resultDescription": "Returned 200" } ```