Skip to navigation
Test your docsFormats & surfacesInput formats

Custom input formats

You can define custom input formats and extend existing formats by modifying your configuration file. This allows you to support new file types or customize test detection for your specific documentation patterns.

Creating a custom file type

To define a custom file type, add an entry to the fileTypes array in your configuration:

{
  "fileTypes": [
    {
      "name": "custom-format",
      "extensions": ["custom", "cst"],
      "testStartStatementOpen": "## TEST START:",
      "testStartStatementClose": "",
      "testEndStatementOpen": "## TEST END",
      "testEndStatementClose": "",
      "stepStatementOpen": "## STEP:",
      "stepStatementClose": "",
      "markup": []
    }
  ]
}

Configuration properties

Required properties:

  • name: A descriptive name for the file type
  • extensions: Array of file extensions to process with this configuration

Optional properties for inline tests:

  • testStartStatementOpen: Pattern that starts a test declaration (for example, "<!-- test ")
  • testStartStatementClose: Pattern that ends a test declaration (for example, " -->")
  • testEndStatementOpen: Pattern that marks the end of a test (for example, "<!-- test end")
  • testEndStatementClose: Pattern that closes the test end marker (for example, " -->")
  • stepStatementOpen: Pattern that starts a step declaration (for example, "<!-- step ")
  • stepStatementClose: Pattern that ends a step declaration (for example, " -->")
  • testIgnoreStatement: Pattern for ignoring blocks

Optional properties for detected tests:

  • markup: Array of markup pattern objects that define how to extract tests from content

Adding custom markup patterns

Markup patterns define how Doc Detective extracts tests from your documentation content. Each pattern includes:

  • name: Descriptive name for the pattern
  • regex: Array of regular expressions to match content
  • actions: Array of actions to generate when pattern matches

Example: Custom Markdown pattern

Add a pattern to detect custom syntax for API calls:

{
  "fileTypes": [
    {
      "name": "markdown",
      "extensions": ["md", "markdown", "mdx"],
      "markup": [
        {
          "name": "apiCall",
          "regex": ["API:\\s*(GET|POST|PUT|DELETE)\\s+`([^`]+)`"],
          "actions": [
            {
              "httpRequest": {
                "method": "$1",
                "url": "$2"
              }
            }
          ]
        }
      ]
    }
  ]
}

This pattern matches content like:

API: GET `https://api.example.com/users`

And generates:

{
  "httpRequest": {
    "method": "GET",
    "url": "https://api.example.com/users"
  }
}

Using capture groups

Regular expression capture groups (numbered 1,1, 2, etc.) are substituted into the action definition:

{
  "name": "customFind",
  "regex": ["Find the (\\w+) element"],
  "actions": [
    {
      "find": {
        "selector": "aria/$1",
        "timeout": 5000
      }
    }
  ]
}

Matches “Find the button element” and generates:

{
  "find": {
    "selector": "aria/button",
    "timeout": 5000
  }
}

Default actions

For basic patterns, you can use action shortcuts. Doc Detective substitutes the first capture group (or entire match) into the default location:

{
  "name": "simpleClick",
  "regex": ["Click: (\\w+)"],
  "actions": ["click"]
}

This is equivalent to:

{
  "name": "simpleClick",
  "regex": ["Click: (\\w+)"],
  "actions": [{ "click": "$1" }]
}

Extending existing formats

You can extend existing formats by adding patterns to the default configuration. Doc Detective merges your custom patterns with the built-in patterns.

Example: Enhanced Markdown

Add patterns for RST-style directives in Markdown:

{
  "fileTypes": [
    {
      "name": "markdown",
      "extensions": ["md", "markdown", "mdx"],
      "markup": [
        {
          "name": "rstDirective",
          "regex": ["\\.\\. (\\w+)::\\s*([^\\n]+)"],
          "actions": ["$1"]
        },
        {
          "name": "codeDirective",
          "regex": ["\\.\\. code::\\s*(\\w+)\\s*\\n\\s+([\\s\\S]+?)\\n\\n"],
          "actions": [
            {
              "runCode": {
                "language": "$1",
                "code": "$2"
              }
            }
          ]
        }
      ]
    }
  ]
}

Running files as shell commands

Not every custom file type contains inline tests or markup patterns. Sometimes the file itself is the test: a script, a notebook, or any file you’d normally run from a shell. For these cases, define an executable file type that maps an extension to a runShell command. Doc Detective turns each matching file into a single test with one runShell step. Pointing it at the file then runs the file and asserts that it exits cleanly.

An executable file type needs only two properties:

  • extensions: The file extensions to run with this command.
  • runShell: The runShell step to perform. Use $1 as a placeholder for the path of the matched file.

For example, this configuration runs every .js file with Node.js, every .py file with Python, and runs Jupyter notebooks through jupyter nbconvert:

{
  "fileTypes": [
    {
      "name": "JavaScript",
      "extensions": "js",
      "runShell": {
        "command": "node $1"
      }
    },
    {
      "name": "Python",
      "extensions": "py",
      "runShell": {
        "command": "python $1"
      }
    },
    {
      "name": "Jupyter Notebooks",
      "extensions": "ipynb",
      "runShell": {
        "command": "jupyter",
        "args": ["nbconvert", "--to", "script", "--execute", "$1", "--stdout"]
      }
    }
  ]
}

The runShell value accepts the same fields as a runShell step, so you can assert on exit codes with exitCodes and on output with stdio. A file whose command exits with a non-zero code fails the test, as does one whose output doesn’t match. That mirrors a standalone runShell step.

A file type is either a markup file type or a runShell file type, not both. When a file type defines runShell, Doc Detective runs the file as a command and doesn’t parse it for inline tests or markup.

Example: Custom documentation format

Here’s a complete example for a hypothetical custom documentation format:

{
  "fileTypes": [
    {
      "name": "custom-docs",
      "extensions": ["cdoc"],
      "testStartStatementOpen": "TEST[",
      "testStartStatementClose": "]",
      "testEndStatementOpen": "END_TEST",
      "testEndStatementClose": "",
      "stepStatementOpen": "STEP[",
      "stepStatementClose": "]",
      "testIgnoreStatement": "IGNORE",
      "markup": [
        {
          "name": "navigate",
          "regex": ["@navigate\\s+(.+)"],
          "actions": ["goTo"]
        },
        {
          "name": "interact",
          "regex": ["@(click|type|find)\\s+\"([^\"]+)\""],
          "actions": ["$1"]
        },
        {
          "name": "verify",
          "regex": ["@verify\\s+(.+?)\\s+contains\\s+\"([^\"]+)\""],
          "actions": [
            {
              "find": {
                "selector": "$1",
                "matchText": "$2"
              }
            }
          ]
        }
      ]
    }
  ]
}

This configuration supports a custom format like:

TEST[testId: "custom-test"]
# Getting Started
@navigate https://example.com
@click "Login"
@type "username"
@verify #welcome contains "Welcome"
END_TEST

Testing custom patterns

To test your custom patterns:

  1. Create a sample documentation file in your format
  2. Add test comments or content that matches your patterns
  3. Run Doc Detective with the --input flag pointing to your file
  4. Check the generated test results to verify patterns were detected correctly
npx doc-detective --input sample.cdoc --config custom-config.json

Best practices

  • Start with basics: Begin with basic patterns and add complexity as needed
  • Test thoroughly: Validate patterns with various input examples
  • Use descriptive names: Give patterns descriptive names so you can understand their purpose
  • Document patterns: Add comments in your config to explain custom patterns
  • Consider escaping: Properly escape special characters in regex patterns
  • Optimize regex: Use specific patterns to avoid unintended matches
  • Version control: Track configuration changes to understand pattern evolution

Common use cases

Custom input formats are useful for:

  • Proprietary formats: Internal documentation formats used by your organization
  • Legacy systems: Older documentation systems with specific markup
  • Domain-specific languages: DSLs for specialized documentation
  • Enhanced detection: More sophisticated pattern matching for existing formats
  • Tool integration: Custom formats from specific authoring tools

Next steps