A pytest plugin for testing bash command examples in markdown documentation.
API documentation gets outdated fast. Manually testing curl examples is tedious. This plugin automatically tests bash code blocks in your markdown files against actual API responses, keeping docs and code in sync.
pip install pytest-bashdoctestOr with uv:
uv add pytest-bashdoctestIn your API.md:
## User API
```bash
$ curl -s "https://api.github.com/users/dnouri"
{
"login": "dnouri",
...
"name": "Daniel Nouri",
...
"bio": "Machine Learning and Programming",
...
}
```The ... pattern matches any content, perfect for skipping dynamic or irrelevant fields.
pytest --bashdoctest API.md -vYour documentation is now executable and verified.
Note: The --bashdoctest flag is required to enable bash doctest collection. This prevents conflicts with other pytest plugins that also process markdown files (like pytest-markdown-docs).
Use standalone ... to skip entire sections:
```bash
$ curl -s "https://api.github.com/users/dnouri"
{
"login": "dnouri",
...
"type": "User",
...
"bio": "Machine Learning and Programming",
...
}
```Show the fields that matter, skip the rest.
Use ... inside strings for dynamic values:
```bash
$ curl -s "https://api.github.com/users/dnouri"
{
"login": "dnouri",
...
"avatar_url": "https://avatars.githubusercontent.com/u/...?v=4",
...
"created_at": "2009-...-...T...Z",
...
}
```Useful for URLs, UUIDs, timestamps, and other values that change but follow a pattern.
Use {...} and [...] to match entire structures:
```bash
$ curl -s "https://api.github.com/repos/dnouri/nolearn"
{
...
"name": "nolearn",
...
"owner": {...},
...
"license": {...},
...
}
```Focus on the structure you care about without showing every detail.
Mix all three for flexible matching:
```bash
$ curl -s "https://api.github.com/users/dnouri"
{
"login": "dnouri",
...
"avatar_url": "https://avatars.githubusercontent.com/...",
...
"bio": "Machine Learning and Programming",
...
"public_repos": ...,
...
}
```Explicit paths:
pytest --bashdoctest README.md -v
pytest --bashdoctest docs/api.md -vVia configuration in pyproject.toml:
[tool.pytest.ini_options]
testpaths = ["tests", "README.md", "docs"]Then pytest --bashdoctest will test both your test suite and documentation.
The Problem: Your bash examples often need API keys, custom URLs, or other configuration. Hard-coding these in documentation is insecure and inflexible.
The Solution: The bashdoctest_env fixture. This session-scoped fixture provides environment variables to all bash examples in your markdown files.
When you DON'T need it: If your bash examples only use public APIs without authentication (like the GitHub examples in this README), you don't need to define this fixture. The plugin provides a default empty fixture.
When you DO need it: If your examples need API keys, custom URLs, or other config, create conftest.py:
import os
import pytest
@pytest.fixture(scope="session")
def bashdoctest_env():
"""Environment variables for bash documentation examples."""
api_key = os.getenv("API_KEY")
if not api_key:
pytest.skip("API_KEY not set - skipping bash doctest examples")
return {
"API_KEY": api_key,
"API_URL": os.getenv("API_URL", "https://api.example.com"),
}Use in your markdown files:
```
$ curl -s "$API_URL/users" -H "Authorization: Bearer $API_KEY"
{
"users": [...]
}
```The environment variables from your bashdoctest_env fixture are merged with os.environ when executing commands, so $API_KEY and $API_URL will be available to all bash examples.
- Plugin activates when you pass
--bashdoctestflag - Collects markdown files you specify (explicit paths or via
testpaths) - Parser extracts bash code blocks (only files with actual bash examples)
- Each bash block becomes a test item
- Commands execute with your
bashdoctest_envvariables merged into environment - Output matches against expected using ELLIPSIS rules
Architecture:
src/pytest_bashdoctest/
parser.py # Extract bash blocks
matcher.py # ELLIPSIS matching
executor.py # Run commands
formatter.py # Format failures
plugin.py # Pytest hooks
Core modules are pure Python (no pytest dependency) for portability.
# Run unit tests
pytest tests/ -v
# Test with coverage
pytest tests/ --cov=pytest_bashdoctest --cov-report=term-missing
# Test this README (dogfooding!)
pytest --bashdoctest README.md -v- Commands timeout after 30 seconds (configurable in
executor.py) - Interactive commands will hang
- Uses
shell=True(commands come from your trusted markdown files) - Output buffering may differ for very large responses
git clone https://github.com/dnouri/pytest-bashdoctest.git
cd pytest-bashdoctest
# Install dependencies
uv sync --dev
# Install pre-commit tool globally (recommended, one-time setup)
uv tool install pre-commit
# Install git hooks (runs automatically on every commit)
pre-commit install# Run unit tests
uv run pytest tests/ -v
# Test README examples
uv run pytest --bashdoctest README.md -v
# Run with coverage
uv run pytest tests/ --cov=pytest_bashdoctest --cov-report=term-missingPre-commit hooks automatically run on every commit to ensure code quality. To run manually:
# Run all pre-commit hooks (after uv tool install pre-commit)
pre-commit run --all-files
# Or using project dependencies
uv run pre-commit run --all-files
# Run specific checks
uv run ruff check . # Linting
uv run ruff format . # Auto-format code
uv run ruff check --fix . # Auto-fix linting issuesTo skip pre-commit hooks (not recommended):
git commit --no-verify -m "message"Update version in pyproject.toml, then:
git tag v2025.10.4 && git push --tagsPublishes to PyPI automatically via GitHub Actions.
MIT License - see LICENSE file for details.