# ContextDoc: Minimal Context and Verifiable Specifications for Development with AI Agents

Abstract

ContextDoc is a lightweight documentation standard for software development assisted by AI agents. Next to each source file there can be a small YAML file (.ctx) holding only what the code cannot say about itself: architectural decisions that look wrong but are intentional (tensions), flows that span several components (workflows) and declarative, language-agnostic behaviour scenarios (conceptualTests). ContextDoc was born in 2026 from a revision of ConceptDoc (2025), dropping its exhaustive JSON approach in favour of minimal, targeted context. This paper describes the format, the rule that makes conceptual tests a specification agents may not change, the ways the context reaches the agent (instructions, a Claude Code hook, an MCP server) and the tools of version 0.4: ctx-run, which evaluates conceptual tests with an LLM and turns them into pytest tests kept in sync; ctx-watch, which measures drift from git history; ctx-lint, which flags .ctx files that betray the format’s principles; a VS Code extension. A first evaluation, on tasks built to conflict with a tension, shows an agent breaking it in 9 of 12 runs without ContextDoc and in 0 of 12 when it reads the .ctx files. A case study, the limitations and future directions close the paper.

1. Introduction

An AI agent reading a file for the first time sees what the code does, but not why it is written that way. It does not know that a seemingly redundant check protects a billing audit trail, that a slow algorithm is slow on purpose, or that an identifier must never be reused because a downstream system relies on it. Without this context the agent does what a good newly hired developer would do: it simplifies, optimises, cleans up. And it breaks something.

ConceptDoc (2025) tackled the problem with JSON files alongside the code that described everything: metadata, dependencies, components with signatures and pre/postconditions, state models, test fixtures, notes for the AI. Real use showed two flaws:

1. Duplication: most of that information (signatures, dependencies, obvious behaviour) can already be read in the code. Repeating it adds noise to the agent’s context without adding signal.
2. Staleness: the more detailed a documentation file is, the more often it needs updating, and the sooner it stops being updated. Exhaustive, stale documentation is worse than none, because the agent believes it.

ContextDoc turns the approach around: document only what would surprise an agent or a developer reading the code cold. The name changes accordingly, from concepts (formal, complete) to context (minimal, targeted).

1.1 From ConceptDoc to ContextDoc

Of the original sections, the purpose and the conceptual tests survive, cut down to the essentials; conceptual tests remain the most distinctive part of the format. Invariants and business rules merge into tensions; the state model, when needed, becomes a workflow or a link to an external document (ref).

2. The .ctx Format

A ContextDoc file is named after the source file plus .ctx (service.py → service.py.ctx) and lives in the same folder. It is YAML. Every section is optional: use only those that add value. The schema is at version 0.3.0.

2.1 Sections

1. purpose: one line saying what the file does, not how.

2. tensions: the most important section. It lists decisions that look wrong but are intentional, and constraints that must not be broken without deliberate reconsideration. They work like small ADRs (Architecture Decision Records) attached to the file they concern.

3. workflows: non-obvious flows, written as readable sequences (validate input → hash password → persist → send email). They are not exhaustive: they are for paths that cross several components or have a peculiar error branch.

4. conceptualTests: declarative scenarios made of action / expect steps. They describe what the system must do, not how to test it; they name no methods or libraries and therefore survive refactors. Section 3 covers them in detail.

5. todos: pending work that only makes sense in the context of that file.

6. ref: links to external documents, for context too large to keep inline.

2.2 Example

# user_service.py.ctx
purpose: "Manages user authentication and session lifecycle"

tensions:
  - "Tokens are stateless JWT: no revocation, a session cannot be forcibly terminated"
  - "bcrypt cost factor is 12: intentionally slow, do not reduce it for performance"
  - "Email is immutable after registration: downstream systems use it as a stable identifier"

workflows:
  registration: "validate input → hash password → persist user (status: unverified) → send verification email"
  login: "validate credentials → check status is active → issue JWT"

conceptualTests:
  - name: "Registration and activation"
    steps:
      - action: "register with a valid email and a strong password"
        expect: "user created with status 'unverified'"
      - action: "verify the email with a valid token"
        expect: "status becomes 'active'"
      - action: "log in before email verification"
        expect: "error: account not active"

ref:
  - "docs/adr/002-auth-strategy.md"

2.3 What Not to Put in a .ctx

• Method signatures and parameters (they are in the code).
• Dependency lists (visible from the imports).
• Version, author, dates (git keeps them).
• Obvious pre- and postconditions any developer would assume.
• Anything that would need updating every time the implementation changes.

The last point is the most useful practical test: if a line of the .ctx changes every time the code changes, it describes the implementation, not the intent. From version 0.4, ctx-lint (section 5.4) flags the most common cases.

2.4 The project.ctx File

Some constraints belong not to a file but to the whole system: “no hard deletes, anywhere”, “stateless authentication, no shared session store”. For these there is the project.ctx convention, a file with the same schema at the project root.

• project.ctx holds cross-cutting tensions, workflows and tests.
• File-level .ctx files hold what is specific to that file.
• The two levels must not duplicate each other.

Unlike a CLAUDE.md or a .cursorrules, project.ctx is tool-agnostic: any agent or developer opening the repository finds the same constraints, whatever IDE or model they use.

2.5 Machine-Readable Schema

The repository includes a JSON Schema (schema/contextdoc.schema.json) that enables autocomplete and inline documentation in editors built on the YAML Language Server (VS Code with Red Hat’s YAML extension, JetBrains IDEs, Neovim). The schema is deliberately permissive: it checks structure (for example that every conceptual test step has action and expect) without requiring any field. ContextDoc stays documentation, not configuration.

3. Conceptual Tests as the Specification

conceptualTests are what sets ContextDoc apart from docstrings, ADRs and API specifications: they describe expected behaviour in a form a human reads as a checklist and an LLM can compare against the code.

3.1 Authority and Immutability

The central rule of the format is that conceptual tests are the specification: the code adapts to them, not the other way round. An AI agent:

• must never change a conceptual test to make its own code pass;
• if the code cannot satisfy a test, must report the conflict and ask, not rewrite the test.

A conceptual test is updated only when the requirement changes (a human decision, not an implementation convenience) or when the author confirms the test was wrong from the start. It is never updated because the implementation finds it inconvenient, because the agent thinks another behaviour is better, or because making it pass would take too much refactoring. When in doubt: ask, don’t modify.

The rule comes from a behaviour observed in coding agents: faced with a failing test, the shortest path to “green” is often to change the test. Without an explicit ban, the specification bends to the implementation and loses all value. From version 0.4 the rule is not only written down: the Claude Code hook (section 4.4) asks the user to confirm any change to conceptualTests.

3.2 Language Independence

A conceptual test names no functions, classes or frameworks. “Register with an email already in use → error, no new user” holds for a Python, PHP or TypeScript implementation, and stays true after renaming a method or switching library. The same scenarios can be evaluated with an LLM (ctx-run run, section 5.1) or turned into executable tests kept in sync (ctx-run generate, section 5.2).

4. Getting the Context to the Agent

The format says what the system is. For it to matter, the agent has to read it: the evaluation in section 8 shows that a .ctx nobody opens changes nothing. ContextDoc offers three ways, which can be combined.

4.1 Agent Instructions

An instructions file at the project root (CLAUDE.md, .cursorrules or equivalent) tells the agent to read project.ctx and the file’s .ctx before changing it, not to violate a tension without discussing it, to use conceptual tests as the specification and to propose a .ctx update when behaviour changes. Instructions are operational and tied to the tool; context (.ctx) is descriptive and tool-agnostic.

4.2 Reusable Prompts

The repository provides four prompts for recurring operations:

4.3 Git Hook

A pre-commit hook warns when a source file is committed without its .ctx, or without the existing .ctx being updated alongside it. The hook warns but never blocks, consistent with the non-binding nature of the format.

4.4 Claude Code Hook

A PreToolUse hook for Claude Code hands over the context without any instruction:

• when the agent reads, edits or writes a file that has a .ctx, project.ctx and the file’s .ctx are added to its context (once per session and .ctx version, then a one-line reminder), including when it writes a file that doesn’t exist yet but already has its spec, in the intent-first workflow;
• when an edit to a .ctx would change its conceptualTests, the decision goes to the user, who has to approve it.

The hook uses only Python’s standard library and never blocks the agent on errors. It was checked with the Claude Code CLI: in a test project the agent quoted a tension found only in the .ctx, without opening it; without the hook it said it had received none. An attempt to change an expect had no effect until someone approved it.

4.5 MCP Server

ctx-mcp exposes the same files to any agent that speaks MCP (Model Context Protocol): get_context for a file, list_tensions, list_scenarios and check_drift. It is read-only, confined to the project folder, and comes with an instruction for agents: call get_context before editing a file, and don’t touch conceptualTests.

5. Tools

In the 2025 paper the tools were only proposals. Today they are one Python package (pipx install contextdoc, version 0.4.0) and a VS Code extension (0.2.0). What the two share (prompts, cache, lint rules) lives in common specifications, checked by parity tests in both implementations.

5.1 ctx-run run: Evaluating Conceptual Tests with an LLM

ctx-run reads .ctx files, sends the source code together with each scenario to an LLM and reports the outcome step by step (PASS/FAIL). It is static analysis via an LLM, not code execution: a FAIL is a signal to check, not a verdict.

• Provider-agnostic through LiteLLM: Anthropic, OpenAI, Ollama locally or in its cloud.
• Temperature fixed at 0, for results as repeatable as possible.
• Result cache keyed on scenario, source code and model, shared with the extension: a result computed by one doesn’t cost a second call to the other.
• Useful options: --output json for CI, --fail-fast, --verbose, --fix (after a FAIL, asks the LLM for a suggested fix), --timeout for slow local models.

ctx-run run src/ --model ollama/gemma4:e2b
# src/auth.py.ctx
#   scenario: Token lifecycle
#     [PASS] create a token with payload {sub: '42'}
#     [FAIL] decode a token signed with a different key

5.2 ctx-run generate and verify: Executable Tests, Kept in Sync

Evaluation by an LLM is portable but not deterministic. For Python sources, ctx-run generate turns each scenario into pytest tests, one file per .ctx. Every block carries the scenario’s name and a hash of its content:

• on the next run, unchanged scenarios are kept as they are, with no model call; changed ones are regenerated; removed ones are dropped;
• ctx-run generate --check reports missing, stale or orphaned tests without calling the model, for CI;
• ctx-run verify runs the generated tests and reports the outcome per scenario, with the failure message.

The scenario stays the source of truth: a generated test is never edited by hand to make it pass. The generation prompt also includes the local modules imported by the source; without them the model guessed constructors and the tests failed with a TypeError instead of on behaviour. On the project-2 example the generated tests give the same picture as ctx-run run, the complete module passes and the three missing wirings fail, but deterministically.

5.3 ctx-watch: Noticing Drift

The risk of any parallel documentation is that it falls behind without anyone noticing. ctx-watch reports two situations:

• ⚠ drift: the source changed after its .ctx;
• → specification without implementation: a .ctx exists but the source doesn’t (the intent-first case, section 6).

Inside a git repository, ctx-watch status measures drift from the history: a source drifts when commits after the last commit that touched its .ctx changed it, or when it has uncommitted changes and its .ctx doesn’t. File modification times, used up to 0.3, mean nothing after a clone or checkout; with git the answer is the same locally, in CI and on a fresh clone. Outside git (or with --mtime) modification times remain. --base origin/main checks only the files changed in a branch, reporting those without a .ctx too; watch warns on save; .ctxignore excludes files and paths. The extension and the MCP server use the same rules.

5.4 ctx-lint: the Principles, Checked

The schema is permissive by choice, but the principles of section 2.3 were a discipline nobody verified. ctx-lint checks them and only warns, unless --strict is given for CI:

L005 comes from the case study in section 7.2: on that project ctx-lint flags exactly the two .ctx files that had made the code overly defensive. The extension shows the same warnings in its Problems panel.

5.5 VS Code Extension

The ContextDoc extension, on the Visual Studio Marketplace, brings the tools into the editor without needing Python:

• a Run Tests button above every conceptualTests section, with results in a side panel; runs can be cancelled, and a whole-project run asks for confirmation with the number of model calls;
• a setup wizard that starts from where the model should run: gemma4:e2b locally with at least 16 GB of RAM, gemma4 on Ollama’s cloud below 16 GB, another Ollama model (with a warning when it’s too heavy for the machine) or a paid API; it ends with a test call;
• Ollama called through its native API with a context sized to the prompt: through the OpenAI-compatible API it reserved the model’s full context, and a 12-billion-parameter model took 16 GB of RAM instead of 8;
• drift detection and lint built in, with the rules of ctx-watch and ctx-lint.

The first published version (0.1.0) didn’t start, because of a library missing from the package, and still needed Python for drift. It was fixed in 0.1.1, and since then CI installs the final package in a clean VS Code and exercises it on every code change.

6. Intent-First Development

ContextDoc enables a workflow similar to TDD: the .ctx is written before the source file exists.

1. You write notification_service.py.ctx with tensions, workflows and conceptual tests. This is the “red” state: ctx-watch status --reverse reports it as a specification without an implementation and exits with code 1.
2. The agent implements the file starting from a complete specification; the hook hands it the .ctx as it creates the file.
3. The file exists: ctx-watch --reverse turns green; ctx-run generate and verify (or ctx-run run) say whether the implementation satisfies the scenarios.

In this workflow the immutability rule of section 3.1 is decisive: if the agent could touch up the tests, “green” would no longer mean anything.

7. Examples and Case Study

7.1 Examples in the Repository

Every example includes a CLAUDE.md showing how context and operational instructions work together. The examples are also the basis of the evaluation in section 8.

7.2 A Real Project: an ASCII 3D Engine in JavaScript

ContextDoc was adopted in a personal project unrelated to the format: a JavaScript graphics engine that projects 3D shapes in perspective and draws them on a canvas or as ASCII text with a z-buffer. Twenty-two source files, fourteen .ctx files, with ctx-run and ctx-watch running in a Docker container with a model served through Ollama, called from a Makefile.

Three observations:

• The .ctx files stay small. The text renderer’s one has four tensions, one workflow and five scenarios: it reads in under a minute and says exactly what must not change (for example that the depth range is explicit to avoid implicit assumptions).
• The .ctx name can collide. For Docker the project uses an environment file called .env.ctx, which is not a ContextDoc. The case is documented together with .ctxignore: exclude it from the checks and associate it with another language in the editor.
• Conceptual tests shape the code, even too much. Several scenarios asked for an “explicit error” for every missing or invalid parameter, and the implementation followed them to the letter: fail-fast checks in every constructor, internal objects included. The specification worked, but produced code more defensive and verbose than needed. ctx-lint’s L005 rule comes from this observation.

8. Evaluation

8.1 Agents with and without ContextDoc

Question: do .ctx files stop an agent from breaking a documented decision? Four tasks on the examples each ask for a change whose obvious implementation breaks a tension, the kind of request made by someone who doesn’t know the constraint:

The agent is Claude Code (Haiku model) in non-interactive mode, on a clean copy of the example, in four conditions: no .ctx; .ctx files only; .ctx files and the example’s CLAUDE.md, which says to read them; .ctx files and the hook of section 4.4, without CLAUDE.md. User settings, plugins and MCP servers are excluded. Violations are detected from the code, or by running it, never from the agent’s answer. Three runs per cell, 48 runs:

• A .ctx nobody reads changes nothing: with the files alone and no instruction the agent didn’t open them, and behaved as without them.
• With the instruction to read them, no violations: in 10 of 12 runs the agent changed no file and stopped to ask, quoting the tension.
• With the hook, 2 violations in 12: the context arrives without instructions, and in 8 of 12 runs the agent stops. In the two violations it carried out the request anyway, once flagging the conflict with the conceptual test only afterwards.
• In none of the 48 runs did the agent modify the conceptualTests.

8.2 Models for ctx-run

On the examples, 68 steps of which 5 correspond to real gaps in the code, checked by hand:

The light local model catches the obvious gaps and misses the most subtle one; even at temperature 0 it changes its verdict on 3–4 steps between runs. These numbers decided the models the extension suggests. The script and the reference are in the repository, to repeat the measurement with other models.

9. Limitations

• ctx-run run doesn’t execute the code. An LLM reasons about the source and can be wrong both ways; different models, and even different runs, give different outcomes. Generated tests (section 5.2) are deterministic, but only for Python for now, and they are as good as the model that wrote them.
• Drift doesn’t look at content. With git the computation is reliable, but it is still about commits: a .ctx changed in an irrelevant way alongside the code passes. Outside git it falls back to modification times.
• Quality depends on the author. ctx-lint catches the most common mistakes, not whether a tension is relevant.
• A small evaluation. One model for the agents, three runs per case, four tasks built for the conflict and examples small enough to read in full. The numbers show a direction, not a rate to quote.
• Cost. ctx-run run makes one model call per uncached scenario; on large projects with paid models the cost needs considering, or use a local model or the generated tests.

10. Future Directions

1. A broader evaluation: other models, more runs, tasks taken from real projects rather than built for the conflict.
2. Generated tests for other languages: Jest and PHPUnit, with the same hashing and sync mechanism.
3. Content-aware drift: reporting not only that the code changed after the .ctx, but whether the change actually touches a tension or a scenario.
4. Other agents and editors: the equivalent of the hook for other tools, and the extension on Open VSX too.

11. Conclusion

ContextDoc is what is left of ConceptDoc after really using it: fewer sections, fewer lines, more discipline about what deserves to be written down. Tensions protect the decisions an agent would be tempted to “improve”; conceptual tests give the agent a specification it cannot bend to its own advantage; the tools make visible when the documentation stops telling the truth, and turn it into tests when certainty is needed. The first measurement confirms the idea and marks its limit: context works when it reaches the agent, and is useless in a file nobody opens.

Further Reading

1. JSON Schema
2. YAML Language Server
3. LiteLLM
4. Model Context Protocol
5. Michael Nygard, “Documenting Architecture Decisions”, 2011
6. Dan North, “Introducing BDD”, 2006
7. Kent Beck, Test-Driven Development: By Example, Addison-Wesley, 2002

Appendix: Resources

• ContextDoc repository
• Code of version 0.4.0
• Schema specification (v0.3.0)
• Benchmarks: agents and models
• The contextdoc package on PyPI
• VS Code extension
• ConceptDoc paper (2025), the previous version
• Paper IT