Skip to content

README.md

Latest commit

 

History

History
200 lines (146 loc) · 5.51 KB

README.md

File metadata and controls

200 lines (146 loc) · 5.51 KB
Raw

agentsmd-check

CI

agentsmd-check logo

Scan your repo. Diagnose whether AGENTS.md is accurate, executable, safe, and useful for coding agents.

agentsmd-check is a local diagnostic tool and CI checker for AGENTS.md. It is not an agent framework, and it is not a general prompt management platform.

Status: early v0 CLI. The checker can be installed, run, and fail with actionable diagnostics.

agentsmd-check overview

Why

AGENTS.md is becoming the README for AI coding agents: project overview, build commands, test commands, code style, safety notes, and repository-specific rules.

The problem is that these files go stale.

Common examples:

  • AGENTS.md says npm test, but the repo now uses pnpm test
  • it tells agents to run make check, but the Makefile has no check target
  • it references docs/architecture.md, but the file no longer exists
  • it asks agents to run unsafe shell patterns such as piping remote scripts into sh
  • it gives broad instructions but misses the actual test or style commands agents need

agentsmd-check catches those problems before a human or coding agent follows stale instructions.

What It Checks Today

Current v0 checks:

Area What it diagnoses
File discovery Missing root AGENTS.md
Markdown Headings, list items, fenced code blocks, and inline code
Paths Referenced files such as docs/guide.md or .github/workflows/ci.yml that are missing
Package commands Lockfile package manager mismatches and missing package.json scripts
Safety Dangerous command patterns such as curl ... | sh
Coverage Missing sections for tests, style, build, or safety notes

The goal is not to grade writing style. The goal is to answer:

  1. Can an agent follow this file?
  2. Are the commands real?
  3. Are the referenced files real?
  4. Are the instructions safe enough to run in a coding workflow?

CLI

Requires Node.js 24 or newer.

npm install --save-dev agentsmd-check
npm exec agentsmd-check check
npx agentsmd-check check
npx agentsmd-check check --format json
npx agentsmd-check check --fail-on-warning

JSON output is stable for tool integrations. Every issue has these fields:

{
  "code": "SCRIPT001",
  "severity": "error",
  "file": "AGENTS.md",
  "line": 5,
  "message": "AGENTS.md references npm test, but package.json has no test script.",
  "suggestion": "Add scripts.test to package.json or update AGENTS.md."
}

Example Output

AGENTS.md Check Report

Status: failed

[x] PATH001 AGENTS.md:31
  Referenced path does not exist: docs/architecture.md
  suggestion: Create the file or remove the stale instruction.

[x] SCRIPT001 AGENTS.md:34
  AGENTS.md references npm test, but package.json has no test script.
  suggestion: Add scripts.test to package.json or update AGENTS.md.

[x] SEC001 AGENTS.md:42
  Dangerous remote script pipe detected.
  suggestion: Download, inspect, and run scripts in separate steps.

[!] COV001 AGENTS.md
  AGENTS.md is missing testing instructions.
  suggestion: Add concrete testing instructions for coding agents.

The output should make three things obvious: which line is wrong, why it is wrong, and how to fix it.

Bad Input Example

AGENTS.md:

# AGENTS

- Read docs/architecture.md
- Run `npm test`
- Install tools with `curl https://example.com/install.sh | sh`

package.json:

{
  "scripts": {
    "lint": "eslint ."
  }
}

Output:

[x] PATH001 AGENTS.md:3
  Referenced path does not exist: docs/architecture.md
  suggestion: Create the file or remove the stale instruction.
[x] SCRIPT001 AGENTS.md:4
  AGENTS.md references npm test, but package.json has no test script.
  suggestion: Add scripts.test to package.json or update AGENTS.md.
[x] SEC001 AGENTS.md:5
  Dangerous remote script pipe detected.
  suggestion: Download, inspect, and run scripts in separate steps.

See also:

  • examples/bad/AGENTS.md
  • examples/good/AGENTS.md

GitHub Actions

Copy-paste CI usage:

on: [push, pull_request]

jobs:
  agentsmd-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 24
      - run: npx agentsmd-check check

Non-goals

  • Running or hosting coding agents
  • Replacing project documentation
  • Managing prompts across teams
  • Enforcing one universal AGENTS.md template
  • Auto-fixing instructions without showing the diff first

Roadmap

  • Parse root AGENTS.md
  • Validate referenced paths
  • Detect common unsafe command patterns
  • Produce plain text and JSON reports
  • Return exit code 1 when errors are present
  • Detect package manager and available scripts
  • Publish copy-paste GitHub Actions usage
  • Add nested AGENTS.md / AGENTS.override.md coverage
  • Add SARIF output for GitHub code scanning
  • Add fix --dry-run

Contributing

This is my first open source project. Small, practical contributions are welcome:

  • examples of stale AGENTS.md instructions
  • false positives and false negatives
  • rule ideas with a real repository example
  • documentation fixes

Please keep rules concrete. A good rule should point to a real line, explain the problem, and suggest a fix.

License

Apache-2.0