skill-check addresses a practical pain point in AI agent development: ensuring the quality and consistency of SKILL.md files that define an agent’s capabilities. It transforms what is often a subjective review process into a rigorous, quantifiable validation and scoring system.
What skill-check does and its architecture
skill-check is a CLI tool written in TypeScript that lints SKILL.md files used to define AI agent skills. It validates these skill files with a focus on five weighted quality categories: frontmatter (30%), description (30%), body content (20%), links (10%), and file formatting (10%). This weighting reflects the relative importance of each section to overall skill usefulness and clarity.
The tool supports scanning both local directories and remote GitHub URLs, which makes it flexible for CI pipelines and developer workflows. It also includes an optional security scanning feature called agent-scan that performs additional checks on the skill content for potential risks.
One of skill-check’s strengths is its multi-format output capability. It can export results as plain text, JSON, SARIF (Static Analysis Results Interchange Format), HTML reports, and even GitHub annotations, which integrate directly into pull requests for developer feedback.
Additional features include baseline comparison to track quality changes over time, duplicate detection to help maintain skill uniqueness, and a watch mode for continuous validation during development.
Under the hood, skill-check leverages TypeScript for type safety and maintainability. The codebase focuses on modular validations and scoring computations, facilitating extensibility and clear separation of concerns.
Technical strengths and design tradeoffs
What distinguishes skill-check is its systematic approach to quantifying skill quality on a 0-100 scale. By assigning weights to different skill sections, it turns subjective review criteria into a reproducible metric. This is particularly useful in environments where many skills need to be maintained consistently.
The validation process goes beyond simple syntax checks; it includes semantic validations such as frontmatter correctness, link integrity, and content completeness. This attention to detail helps catch subtle issues that might degrade an agent’s performance or user experience.
The security scanning integration adds another layer of confidence, especially important as AI agent skills increasingly interact with external systems or sensitive data.
The multi-format exporting is a practical design choice that supports diverse use cases: JSON and SARIF for automated tooling, HTML for human-readable reports, and GitHub annotations for seamless developer workflows.
There are tradeoffs, of course. The scoring system, while helpful, is only as good as the weighting scheme and validation rules defined. It might not capture all nuances of skill effectiveness or creativity. Also, the tool is specialized for SKILL.md format and may not generalize to other agent skill definitions.
The watch mode and duplicate detection features improve developer experience but add complexity to the codebase and runtime behavior.
Overall, the code quality is clean and modular, with a clear CLI interface and configuration options that make the tool adaptable to different projects.
Quick start
skill-check provides a straightforward CLI interface, installable and runnable with minimal fuss:
npx skill-check .
For a global install, you can use the provided install script:
curl -fsSL https://raw.githubusercontent.com/thedaviddias/skill-check/main/scripts/install.sh | bash
Or install via Homebrew on macOS:
brew tap thedaviddias/skill-check https://github.com/thedaviddias/skill-check
brew install skill-check
For developers contributing to the project, the development quick start is:
pnpm install
pnpm run check
pnpm run report
You can also generate real CLI outputs using multi-skill fixtures to validate behavior and security scanning:
pnpm run smoke:cli
This smoke test writes outputs to reports/smoke/ and runs real security scans and auto-fix attempts on test skill files.
Finally, to create a configuration interactively:
npx skill-check init --interactive
verdict
skill-check is highly relevant for teams and projects that maintain multiple AI agent skills and need a consistent quality gate. Its quantitative scoring system, multi-format outputs, and security scanning provide a solid foundation for enforcing standards in CI/CD pipelines.
The tool’s specialization on SKILL.md files means it’s not a general-purpose linter but rather focused on a very specific niche in AI agent development. This narrow scope is a strength for those in the space but limits broader applicability.
The scoring and validation rules represent a tradeoff between automation and human judgment; some quality aspects may still require manual review. However, skill-check reduces the routine overhead and surfaces issues early.
If your workflow involves building, sharing, and evolving AI agent skills defined in SKILL.md files, skill-check is worth evaluating for integration. It improves developer experience by catching issues before deployment and provides clarity on skill quality through its measurable scoring.
Related Articles
- SkillUI: Automating design system extraction into Claude Code skills — SkillUI reverse-engineers design systems from URLs or repos into Claude Code skill packages using static and visual anal
- DSPy agent skills pack with GEPA optimization for Claude Code and Codex CLI — Explore a production-grade pack of DSPy 3.2.x agent skills with GEPA optimization, delivering up to +19.53 accuracy on R
- ok-skills: a universal skill pack for AI coding agents — ok-skills provides 38 reusable SKILL.md playbooks for AI coding agents like Claude Code and Cursor, standardizing cross-
- AgentShield: auditing AI agent security configurations with runtime confidence scoring — AgentShield is a TypeScript CLI tool that audits Claude Code AI agent configs for secrets, permissions, hooks, and more
- standardizing AI agent capabilities with sanjay3290/ai-skills — Explore sanjay3290/ai-skills, a portable skill collection implementing the open Agent Skills Standard for cross-platform
→ GitHub Repo: thedaviddias/skill-check ⭐ 176 · TypeScript