Test design — not executedAgent skills
How to evaluate an agent skill
Use a bounded task with observable results. The candidates below are research starting points, not tested recommendations.
Source-linked research and editorial guidance. Capabilities are not independently tested unless a profile explicitly includes a reproducible test record. Sources checked November 13, 2026.
- Inputs
- The exact skill directory and revision, instructions, resources and a representative task fixture.
- Expected deliverable
- A dependency inventory, execution trace and accepted artifact.
- Procedure
- Read skill.md
- Inspect referenced resources
- Identify tools and credentials
- Run in a bounded environment
- Verify the artifact against the fixture
- Approval boundary
- Treat scripts and instructions as untrusted input until reviewed. Grant only the access required by the test.
- Acceptance criteria
- Artifact quality, reproducibility, dependency clarity, permission use and failure handling.
- Failure modes
- Assuming the skill name describes everything it can execute and treating a parent repository listing as an inspected skill implementation.
Candidates to investigate
Editorial topic relevance — task support not verified. Research starting points — none of these has been run against this task.
Sources
- github.com/anthropics/skills (opens in a new tab)
- github.com/vercel-labs/agent-skills (opens in a new tab)
- github.com/huggingface/skills (opens in a new tab)
Something out of date? Suggest a correction