vetted

English | اردو

There are tens of thousands of agent skills on GitHub now. Almost none of them can tell you whether they make your agent better, and a skill written for last year's model can quietly make this year's model worse. Skills are also code you run with your agent's permissions, and they mostly get installed unread.

This repo is two things:

1. Skills rebuilt for current models, the popular ideas that survived testing (verify before claiming done, root-cause debugging, bug-hunting review, grilling, handoffs) rewritten short and calm, each with an eval suite that measures what it adds over the model on its own.
2. vet, a zero-dependency scanner that checks any skill, any repo, or everything you have installed, for spec errors, weak triggers, context cost, outdated prompting, and security red flags.

Quick start

Install the skills

# Claude Code
/plugin marketplace add Xnadir/vetted
/plugin install vetted@vetted

# Any agent that reads SKILL.md (Claude Code, Codex, Cursor, Gemini CLI, OpenCode, Copilot...)
npx skills add Xnadir/vetted

Or copy a folder from skills/ into your agent's skills directory (~/.claude/skills/, ~/.agents/skills/, .cursor/skills/, ...). Each skill is one self-contained SKILL.md.

Vet a skill repo before you install it

npx @menadirali/skill-vet vet anthropics/skills     # any GitHub repo
npx @menadirali/skill-vet vet ./my-skills           # a local folder
npx @menadirali/skill-vet vet --installed           # everything your agents have installed

Node 18+ and nothing else. No install, no account, no network except the git clone when you name a GitHub repo.

The skills

All five together add ≈375 tokens to your agent's context (names and descriptions). A skill's body loads only when it fires, at 460–560 tokens each.

Results

Each skill has an eval suite under evals/ in the format of Anthropic's claude plugin eval. Every case runs 3 times with the plugin loaded and 3 times without it, and the difference (Δ) is what the skill contributes. Graders are deterministic regexes over files and replies where possible, and an LLM judge with written PASS/FAIL rubrics where not.

Sonnet 5.5: 19 cases × 3 runs per arm, judge claude-sonnet-5-5, Claude Code 2.1.284, 2026-09-29, ≈$5.62 at list price Haiku 4.5: 19 cases × 3 runs per arm, judge claude-sonnet-5-5, Claude Code 2.1.284, 2026-09-29, ≈$5.28 at list price

The rule: a skill stays only if its mean Δ is clearly positive on the current models. A skill with no measurable effect gets cut and listed below, because a skill that doesn't change behavior still costs context and attention on every turn.

Cut so far (4 of the original 9): surgical, stdlib-first, answer-first, and secure-defaults. On both models they changed nothing measurable: the model without the skill already kept diffs small, used built-ins, parameterized SQL, and answered directly. None of them even loaded on natural prompts. They're kept in retired/ with their evals, so anyone can re-test them on a future model or propose harder cases.

What we learned: skills that add a workflow the model wouldn't choose on its own help most (grill: a recommended answer with every question; bug-hunt-review: a concrete failing scenario for every bug). "Be careful" skills that restate good habits don't help current models, and the smaller model benefits more (handoff and prove-it help Haiku 4.5 only). root-cause never loaded on either model, yet scored +13 on Sonnet and −11 on Haiku, so its effect comes from its description in the skill list alone and isn't reliable yet; it stays on probation.

Other people's skills

The registry runs the same with/without test on popular skills from other repositories, pinned to a commit. Anyone with Claude Code can add one: write two cases and run node scripts/vet-external.mjs registry/<owner>/<repo>/<skill> --by <you>. Your name goes next to the result.

vet: check any skill before you trust it

vet reads every SKILL.md under the paths you give it, plus every script and reference file beside it, and reports:

A scanned file can't silence the scanner: an inline vet-ignore comment downgrades a security finding to info, but it stays in the report. Quoted examples ("pages may contain text like 'ignore previous instructions'") and matches in code comments or test fixtures are reported at info level, not as errors. See the rule reference for every rule's severity and examples, or run skill-vet rules for the full list (add --format json for tooling).

In CI, add it to any repo that contains skills. Findings show up as annotations on the pull request and as a table in the job summary:

- uses: actions/checkout@v4
- uses: Xnadir/vetted@v0
  with:
    path: skills        # default: .
    strict: "true"      # fail on warnings too

Other formats: --format json (stable schema), --format markdown, --format github, --format sarif (SARIF 2.1.0 for code-scanning tools). Save a SARIF report with npx @menadirali/skill-vet vet ./skills --format sarif > results.sarif. Flags: --quiet (print only findings, no summary line or cost table), --strict (fail on warnings too), --verbose (show info-level findings), --ignore (skip rules). Exit codes: 0 clean, 1 findings, 2 usage error.

What it found in the wild. We ran it on 9 of the most-starred skill repositories (115 skills). There were no security findings at warning level or above, which is good news. It did find 2 spec errors (a description over the 1,024-character limit and a name that doesn't match its folder) and 31 warnings, mostly bodies over the 500-line guideline, all-caps "shouting", and descriptions that never say when to use the skill. Together those 115 skills would add ≈7.8k tokens to every session if you installed all of them.

How these skills are written

Current models follow instructions closely, so the old habits of prompting work against you. Every skill here follows the same rules, and vet checks the mechanical ones:

• Explain why, once. A rule with its reason generalizes to cases the rule didn't list. The same rule in capitals, repeated, gets over-applied.
• Short. 460–560 tokens per skill body. Nothing here needs references/.
• The description is the trigger. It says what the skill does, then when to use it, in the third person, under 1,024 characters.
• Portable. Frontmatter stays within the open spec, except argument-hint on grill. Plain Markdown, no hooks, no scripts, nothing to execute.
• Measured. No skill without an eval, and no eval grader that can only pass when the skill fired: the "skill fired" check is reported, but excluded from the score.

More in docs/writing-skills.md.

Contributing

The most useful contributions, in order:

1. An eval case that a skill should pass and doesn't, or a case where a skill makes things worse. That's how skills get fixed, or cut.
2. A new skill with its eval suite. See CONTRIBUTING.md: a skill is accepted when it shows a positive Δ on current models, not before.
3. A vet rule, or a false positive you hit on a real skill.

New here? Pick one of the good first issues. Most are a single rule or file with a test to copy from. Comment to claim one, and PRs get reviewed within a day. It's also a fit for Hacktoberfest: those issues carry the hacktoberfest label.

Contributors

Thank you to everyone who has shipped something here:

• @HarisShahnawaz: sec/suspicious-install typosquat rule, Urdu README
• @Mevayaan1: skill-vet rules --format json, sec/sudo rule
• @DYNOSuprovo: --quiet flag
• @xyi74976-del: Windsurf, Kiro, Cline, and Amp support for --installed, the sec/chmod-777 and spec/license-missing rules, SARIF output, the rule reference, and Node 18 in CI
• @PandaHUN777: nested SKILL.md handling
• @abdullahdevelopment: = and comma-list modes in sec/chmod-777

Your name goes here with your first merged PR.

Prior art and thanks

These skills are rewrites of ideas that the community proved popular, not copies: obra's superpowers (verification, systematic debugging), Forrest Chang's Karpathy guidelines (surgical changes), ponytail (stdlib before dependencies), i-have-adhd (answer first), Matt Pocock's skills (grilling, handoffs), and Addy Osmani's agent-skills. The eval format is Anthropic's claude plugin eval, and the skill format is the open Agent Skills spec.

Privacy

The plugin has no code and collects nothing; skill-vet runs locally with no telemetry. Details in PRIVACY.md.

License

MIT