vibe-check

AI-built apps usually work on the happy path. What gets skipped is everything nobody asked for: the admin key that ends up in the browser bundle, the webhook nobody verifies, rate limits on some routes but not others, backups no one has ever restored, a privacy policy that doesn't match what the code collects.

vibe-check is an agent skill that takes your project through 84 checks for exactly those things and writes the result to VIBE_CHECK.md:

recon -> 84 checks -> evidence for each -> VIBE_CHECK.md -> fixes, when you ask

It reads your code and doesn't change it. Item numbers never change, so you can rerun it before every release and compare results ("#29 is still open").

What it checks

The full list, with where to look and what "good" means for each item, is in checklist.md.

Install

npx skills add barkinsarikartal/vibe-check -g -a claude-code

-g installs it for all your projects; leave it out to add it to the current project only. Leave out -a claude-code to choose other agents (Codex, Cursor, OpenCode and more).

git clone https://github.com/barkinsarikartal/vibe-check.git
cp -r vibe-check/skills/vibe-check ~/.claude/skills/

For a single project, copy it into that project's .claude/skills/ instead.

Requires Python 3 for the two helper scripts (standard library only). Git is optional; without it, the history scan is skipped.

Use

Open your project in your agent and ask in plain words:

The chat reply stays short: the verdict, the critical gaps and counts for the rest. Everything else is in VIBE_CHECK.md.

What you get

Each item gets one of five verdicts:

The report has four parts: gaps ranked by severity, owner checks, recommended skips, and the full 84-row table. Below is an excerpt from a test run on a deliberately broken demo app (Next.js, Supabase, Stripe, OpenAI):

**Tally:** 84 items · 15 OK · 55 gaps (16 critical, 23 high, 11 medium, 5 low) · 5 owner checks · 7 N/A · 2 skip

### G4. Pro can be had for free: CRITICAL (items #15, #31, #32)
**What:** There are four separate holes:
- **The unlock is client-only.** `/pro` saves `isPro` in localStorage whenever the URL
  contains `?success` (`app/pro/page.tsx:12-13`). Visiting `/pro?success=1` unlocks Pro.
- **The client sets the price.** The server passes `priceAmount` to Stripe as
  `unit_amount` (`app/api/checkout/route.ts:5,15`).
- **The webhook is unsigned.** No `stripe.webhooks.constructEvent`; anyone can POST a
  fake `checkout.session.completed` (`app/api/stripe/webhook/route.ts:5`).
...

| #  | Item                     | Status | Evidence / note                                          |
|----|--------------------------|--------|----------------------------------------------------------|
| 17 | SQL injection            | OK     | All DB access goes through the supabase-js query builder; no raw SQL or `.rpc` |
| 21 | Insecure file uploads    | GAP (HIGH) | G8: only `accept="image/*"` on the client (`app/upload/page.tsx:29`) |
| 44 | No backup or restore plan | GAP (HIGH) | G13: no backup or restore notes in the repo |
| 83 | Trademarking the brand   | OWNER  | Name clearance can't be seen from the repo; search steps in section 2 |

[!NOTE] Legal and business items (privacy policy, terms, DMCA, trademark, patent) come with concrete gaps and an honest opinion. They are not legal advice.

Safety

• The audit is read-only: no code changes, no commits.
• Live checks are a few passive GET requests to your own deployed site (headers, robots.txt, /.env, source maps). Nothing active runs unless you ask: no scanners, fuzzing or login attempts.
• Secret values never appear in the report. verify_report.py fails the report if one slips in.

Platform support

Test results

The skill was tested on two deliberately broken demo apps (Next.js + Supabase, FastAPI + React) with 38 planted problems, running the same prompts with and without it:

Without the skill, the agent still finds the obvious bugs. What it skips is the rest of the list: backups, monitoring, legal pages, trademark, load tests, rollback. That's the part vibe-check adds.

The demo apps live in evals/fixtures/. Fake keys are generated when a fixture is built, so nothing secret-shaped is stored in this repo.

mkdir -p /tmp/vc/fakes

# 1. Build a test app
python3 evals/build_fixture.py splitly /tmp/vc/splitly-full-audit-with_skill \
  > /tmp/vc/fakes/splitly-full-audit-with_skill.fakes.json

# 2. Run your agent on /tmp/vc/splitly-full-audit-with_skill with the prompt from
#    evals/evals.json. Save its VIBE_CHECK.md and final_reply.md to:
#    workspace/iteration-1/eval-1-splitly-full-audit/with_skill/run-1/outputs/

# 3. Score it
python3 evals/grade.py workspace/iteration-1 --fakes-dir /tmp/vc/fakes

workspace/ is in .gitignore.

Repo layout

skills/vibe-check/
├── SKILL.md                  workflow, verdict rules, report template
├── references/checklist.md   the 84 items: where to look, what "good" means, usual severity
└── scripts/
    ├── recon.py              stack fingerprint + redacted secret scan of files and git history
    └── verify_report.py      checks the report covers every item with evidence; prints the tally
assets/banner.svg             the README banner
evals/
├── fixtures/                 two broken demo apps used for testing
├── build_fixture.py          turns a fixture into a git repo with generated fake keys
├── evals.json                test prompts and pass/fail checks
└── grade.py                  scores a test run

Update and uninstall

npx skills update vibe-check
npx skills remove vibe-check

If you installed manually, delete ~/.claude/skills/vibe-check (or the copy in your project's .claude/skills/).

Issues and pull requests are welcome, especially new checks and better detection for stacks the checklist covers thinly.

• Item numbers are permanent. Add new items at the end (85, 86, …) and never renumber, because reports refer to items by number.
• Run the evals before and after you change SKILL.md or checklist.md, and include the before/after numbers in the PR.
• Keep the scripts standard-library Python so the skill installs with no setup.

License

MIT