pantheon

Pantheon

A Claude Code skill that pits frontier AI models against each other on your coding tasks. Each model produces a solution, all judge each other anonymously, and the best implementation wins.

┌───────┐ ┌─────┐ ┌──────┐
│ Opus  │ │ GPT │ │Gemini│    2-4 models solve in parallel
│  4.6  │ │ 5.2 │ │  3   │    (~60-120s, all concurrent)
└───┬───┘ └──┬──┘ └──┬───┘
    │        │       │
    ▼        ▼       ▼
┌──────────────────────────────────┐
│    CROSS-MODEL EVALUATION        │   Each model judges ALL solutions
│    (anonymized, no self-grading) │   anonymously. Self-scores excluded.
└──────────────┬───────────────────┘
               ▼
         Best solution
         + cherry-picks
         + tech spec

How It Works

1. You describe a task — "add bulk actions to the pipeline page"
2. Pantheon gathers context — reads your files, conventions, file tree
3. Your chosen models solve it in parallel — each produces complete, implementable code
4. All models judge all solutions anonymously — scores on precision, accuracy, creativity, simplicity
5. Self-scores are excluded — Opus can't grade Opus, GPT can't grade GPT
6. You review the verdict — score table, approach summaries, full tech spec, or raw code
7. You choose what to implement — winner, runner-up, or cherry-pick the best of each

Install

# Clone into your Claude Code skills directory
git clone https://github.com/zachjxyz/pantheon.git ~/.claude/skills/pantheon

# Install dependencies
cd ~/.claude/skills/pantheon/scripts && npm install

Or if you prefer keeping the repo separate:

# Clone wherever you like
git clone https://github.com/zachjxyz/pantheon.git ~/Projects/pantheon

# Symlink into skills
ln -s ~/Projects/pantheon ~/.claude/skills/pantheon

# Install dependencies
cd ~/Projects/pantheon/scripts && npm install

Setup

On first use, Pantheon walks you through a guided setup wizard right inside Claude Code:

1. Choose a strategy — The Big Three, Double Trouble +1, Flash Round, or Mix and Match
2. Select providers — Anthropic, OpenAI, Google (with models shown as subtext)
3. Confirm your lineup — review and optionally swap models
4. Enter your API key — one Vercel AI Gateway key for all providers

All models route through Vercel AI Gateway — one API key, no per-provider keys needed. Get yours at sdk.vercel.ai/docs/ai-sdk-core/gateway.

Config is saved to ~/.claude/pantheon.json. Reconfigure anytime with /pantheon models.

You can also run setup manually in your terminal:

cd ~/.claude/skills/pantheon/scripts && npx tsx setup.ts

Usage

In any Claude Code session:

> /pantheon
> use the pantheon for this
> compete on this task

Or just mention it naturally — "summon the pantheon to add a search bar to the accounts page."

Commands

Review Modes

After the models compete and cross-evaluate, you can review results in several ways:

Model Catalog

18 models available across 3 providers and 2 tiers. Pick 2-4 during setup.

Frontier (latest flagships)

Flash (fast + cheap)

Custom models can be added during setup — any model available on AI Gateway works.

Unbiased Evaluation

The key insight: no model grades itself.

                   Judges
             Opus   GPT   Gemini
Solutions ┌─────────────────────────┐
  Opus    │  [x]    4.2    3.8    │  avg = 4.0  (2 judges)
  GPT     │  3.5    [x]    4.0    │  avg = 3.75 (2 judges)
  Gemini  │  3.8    3.6    [x]    │  avg = 3.70 (2 judges)
          └─────────────────────────┘

Solutions are anonymized ("Solution 1", "Solution 2", etc.) before judging. After scoring, the orchestrator maps labels back to models and drops self-scores.

Scoring Rubric

Cost

Depends on which models you pick:

Project Structure

pantheon/
├── SKILL.md                     # Claude Code skill definition
├── TECH-SPEC.md                 # Full architecture spec
├── scripts/
│   ├── config.ts                # Config loading/saving (~/.claude/pantheon.json)
│   ├── setup.ts                 # Interactive terminal setup wizard
│   ├── models.ts                # Model management (list/add/remove)
│   ├── solve.ts                 # Parallel dispatch to frontier models
│   ├── evaluate.ts              # Cross-model judging with self-exclusion
│   ├── display.ts               # Output formatting (rationales, spec, code)
│   ├── utils.ts                 # Prompt builders, parser, scorer
│   ├── types.ts                 # Shared TypeScript interfaces
│   ├── providers/
│   │   └── registry.ts          # 18-model catalog + AI Gateway helpers
│   └── package.json             # ai + @ai-sdk/gateway + tsx
└── .gitignore

License

MIT