Skip to content
Conduit

skill

harness-evolver

Automated harness evolution for AI agents. A Claude Code plugin that iteratively optimizes system prompts, routing, retrieval, and orchestration code using full-trace counterfactual diagnosis. Based o

raphaelchristi/harness-evolvercommunitylistedunreviewed
Use skill

Set up

Skills are installed by placing their folder where your agent loads skills.

Download ZIP from GitHub Open the repository

The download comes from github.com/raphaelchristi/harness-evolver itself, not from our server, so what you get is exactly what its author published.

Install instructions come from the README

Skills do not use an MCP config. Follow the steps in the README below.

README

synced Oct 17, 2026

Harness Evolver

Point at any LLM agent codebase. Harness Evolver will autonomously improve it — prompts, routing, tools, architecture — using multi-agent evolution with LangSmith as the evaluation backend.

Install

Claude Code Plugin (recommended)

/plugin marketplace add raphaelchristi/harness-evolver-marketplace
/plugin install harness-evolver

npx (first-time setup or non-Claude Code runtimes)

npx harness-evolver@latest

Works with Claude Code, Cursor, Codex, and Windsurf.

Quick Start

cd my-llm-project
export LANGSMITH_API_KEY="lsv2_pt_..."
claude

/harness:setup      # explores project, configures LangSmith
/harness:health     # check dataset quality (auto-corrects issues)
/harness:evolve     # runs the optimization loop
/harness:status     # check progress (rich ASCII chart)
/harness:deploy     # tag, push, finalize

What It Looks Like

Tested on a RAG agent (Agno framework, Gemini 3.1 Flash Lite, light mode):

xychart-beta
    title "agno-deepknowledge: 0.575 → 1.000 (+74%)"
    x-axis ["base", "v001", "v002", "v003", "v004", "v005", "v006", "v007"]
    y-axis "Correctness" 0 --> 1
    line [0.575, 0.575, 0.950, 0.950, 0.950, 0.950, 0.950, 1.0]
    bar [0.575, 0.333, 0.950, 0.720, 0.875, 0.680, 0.880, 1.0]
IterScoreMerged?What the proposer did
baseline0.575—Original agent — hallucinations, broken tool calls, no retry logic
v0010.333YesAnti-hallucination prompt (100% correct when API responded, but 60% hit rate limits)
v0020.950YesBreakthrough: inlined 17-line KB into prompt, eliminated vector search entirely. 5.7x faster, zero rate limits
v0030.720NoAttempted hybrid retrieval — regressed, rejected by constraint gate
v0040.875NoResponse completeness fix — improved one case but regressed others
v0050.680NoReduced tool calls — broke edge cases, rejected
v0060.880YesEvolution memory insight: combined v001's anti-hallucination with one-shot example from archive
v0071.000YesOne-shot example injection + rubric-aligned responses — perfect on held-out

The line shows best score (only goes up — regressions aren't merged). The bars show each candidate's raw score. 4 merged, 3 rejected by gate checks. Not every iteration improves — that's the point.

How It Works

LangSmith-NativeNo custom scripts. Uses LangSmith Datasets, Experiments, and LLM-as-judge. Everything visible in the LangSmith UI.
Real Code EvolutionProposers modify actual code in isolated git worktrees. Winners merge automatically.
Self-Organizing ProposersTwo-wave spawning, dynamic lenses from failure data, archive branching from losing candidates. Self-abstention when redundant.
Rubric-Based EvaluationLLM-as-judge with justification-before-score, rubrics, few-shot calibration, pairwise comparison.
Smart GatingConstraint gates, efficiency gate (cost/latency pre-merge), regression guards, Pareto selection, holdout enforcement, rate-limit early abort, stagnation detection.

Full feature list

Evolution Loop

/harness:evolve
  |
  +- 1. Preflight  (validate state + dataset health + baseline scoring)
  +- 2. Analyze    (trace insights + failure clusters + strategy synthesis)
  +- 3. Propose    (spawn N proposers in git worktrees, two-wave)
  +- 4. Evaluate   (canary → run target → auto-spawn LLM-as-judge → rate-limit abort)
  +- 5. Select     (held-out comparison → Pareto front → efficiency gate → constraint gate → merge)
  +- 6. Learn      (archive candidates + regression guards + evolution memory)
  +- 7. Gate       (plateau → target check → critic/architect → continue or stop)

Detailed loop with all sub-steps

Agents

AgentRole
ProposerSelf-organizing — investigates a data-driven lens, decides own approach, may abstain
EvaluatorLLM-as-judge — rubric-aware scoring via langsmith-cli, few-shot calibration
ArchitectULTRAPLAN mode — deep topology analysis with Opus model
CriticActive — detects evaluator gaming, implements stricter evaluators
ConsolidatorCross-iteration memory — anchored summarization, garbage collection
TestGenGenerates test inputs with rubrics + adversarial injection

Requirements

  • LangSmith account + LANGSMITH_API_KEY
  • Python 3.10+ · Git · Claude Code (or Cursor/Codex/Windsurf)

Dependencies installed automatically by the plugin hook or npx installer.

LangSmith traces any AI framework: LangChain/LangGraph (auto), OpenAI/Anthropic SDK (wrap_*, 2 lines), CrewAI/AutoGen (OpenTelemetry), any Python (@traceable).

Companion: LangSmith Tracing

For full observability into what each proposer does during evolution (every file read, edit, and commit), install the LangSmith tracing plugin:

/plugin marketplace add langchain-ai/langsmith-claude-code-plugins
/plugin install langsmith-tracing@langsmith-claude-code-plugins

With both plugins installed, the evolution loop traces to LangSmith as a hierarchy: iteration → proposers → tool calls.

References

License

MIT

↑↓ navigate openesc close