policy-reasoning-bench

policy-reasoning-bench

72 hand-built cases testing conceptual reasoning across the three areas Anthropic's Conceptual Reasoning Fellowship names: philosophy (24), economics (24), and law/policy (24).

Problem

Most eval sets test recall: does the model know the answer? The fellowship work is about reasoning — applying a principle to a new situation, tracing an incentive through its behavioral consequences, or classifying a novel AI deployment under a regulatory framework. This bench isolates that: every case asks the model to apply, not recite.

Method

Each case carries five things:

• prompt — a scenario or question (no verbatim textbook item),
• gold answer — a worked model answer,
• grading rubric — 2–3 criteria, each with keywords and a weight (weights sum to 1.0),
• principle — the concept the case tests (e.g., "moral hazard", "EU AI Act: high-risk recruitment AI"),
• reasoning note — one line on why the case tests reasoning rather than recall.

Coverage:

Domain	Cases	What they test
Philosophy	24	Moral frameworks (consequentialism, deontology, virtue ethics), fairness, paternalism, collective responsibility
Economics	24	Incentive analysis: perverse incentives, Goodhart, moral hazard, adverse selection, externalities, mechanism design
Law / policy	24	EU AI Act risk tiers (prohibited / high-risk / limited / minimal) with rationale; NIST AI RMF functions (Govern, Map, Measure, Manage)

Where a classification is genuinely debatable (e.g., emotion recognition at work, GPAI value-chain duties), the gold answer says so and the rubric rewards flagging the uncertainty.

Grading is a stated heuristic (grader.grade): score = Σ weight × keyword hit-rate. Run any model with runner.evaluate:

from policy_reasoning_bench import evaluate, summary

def my_model(prompt: str):
    answer = call_your_model(prompt)   # -> str
    confidence = 0.8                   # self-reported, in [0, 1]
    return answer, confidence

report = evaluate(my_model)
print(summary(report))

The bench also bridges to reasoning-eval-harness via to_harness_cases() for calibration and failure-taxonomy analysis.

Results

Measured by running the repo's own test suite and demo (no invented numbers):

• 72 cases, 24 per domain, all passing schema validation (unique IDs, rubric weights summing to 1.0, non-empty fields).
• Self-consistency: the rubric grader scores every gold answer at 1.0 (enforced by test); an empty answer scores 0.0 on every case.
• Demo run (examples/demo.py, stub model — gold on philosophy, paraphrase on economics, "I don't know" on law): overall accuracy 0.333; philosophy 1.000, economics 0.000, law/policy 0.000 (n=72, threshold 0.70).
• Test suite: 15 unittest cases, all passing (python3 -m unittest discover -s tests).

Limitations

• The grader is a keyword heuristic, not a judge of reasoning. It measures whether an answer engages the rubric's concepts; a fluent but wrong answer can score well, and a correct answer in different words can score poorly. Treat scores as coverage checks, not truth.
• Gold answers are one person's worked answers, not ground truth — philosophy especially admits reasonable disagreement.
• EU AI Act classifications reflect the Act's text as publicly understood; edge cases and implementing guidance evolve, and nothing here is legal advice.
• 72 cases is a probe, not a measurement instrument: do not report domain accuracies as capability claims.

Layout

• src/policy_reasoning_bench/ — package (stdlib only)
  • philosophy.py, economics.py, law_policy.py — the 72 cases
  • schema.py — data model + validation
  • grader.py, runner.py — scoring and run loop
• tests/ — unittest suite
• examples/demo.py — worked demo

License

MIT — see LICENSE.