skill
judgment-or-code
Agent skill that decides whether an output belongs to deterministic code or to an LLM judge with a written rubric, before the blocklist gets written.
Set up
Skills are installed by placing their folder where your agent loads skills.
The download comes from github.com/farhan1188/judgment-or-code itself, not from our server, so what you get is exactly what its author published.
Install instructions come from the README
README
synced Oct 23, 2026judgment-or-code
A skill that stops your coding agent from writing a regex for a question that needs a judge, and an LLM call for a question that needs ==.
npx skills add farhan1188/judgment-or-code
This is an Agent Skill: one SKILL.md your agent loads when the work
matches, not a library that runs in your app. It changes what the agent builds, not what your program
does at runtime.
What changes
| You ask for | What you usually get | What you get with this loaded |
|---|---|---|
| "Don't let the copy sound AI-written" | A banned-words list that grows every week | A judge with a rubric and your real rejects as examples. The list survives only for literal strings you have actually forbidden |
| "Mark the verified records" | verified = true when the scraped name matches | confirmed only on independent corroboration. unconfirmed and conflicting become real states that cannot be rounded up downstream |
| "Score the leads and cut below 0.7" | Code thresholds the model's own confidence number | A category plus a quoted reason. The threshold survives only if that score is actually calibrated |
| "Check the totals line up" | An LLM call per row | Arithmetic |
| "Have a human review the output" | A 4,000-row queue nobody reads | A queue sized to human attention: everything flagged, plus a stated sample of the approved |
The problem it solves
Tell a coding agent your generated copy must not sound like AI wrote it, and watch what happens. It
writes a banned-words list. delve, tapestry, elevate, in today's fast-paced world. Testable,
fast, and the wrong shape for the problem, because "sounds like AI wrote it" is not a set of words.
The list catches the twelve phrases you named, misses the hundred you did not, and fires on the one
sentence where delve was the right verb. Every miss produces another entry.
A list that keeps growing to chase an open boundary is judgment mechanised, and it never converges.
The reflex runs the other way too: a model call where three lines of code compute the one right
answer. Asking an LLM whether two order IDs match buys you a bill, a latency budget, and a component
that is wrong sometimes, in exchange for a comparison == was already doing.
The third version does the real damage, because it looks like it worked. A pipeline generates 203 personalised outreach lines. Every one passes the check "mentions the company name". Every one is worthless, because the model was told to be specific about companies whose input records differed only in trivia. The check was green and the output was garbage. Passing a mechanical check never certifies that creative output is good.
None of this is the agent being stupid. It is the agent picking a tool by habit and then writing the acceptance rule backwards from the tool it already picked. Phrase the goal as "reject copy containing banned words" and a regex is obviously right. Phrase it the way you meant it, "reject copy that reads like a machine wrote it", and it obviously is not.
What the skill does
It forces the decision to happen in the right order: write the acceptance rule down the way you would state the goal, then pick the tool that fits it. Then it routes each output to an owner.
| Deterministic code owns | An LLM judge owns |
|---|---|
| Exact match, schema, enum, arithmetic, state machines | Open meaning boundaries: vacuous, off-topic, wrong entity, off-brand |
| Dedup, spend caps, send and publish gates | "Is this any good?" on anything generated |
| A finite table the business maintains as policy | Any category where new members keep appearing |
The judge side always needs a written rubric plus real failing examples, because a rubric with no failures attached grades everything a B. And before reaching for a judge at all, the skill tries the cheaper exits: normalise the input, fix the source data, drop the unreliable feature. A model is not the first answer to a data problem.
What it triggers on
You mostly do not invoke this. The description is written to fire on the work itself: "filter out the bad ones", "add validation for this field", "flag anything that looks wrong". It fires the moment before the wrong choice gets made. Specifically:
a keyword blocklist · a name or entity matcher · a regex validator · a scoring threshold · a
classification or validation step · a confirmed / verified flag · a quality gate on generated
output · a dedup or filter rule · a human review queue · adding or removing an LLM call from a
pipeline · a mechanical rule that keeps needing another exception
Type /judgment-or-code to force it, or to audit code you already have. As an audit it reads as:
find the blocklist that keeps growing, the confirmed flag resting on one source, the threshold
sitting on an uncalibrated score.
The nine rules
Each one is generalised from a measured failure. Full text with the reasoning:
skills/judgment-or-code/SKILL.md.
- Code owns the checkable; checkable never certifies quality.
- An LLM judge owns every open meaning boundary, with a rubric plus real failing examples.
- Grounding before model tier. If two subjects get inputs differing only in trivia, no prompt and no model upgrade saves the output. Fix what is fetched first.
- Judge creative batches against sampled siblings. Per-row judging is a blind spot: every row passes, and the batch is visibly machine-made.
- Real-world claims need evidence that could disagree. N fields scraped from the claim's own subject is one signal, not corroboration.
- Verdicts are categorical, with evidence attached. Code thresholding a model's own uncalibrated score is judgment laundering.
- Hard invariants stay mechanical. Spend, schemas, dedup, send gates: code, enforced once.
- A human gate is sized to human attention. Bulk approve is not review.
- Spend on judgment where wrongness is expensive. Outside-facing or money-gating output gets judged whatever it costs; an internal low-stakes field does not buy a jury.
Tripwires, any one of which means stop and re-route: an open-category word list that needed another entry · a model told to "be specific" on class-shared input · "confirmed" from single-provenance evidence · an uncalibrated self-score thresholded by code · an LLM where code already computes the answer · a review step that has quietly become bulk approve.
Install
With the skills CLI:
npx skills add farhan1188/judgment-or-code
As a Claude Code plugin. Run inside Claude Code:
/plugin marketplace add farhan1188/judgment-or-code
/plugin install judgment-or-code@judgment-or-code
By hand:
git clone https://github.com/farhan1188/judgment-or-code
cp -r judgment-or-code/skills/judgment-or-code ~/.claude/skills/
What it will not do
- It is instructions, not enforcement. A skill is text the model loads and follows. It shifts the default and gives you the vocabulary to push back with; it cannot stop a model that ignores it. Nothing installed here executes at runtime.
- It will not tell you a model is always better. Rules 1, 7 and 9 exist to stop LLM calls being added where code computes the one right answer.
- It will not write your rubric. It tells you a rubric needs real failing examples attached. You supply the failures.
- It does not measure anything. This is a routing decision made before you build, not a benchmark.
Requirements
The SKILL.md uses only Agent Skills spec fields, so nothing in it is
Claude-specific. It is written for Claude Code and works as a plugin there; the
skills.sh CLI also lists Cursor, Codex, Copilot, Windsurf and others as install
targets, each in that tool's own format.
License
MIT. See LICENSE.