{"id":"053de792-81ba-47cd-89c4-30e211e1b121","slug":"ai-writing-audit","kind":"skill","name":"ai-writing-audit","summary":"Two-layer audit for text that reads as machine-written: surface tells plus discourse tells, layered by text type.","owner":"thebgtagency","repoUrl":"https://github.com/thebgtagency/ai-writing-audit","license":"CC-BY-SA-4.0","licenseTier":"weak-copyleft","trust":"community","stars":0,"match":null,"fewStars":true,"format":"skill","createdAt":"2026-08-31T14:55:07.000Z","addedAt":"2026-10-21T10:24:55.054Z","logoUrl":"https://avatars.githubusercontent.com/u/259112911?u=0590924b40d3ee821ae66b7d5f4e0772d9e9acfb&v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["writing"],"tags":["ai-detection","ai-writing","copywriting","llm","writing-tools"],"runtime":"binary","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-23T12:32:13.102Z","sample":false,"url":"https://conduits.dev/tools/ai-writing-audit","language":"","transports":["stdio"],"install":{},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# ai-writing-audit\n\nA skill for auditing and repairing text that reads as machine written. One skill, every\nformat: a two line direct message, an automated message sequence, an objection reply, a\ncaption, a carousel slide, a spoken script, a landing page, an essay, an email. Formats it\nhas never heard of are handled by deriving a profile for them, which is the section below on\nadapting it to your own work.\n\n## What makes it different from a word list\n\nMost tools in this space are a list of banned words plus a dash ban. That catches an\nunedited first draft and nothing else, because those signatures are the part that gets\nremoved by the next model release or by a single editing pass.\n\nThis skill runs two layers.\n\nThe **surface layer** is the word, punctuation, and formatting catalogue, built from\nWikipedia's \"Signs of AI writing\" field guide.\n\nThe **discourse layer** asks structural questions instead: does the text explain its own\npoint, does everything pull one way, does it resolve by asking the reader to decide, is\nanything named. It comes from StoryScope (COLM 2026), which measured 61,608 stories and\nseparated human from machine text at 93.2 macro-F1 using narrative structure alone, with\nno style signals at all. That paper also tested what happens when you clean the surface:\nrunning a rewriter over cliche, redundant exposition, and purple prose moved its detection\nby 1.6 points. The structure survived the polish.\n\nBoth layers run behind a **false positive gate**, because the sources spend a large part of\ntheir text warning against over-reading, and a tool that finds seven problems in every text\nis not a detector.\n\n## What it will not tell you\n\nWhether a machine wrote it. That question has a bad evidence base: one 2025 study puts human\naccuracy at chance level, another at 57 percent, and a 2025 preprint puts heavy model users\nat about 90 percent, which is one false accusation in every ten. Detector tools are recorded\nin the same source as having non-trivial error rates and as breaking under paraphrase.\n\nSo the output is a reading verdict, not an authorship verdict, and the skill says so on\nevery run.\n\n## Watch it work\n\nA client direct message, audited and rewritten. 26 seconds.\n\n![Watch it work](docs/demo.gif)\n\n## An example run\n\nInput, 57 words:\n\n> Hey Marcus! Hope you're doing well. I've been following your gym content for a while and\n> I have to say, your commitment to helping people transform their lives is truly\n> inspiring. I work with fitness coaches to streamline their client onboarding and unlock\n> more consistent revenue. Would you be open to a quick chat this week to explore some\n> ideas?\n\nVerdict: reads machine, seven weighted findings, and the gate held back three things that\nlook like tells and are not. Repair:\n\n> Marcus, your intake form asks for goals before it asks for a phone number, so people\n> bounce at question four. I moved that question to the end for two coaches this spring and\n> both stopped losing the form. Want the two lines I changed?\n\nWhat moved: the compliment is replaced by something only Marcus's page could have prompted,\nthe claim carries a number and a timeframe, and the ask is smaller than the sale. Thirteen\nmore runs, including three human-written controls that must come back clean, are in\n`TESTS.md`.\n\n## Install\n\n```\ngit clone https://github.com/thebgtagency/ai-writing-audit ~/.claude/skills/ai-writing-audit\n```\n\nOr copy the folder into your skills directory. Works with any AI assistant, not only\nClaude: the skill is plain markdown, so you can also hand `SKILL.md` and the `references/`\nfolder to any model or agent as its instructions. Then ask for an audit on a draft. The\nskill is for text that already exists. It is not for writing a first draft, and\n`references/conflicts.md` item 14 explains why composing against a tell list makes text\nworse rather than better.\n\n## Adapting it to your own formats\n\nHand `SKILL.md` and `references/` to any assistant and tell it what you actually write:\n\n> Adapt this skill for our support replies. Run \"Adapting to a new format\" in SKILL.md, ask\n> me the six questions, and print the silence map.\n\nWhat happens then is bounded on purpose. A format does not get its own rules. It gets its\nown answer to which of the fixed rules can fire in it, and why the rest cannot.\n\n**Fixed, and a derived profile cannot touch any of it:** the ten discourse checks, the\nsurface catalogue and its weights, the four gate questions, the density bands, the seven\nthemes and the bar for a verdict.\n\n**Adapted:** the length band, which layers run, which catalogue entries are hard flags in\nyour format, which patterns are native to it, the pass bar, and the silence map.\n\nThe six questions each change exactly one check. Is the default grammatical person third.\nDid the reader ask for this text. Is the structure itself the product. Could the writer have\nknown anything specific here. Is any feeling in scope. Is the order fixed by convention.\nAn answer that is not in your description is asked for, not guessed.\n\nTwo guards stop an assistant from improvising a profile that merely looks derived. Every\nhard flag has to name its source in the fixed catalogue, so a profile cannot introduce a\ntell by asserting one. And every derived profile prints a silence map covering all ten\nchecks, so a check that cannot fire in your format is reported rather than quietly dropped.\nA silent check with no stated reason is inadmissible.\n\n`references/conflicts.md` item 17 records the tension this resolves and what it costs.\n\n## Files\n\n```\nSKILL.md                      the audit loop, the modes, the output contract\nreferences/surface-tells.md   word, punctuation, and formatting catalogue\nreferences/discourse-tells.md the structural layer, with the measured rates\nreferences/false-positives.md what not to flag, and confidence by format\nreferences/formats.md         profiles for ten formats, plus how to derive more\nreferences/conflicts.md       seventeen disagreements between the sources, and the rulings\nTESTS.md                      sixteen runs across ten formats, including three controls\n```\n\n## What it has actually been tested on\n\nTen formats have a recorded run in `TESTS.md`: direct message, automated sequence,\nobjection reply, caption, carousel, spoken script, long page, email, support reply, product\ndescription. The last two are derived profiles, built by the procedure below rather than\nwritten by hand, which is how the procedure is tested rather than only described.\n\nThree controls, human-written, come back with zero findings. Two of them are rough. The\nthird is tidy and carries three machine-leaning values on purpose, because a tool that only\nclears messy writing has learned the wrong lesson.\n\nFormats without a recorded run are covered by the four-question derivation procedure at the\nend of `references/formats.md`, and the skill does not claim measured coverage of them.\n\n## Honest limits\n\nNeither source measured the formats most text actually takes. The Wikipedia guide was built\non encyclopedia articles and says outright that some of its signs may not apply outside that\ncontext. The fiction study was built on roughly 5,000 word stories and says that shorter\ntexts cannot support its features. Everything between those two, meaning messages, captions,\nslides, ads, emails, and scripts, is covered here by argued transfer. Each transfer states\nthe reasoning that carries it so the reasoning can be checked rather than trusted, and the\nparts that do not transfer are listed as fiction-only and are not applied.\n\nThe vocabulary lists are English. They do not survive translation, and the skill reports the\nvocabulary layer as not run on text in another language rather than translating the list.\n\nEvery word list here ages. Human writing and speech are measurably drifting toward model\noutput, with a 2024 study finding significant influence in conversational podcasts. The\nstructural layer ages more slowly, which is the argument for keeping it even though it is\nthe harder pass to run.\n\n## Sources\n\n- Wikipedia, \"Signs of AI writing\", maintained by WikiProject AI Cleanup. CC BY-SA 4.0.\n- Russell, Rajendhran, Pham, Iyyer, Wieting. \"StoryScope: Investigating idiosyncrasies in AI\n  fiction.\" COLM 2026.\n\n## Licence\n\nCC BY-SA 4.0. The skill incorporates and adapts material from a CC BY-SA 4.0 source, so the\nsame terms carry through. See `LICENSE`.\n"}