AI Content Detection: What It Catches, What It Misses, and Why It Matters
Detectors are wrong in both directions, and that is not a bug anyone is about to fix. The question worth asking is not who wrote it, but who checked it and who is answerable for it.
On This Page
A draft arrives from a writer you have worked with for two years. Somebody runs it through a detector out of curiosity. The tool returns a confident percentage, and by lunchtime there is a meeting. The writer says they wrote it. The tool says otherwise. Nobody in the room can explain what the number was calculated from, and the number is the only evidence anybody has brought.
That meeting is a mistake from the first minute, and not because the writer is necessarily innocent. It is a mistake because AI content detection does not do what people believe it does. It is unreliable when it accuses and unreliable when it clears, and the two failures are structural rather than temporary. No version bump resolves them.
This is not an argument for pretending the problem away. Machine-drafted copy published without care does real damage, to readers and to the organisation that signed it. But every part of that damage is visible by reading the work, and none of it requires knowing which keys produced the first draft. So the policy we recommend is built on quality and accountability, and it holds regardless of what tools anybody used.
What an AI content detector is actually measuring
A detector does not recognise machine writing the way a bank recognises a forged signature. It holds no record of what any model produced. What it has is a statistical view of how ordinary a passage looks: how predictable each word is given the words before it, and how much that predictability varies across the piece.
The statistical signal
Language models are trained to produce likely text, so their output tends to be smooth. Sentence lengths cluster. Word choices sit near the middle of what is plausible. Human writing, on average, is lumpier: an odd word, a fragment, a sentence that runs too long because the writer was thinking while typing. Detectors score that lumpiness. That is essentially the whole mechanism, dressed in different interfaces.
Which means the detector is measuring a property of the prose, not a fact about its origin. Smooth prose scores as machine-like whoever produced it. Lumpy prose scores as human whoever produced it. Those are not the same claim, and the gap between them is where every bad outcome lives.
The number is not a probability
A result presented as “ninety-eight per cent” invites a reading it cannot support. It is not the chance that a machine wrote the text. It is a score on an internal scale, calibrated against whatever the vendor sampled, expressed in a format borrowed from statistics because that format looks authoritative. Two tools returning similar numbers are not corroborating each other; they are agreeing about style, which they would do for an unusually tidy human writer as readily as for a model.
Wrong in both directions, and not fixable
Most coverage of detector accuracy treats it as a race that better tools will eventually win. We do not think that is what is happening. The two error types have different causes, and both get worse as models improve and as writers adapt.
Who the false positives land on
False positives are not randomly distributed, which is what makes them a fairness problem rather than a nuisance. They concentrate on writers with a smaller working vocabulary and a more regular sentence rhythm, which describes many people writing in a second or third language. They concentrate on technical documentation, where the whole point is to say the same thing the same way every time. They concentrate on legal and compliance copy, on release notes, on anything template-driven, and on the plain, careful house style that good editing produces.
Read that list again and notice what it has in common. The writers most likely to be accused are the ones who followed the brief, wrote clearly and stayed on-brand. A tool that punishes disciplined writing is not a quality control. It is a style filter with an accusation attached.
Why the false negatives are trivial to produce
On the other side, defeating a detector takes almost no effort and no bad intent. Ask the model for a different register. Rewrite the openings. Break three sentences and join two. Run the text through a paraphraser. Or simply edit it properly, which is what a responsible person does anyway, and which shifts the statistical fingerprint as a side effect. Anyone determined to hide the provenance of a text will hide it. The people who get caught are, disproportionately, the people who were not hiding anything.
| What the result gets read as | What is defensible | What is not |
|---|---|---|
| A high score | The prose is statistically unsurprising to this model | That a machine produced it |
| A clean, low score | Nothing crossed this model’s threshold | That no model was involved |
| Two tools agreeing | Two similar methods reached a similar view | Independent corroboration |
| A score on a short passage | Very little; short inputs are unstable | A verdict on the whole document |
| A score on translated text | That the prose reads as regular | Anything at all about the author |
| A score inside a review | A prompt to read the work more closely | Grounds for a decision on its own |
Provenance is the wrong question
Even if detection worked perfectly, it would answer a question that has stopped being answerable. Consider the ordinary week of a working writer. A model suggests an outline, which they discard. A model cleans up an interview transcript they recorded themselves. A grammar tool rewrites a clause. A translation pass moves a paragraph from one language to another and back. A colleague asks a model to shorten a section, then rewrites the result because it lost the argument.
Which of those produced “AI content”? There is no line there, only a gradient, and any policy that depends on locating the line will be argued about forever by people who all believe they are being honest. Meanwhile the failures that actually hurt you sit somewhere else entirely.
Nobody is harmed by how a sentence was produced. They are harmed by a sentence that is wrong, and that nobody was answerable for.
So we replace the provenance question with an accountability question, and it turns out to be the one that was always doing the work. Who is named against this piece? Who read it against the sources? Who would be asked to explain it if a reader acted on it and it was wrong? Those questions have answers, the answers can be recorded, and they apply identically to a human first draft, a machine first draft and a piece assembled from both. They are also the questions we ask before wiring a model into anything customer-facing, which is a related discipline covered in our note on building an evaluation set first.
What actually goes wrong, and how you find it
Machine-drafted copy published without review fails in recognisable ways. Every one of them is findable by an editor who knows the subject, and none of them needs a detector.
- Confident specifics that are wrong. A version number, a regulation, a date, an API parameter, a price band. The prose gives no signal of uncertainty, which is precisely why it slips through.
- Citations that do not resolve. A plausible title, a plausible publisher, a URL that 404s or points at something unrelated. Opening every link catches this in minutes.
- Statistics with no owner. A percentage that appears without a source, survives editing because it sounds specific, and cannot be traced back to anything when a reader asks.
- No position. Both sides presented evenly, no recommendation, no condition under which the advice reverses. Readable, useless, and instantly forgettable.
- No first-hand detail. Nothing that could only be known by someone who has done the work: the failure mode nobody documents, the step everyone skips, the argument the team actually had.
- Stale ground truth. Advice that was correct two product versions ago, phrased as though it were current, because the model’s picture of the world stopped at a point it will not tell you.
Notice that a competent human writing carelessly produces the same list. That is the argument in miniature. These are quality defects, they have always existed, and the correct response is a review step designed to catch them rather than a score designed to assign blame. We have written about designing that step in human in the loop, and the same reasoning drives how we scope content and organic growth work: the editor is the control, not the tooling.
Where a detector still earns its place
The concession, and it is a real one. Detectors are not useless. They are useless as evidence, which is a different claim, and there are three narrow jobs where we would still run one.
As triage at volume: if you receive hundreds of submissions and can only read a fraction closely, a score is one input among several for deciding reading order. As an internal smoke alarm: a team publishing at pace can sample its own output and notice that everything has started to sound the same, which is a genuine problem and one these tools are fairly good at spotting. And as a prompt to look harder at a specific passage, in the same spirit as a spellchecker flagging a word you then decide about yourself.
If a result is going to affect somebody’s payment, reputation or job, it cannot be the basis of the decision. Ask for the working instead: notes, drafts, sources, the recording, the version history. A writer who did the work can produce it in an afternoon. A tool that cannot explain its reasoning, cannot be cross-examined and cannot be audited has no business standing in for that conversation.
An editorial policy that works either way
Here is the policy we would put in place for any team publishing regularly. It says almost nothing about tools, which is the point: it produces the same standard of work whether a model was involved or not, and it asks nobody to police an unanswerable question.
- Name a person against every piece. Not a department, not a byline like “the team”. One human who is answerable for the claims in it and who would be asked first if a reader relied on it and was misled.
- Require sources that were opened. Every external claim carries a link the reviewer actually followed. If the link does not resolve, or does not say what the sentence says, the sentence goes.
- Ban unattributed numbers outright. No percentage, benchmark, currency figure or survey result without a citation. This single rule removes most of the damage careless drafting causes, and it improves human drafting too.
- Demand something only you could write. A failure you have seen, a sequence you actually run, a decision rule with a condition attached. If the piece could have been written by someone who has never done the work, it should not go out under your name.
- Disclose method, not tools. Readers care whether a person stands behind the content and how it was checked. A list of software is theatre; a clear statement of who reviewed it, and against what, is information.
- Keep a record of the approval. Who approved this wording, and when. Not for compliance drama, but because in a year somebody will ask and a confident guess is worthless.
What to write down
The record matters more than teams expect, and it is far easier to get for free than to retrofit. When content lives as files under version control, the approval record is a side effect of publishing: an author, a timestamp, a diff and usually a reason, with no reporting to build. A workspace like Acrosite works that way, generating the files, committing them to GitHub and triggering the configured deployment, so “who approved this sentence” is a two-command question rather than a search through chat history. Our published position on how these tools are used sits on the responsible AI page, and it is deliberately about accountability rather than abstinence.
The review step is the control
A review that consists of reading for tone catches nothing on the list above. A review with a defined pass, where the reviewer checks specific classes of error in a specific order, catches most of it. That distinction is the whole subject of our parent company’s piece on the reviewer’s pass, and it applies with more force to documentation, where readers act on the words immediately. That case is argued in should AI write your product documentation.
What search engines have actually said
Publishers keep asking whether machine-assisted content is penalised, usually hoping for a rule they can comply with mechanically. The public guidance has been consistent, and it is worth reading in the original rather than in summary: the concern is content produced primarily to manipulate rankings rather than to help people, and production method is not the test. Automation used to churn out pages at scale for search engines is the described problem. Automation used in the course of producing something genuinely useful is not.
That is a quality standard rather than a provenance standard, which is the conclusion this piece reaches from a different direction. It also carries a practical edge. Guidance published for creators repeatedly emphasises evidence of first-hand experience and clarity about who stands behind a page. Those are things you can add to a draft this afternoon. A detector score is not.
The commercial consequence deserves stating plainly, because it changes what a content budget should buy. Volume produced cheaply has never been scarce and is now abundant. What is scarce is a page that answers a specific question, carries a position, and is signed by somebody who will defend it. That is also what makes documentation work as a growth channel, and it is close to the only durable thing left to compete on.
The question to ask instead
When somebody hands you a draft and asks whether a machine wrote it, the honest reply is that you cannot know, they cannot know, and the tool claiming otherwise is guessing from sentence rhythm. Then ask the three questions that are answerable.
Is every checkable claim in it actually checkable, and did somebody check it? Does it contain anything that could only come from doing the work? And whose name goes on it when a reader acts on the advice and it turns out to be wrong? A draft that passes those three is fine no matter how it started. A draft that fails them was never rescued by having been typed by hand.
Take the last ten pieces you published. Open every outbound link, trace every number to a source, and write down who approved each one. Most teams find their real problem inside the first hour, and it is almost never authorship. It is that nobody was named, and nothing was checked twice.
One last caution about tooling in general. A detector bolted into a publishing workflow becomes another system somebody has to maintain, explain and eventually defend, which is the pattern described in the standing cost of an automation nobody owns. If you are working out where models genuinely belong in your operation, and where they are being added simply because they are available, that is the conversation we would rather have than an argument about a score. Our AI development and integration work starts there, and you can send us the specifics of what you publish and who reviews it now.
And if the question behind the question is how long any of this takes to show up in results, the sober version is set out in what ninety days can honestly show.
Common questions.
How accurate are AI content detectors?
Accurate enough to be interesting and nowhere near accurate enough to act on alone. Detectors score how predictable and how uniform a passage is, so unusually tidy human writing scores as machine-like and lightly edited machine writing scores as human. Accuracy also collapses on short passages, translated text and technical documentation. Treat any result as a prompt to read the work, never as a finding.
Why was my writing flagged as AI when I wrote it myself?
Because the tool rewards irregularity, and your writing was probably regular. Consistent sentence length, a controlled vocabulary, a house style applied carefully and plain technical phrasing all push a score upward. Writers working in a second or third language are affected more often for the same reason. Ask whoever flagged it to look at your drafts, notes and version history instead, which is evidence a score can never be.
Does Google penalise AI-generated content?
Public guidance treats production method as the wrong test. The stated concern is content produced primarily to manipulate search rankings rather than to help people, and that concern applies equally to mass-produced human writing. Content created with automation can rank when it is genuinely useful, shows first-hand experience and has somebody accountable for it. Content created to fill a page with keywords does not, however it was produced.
Should we disclose that we use AI in our content process?
Disclose your method rather than your software list. Readers want to know that a named person stands behind the claims and that the work was checked against sources, which is information they can use. A generic notice that tools were involved tells them nothing and ages badly. If a specific piece relied heavily on generation with little review, that is worth saying, and probably worth fixing first.
Can you tell if text is AI-generated just by reading it?
Not reliably by style, but often by substance. The signals worth trusting are checkable ones: citations that do not resolve, statistics with no source, specifics that are subtly wrong, advice that stops short of any real recommendation, and the total absence of detail that could only come from doing the work. An editor who knows the subject finds these quickly. Style alone proves nothing.
What should an editorial policy say about AI?
It should say who is accountable, what must be verified, and what is never allowed. Name one person against every published piece. Require that every external claim carries a source somebody actually opened. Prohibit unattributed statistics entirely. Require at least one element that could only come from first-hand work. Keep a record of who approved the final wording. None of those rules depends on knowing which tools were used.
Facing this in your
own business?
Tell us where you’re headed — we’ll map the shortest honest route.