- A detector does not measure where text came from. It measures how statistically ordinary the writing is, and those two things only correlate.
- In a 2023 evaluation of 14 detection tools, all scored below 80 percent accuracy and only 5 scored above 70 percent.
- Seven detectors tested against essays by non native English writers produced an average false positive rate of 61.3 percent.
- Turnitin marketed a false positive rate under 1 percent, then disclosed roughly 4 percent at sentence level and has not published a revised document level figure.
- The same tool has acknowledged a 15 percent false negative rate, so it misses about one AI passage in six while flagging innocent text.
- Everything that works instead produces evidence rather than a number: drafts, a conversation about the argument, better task design, signed provenance.
Seven detectors were run over essays written by people whose first language is not English. On average, 61.3 percent of those essays came back flagged as machine written. The writers had done nothing but write in a second language, in the plain, careful register that second language writing tends to produce.
That one figure, reported in a July 2023 paper and recorded on the Wikipedia survey of AI content detection research, is the fastest route to understanding what these tools do. They are not reading for origin. They are reading for ordinariness, and some people write in ways that look ordinary to a statistical model for reasons that have nothing to do with a chatbot.
This page is written for whoever is holding a score from an ai content detector and deciding what to do next. A teacher with a number on a screen. A client who ran a copywriter's draft through a checker. A seller whose product description was flagged by a marketplace. The question is always the same: what does this number actually support?
What does an AI detector actually measure?
It measures how predictable the text is. These systems estimate how likely each word is given the words before it, and text where almost every choice is the most probable choice scores as machine written. Language models produce such text because they are trained to pick likely continuations. The problem is that plenty of humans produce it too.
Consider who writes in the most predictable register. Someone writing in a second language, drawing on a smaller active vocabulary. Someone following a template because the task demanded one. A technical writer whose house style forbids flourish. A person who has been taught to write simply. An autistic writer with a precise, consistent style. None of them used a model, and all of them look like one to a detector.
The same mechanism explains the opposite failure. Ask a model to write with unusual vocabulary and sentence shapes and its output stops looking statistically ordinary. The Stanford group behind GPT detectors are biased against non native English writers demonstrated exactly that: simple prompting strategies both reduced the bias against second language writers and effectively bypassed the detectors. One instruction defeats the tool. Nothing a careful student can do will protect them, and one lazy prompt protects whoever wanted to cheat.
How accurate are AI detectors in independent testing?
Below the level anyone would accept for a decision that matters. The most cited independent evaluation, by Weber Wulff and colleagues in 2023, tested 14 detection tools and found that all scored below 80 percent accuracy and only 5 scored above 70 percent. The same work found the tools biased toward calling text human, and found accuracy getting worse when text was paraphrased.
Read that bias carefully, because it cuts both ways and almost every write up reports only half of it. A tool tilted toward the human verdict produces fewer false accusations per document, which is why vendor numbers look reassuring. It also misses more real cases, which is why the same tools cannot be relied on to establish that writing is genuine either.
The vendor disclosure history matters as much as the research. When Turnitin launched its detector in April 2023 it promoted a false positive rate of less than 1 percent. By June, as Inside Higher Ed reported from a statement by chief product officer Annie Chechitelli, the company acknowledged the real rate was higher than it had originally asserted, put the sentence level false positive rate at approximately 4 percent, and did not disclose a revised document level rate. The company attributed the gap to the difference between laboratory testing and what users saw in practice.
Two details from that statement are worth keeping. Documents scoring under 20 percent machine written showed a higher incidence of false positives, which is why the company added a warning asterisk to low scores. And the false positives cluster: 54 percent of wrongly flagged human sentences sat directly next to a machine written sentence, with another 26 percent two sentences away. Mixed documents, which is to say most real documents in 2026, are where the tool is least safe.
What does a 4 percent sentence rate mean for a whole document?
More than it sounds, and this is arithmetic the vendors do not put on the page. Take a 60 sentence piece of writing, which is a short essay or a long product description. At a 4 percent sentence level false positive rate, the expected number of wrongly flagged sentences is around two. Two highlighted sentences in a document is exactly what an accusation looks like.
If those flags fell independently of each other, the chance of at least one appearing in a 60 sentence human document would be above 90 percent. They do not fall independently, by the company's own clustering data, so treat that as an upper bound rather than a prediction. The direction is what matters: a small per sentence error rate becomes a near certainty that something gets marked once a document is long enough, and the person reading the report sees marked sentences rather than a rate.
Here is the piece of the picture that is almost always left out. Turnitin has stated a 15 percent false negative rate. One AI written passage in six is not flagged at all. So the tool accuses the innocent at a measurable rate while missing the guilty at a larger one, which is the worst combination a screening instrument can have.
| Independently measured figure | Value | Source of the number | What it constrains |
|---|---|---|---|
| Accuracy across 14 detection tools, 2023 | All below 80 percent, only 5 above 70 percent | Weber Wulff et al., recorded in the research survey | No tool on the market clears the bar for a decision taken alone |
| False positive rate, non native English essays | 61.3 percent average across 7 detectors | July 2023 paper, recorded in the research survey | A flag on second language writing carries almost no information |
| Turnitin sentence level false positive rate | Approximately 4 percent | Chechitelli statement, June 2023 | Expect around two flagged sentences in a clean 60 sentence document |
| Turnitin marketed document level rate | Under 1 percent, then withdrawn with no replacement figure | Launch marketing, then the June 2023 statement | There is no current published document level rate to rely on |
| Turnitin false negative rate | 15 percent | Company statement, recorded in the research survey | A clean score does not establish that writing is human |
| False positive rate by student group, September 2024 | 20 percent Black, 10 percent Latino, 7 percent White | Common Sense Media, recorded in the research survey | The error is not distributed evenly across people |
Who gets wrongly flagged most often?
People who already have the least room to argue. The September 2024 Common Sense Media figures put the false positive rate at 20 percent for Black students against 10 percent for Latino students and 7 percent for White students. The non native English figure is 61.3 percent. Research cited in the same survey finds detectors prone to flagging neurodivergent writers.
Published anecdote runs the same way. The author Janelle Shane found parts of her own book flagged as machine written in June 2023. A June 2024 Gizmodo report, also recorded in the survey, described freelance writers and journalists losing work because detection software misclassified what they had written by hand.
That last one is the version of this story that reaches small businesses, and it reaches them from both ends. A shop owner runs a commissioned draft through a checker, sees 78 percent, and stops paying an honest writer. A platform runs its own classifier over a listing and restricts it. The number did the deciding in both cases, and in both cases the number was never evidence of origin. If you commission writing, the disclosure question is worth settling in the contract rather than in a checker, which is the ground covered in what to do when a freelancer used AI on work you paid for.
Why is the mixed document the hardest case?
Because it is the normal case now, and it is the one the measurement is weakest on. Almost nobody generates a product page and ships it untouched. People draft with a model and then rewrite, or write by hand and ask for a tighter version of one paragraph. What comes out is interleaved, and interleaving is exactly where the vendor data shows the errors concentrate: 54 percent of wrongly flagged human sentences sat next to a machine written sentence.
There is no sensible way to read a percentage on such a document. A score of 40 percent does not mean 40 percent of the words came from a model. It means the tool assigned that share of sentences to a category, with a known tendency to be wrong about the ones adjacent to real machine text. Apportioning credit between a person and a tool inside one paragraph is not something these systems can do, and no vendor claims otherwise in its documentation, even where the interface implies it.
The practical consequence for anyone running a small operation is that the question has to change. Asking what share of this was written by AI has no answerable form. Asking whether the claims are true, whether the specifics are right and whether somebody competent stands behind the page does have one.
What can a score support, and what can it not rule out?
This is the table worth keeping. Each row is a situation people actually find themselves in, and the columns separate what the number is consistent with from what it cannot exclude.
| The situation | What the score is consistent with | What it cannot rule out | What a fair next step looks like |
|---|---|---|---|
| High score across a whole document | Highly predictable writing, possibly machine written | A plain writing style, a template, a second language writer | Ask for drafts and version history before anything else |
| High score on a few scattered sentences | Normal statistical noise at the stated 4 percent rate | Nothing at all; this is the expected pattern in clean text | Treat as no signal, especially next to any edited passage |
| Any score on text by a non native speaker | Almost nothing, given a 61.3 percent average false positive rate | Entirely human authorship | Set the tool aside and look at process evidence |
| High score on formulaic or technical writing | A house style that suppresses variation | Human authorship under a style guide | Compare against the author's earlier work in the same format |
| Score on text drafted by AI then heavily edited | A mixed document, where false positives cluster hardest | Which sentences came from where | Ask what was used and how, since the tool cannot apportion |
| A zero or very low score | Writing with ordinary variation in it | Machine writing that was prompted to vary, or one case in six missed | Do not treat a clean score as proof of authorship |
What works instead of a detector?
Evidence about process, in rising order of effort. None of these produces a percentage, and that is precisely why they are fairer: they can be examined, contested and explained, which a score cannot.
Drafts and version history. Cheapest and strongest. Writing that happened over time leaves a trail of revisions, abandoned sentences and timestamps. Pasted text arrives whole. This is not conclusive either, and it is far closer to evidence than a classifier output.
A short conversation about the argument. Ask why a particular choice was made, or what was left out. Someone who wrote the thing can discuss the decisions inside it. This takes ten minutes and resolves most honest cases immediately.
Task design that makes generic prose useless. Ask for the specific thing: this week's numbers, the objection a named customer raised, what happened when the method was tried. Generic fluent text cannot answer a question that depends on facts only the author holds.
Provenance signals attached at creation. Credentials recorded when a file is made travel with it and can be verified later, which is the opposite of guessing after the fact. We looked at what that standard does and does not establish in what content credentials actually prove about an image. For text the ecosystem is thinner, and the principle is the same: record origin at the source rather than inferring it from the words.
What should a business do when its own content gets flagged?
Separate the two situations, because they need different responses. Either a platform flagged your content, or you are the one holding a detector score about somebody else's work.
When a platform flags your listing, advert or review response, the practical move is not to argue about detection accuracy. It is to supply process evidence and to understand the rule you are accused of breaking, which is usually about disclosure or about content quality rather than about origin as such. Publishing your own position on AI use helps here, in the way our published AI policy sets out where machine assistance is used and where it is not. A stated policy turns an accusation about a hidden practice into a question about a documented one.
When you are holding the score, the useful question is what you would do if it were wrong, since at these error rates it often is. Reasonable practice looks like this. Never act on a score alone. Never store one as a finding about a person. Ask for drafts first. And if the writing is the deliverable, judge the deliverable: whether the claims are accurate, whether the page answers the customer, whether anything in it was invented. Those checks matter regardless of who typed it, and they are the ones that protect you. Search behaves the same way, which is why our piece on what actually ranks when content is AI assisted comes down on quality and verification rather than on origin detection.
If you must have a policy, what should it say?
Four lines, and they fit on one page. Teams that write this down before an incident handle the incident calmly. Teams that write it during one usually accuse somebody.
First, a detector score is never a finding on its own and is never recorded as one. Second, any concern starts with a request for drafts or a conversation, not with an accusation. Third, the standard being applied is disclosure and quality, which can be stated and met, rather than origin, which cannot be established from the text. Fourth, whoever publishes something is responsible for its accuracy, which puts the obligation in the right place and makes the authorship question much less load bearing than it feels.
That last line does most of the work. A business does not actually need to know whether a paragraph came from a model. It needs the paragraph to be true, to match the product and to not have invented a certification. Those things are checkable. Origin is not.
Will detection get better, or is this unsolvable?
The honest answer is that the words alone are a dead end, and the useful work has moved elsewhere. A detector works by spotting text that sits in a narrow statistical band. As models improve, their output spreads into the same range good human writing occupies, so the signal the detector relies on keeps shrinking. The prompting bypass in the Stanford paper is the practical demonstration.
Two directions remain. Watermarking embeds a detectable pattern at generation time, and the research survey notes a 2023 method that flags watermarked text with high accuracy even after light paraphrasing. Its limit is structural: it only works for text from models that chose to watermark, so it is a tool for cooperating providers rather than a test anyone can apply to arbitrary text. Provenance, the second direction, records origin when content is made. Both share the quality that makes them more honest than detection, which is that they are claims made at the source rather than inferences drawn from the surface.
So the shape of the answer is unlikely to change. Detection of AI text is not a technology that is nearly ready. It is closer to an unsolvable problem, because a sufficiently good model writes sentences drawn from the same distribution as a sufficiently good human, and no amount of measuring the sentences recovers the difference.
One more reason to distrust the percentage: the thing being measured drifts. Detectors are calibrated against the output of models that existed when they were built. Every new model shifts the distribution of ordinary text a little, so a tool that was mediocre in 2023 is being applied in 2026 to writing produced by systems it never saw. Nobody recalibrates the dashboard in a school or a marketplace when a new model ships. The score keeps its two decimal places and quietly means less each year.
Where these figures come from, and when they were read
Every number on this page was taken from a primary statement or a research survey in October 2026, and each is dated where the source dates it. The Turnitin figures come from a June 2023 company statement reported by Inside Higher Ed, and no revised document level rate has been published since. The 14 tool evaluation is from 2023. The non native English figure is from July 2023. The group level false positive rates are from a September 2024 report. The bias finding is from the Stanford paper first posted in April 2023.
These are the best public numbers available, and they are old enough to warrant care. Detectors have been revised since, models have improved, and vendors have mostly stopped publishing rates at all. That absence is itself worth noting: a category that competed on accuracy claims in 2023 has largely gone quiet on them. This page is maintained, and if an independent evaluation of current tools is published with its methodology attached, the table above changes to match it.
The short version
A detector score tells you that a piece of writing is statistically unremarkable. It does not tell you who wrote it. On the public evidence, no tool clears 80 percent accuracy, the error lands hardest on second language writers and on Black students, a clean score misses roughly one AI passage in six, and a single prompt instruction walks past the whole category. Whatever you are deciding, decide it on drafts, on a conversation and on the quality of the work itself. If the number cannot be wrong without changing your decision, you were never using the number as evidence.