← Back to AI Tools
AI Tools

AI text detectors are not evidence. Here is what they can tell you

Every few months a story lands about a student accused, a freelancer dropped, or a writer questioned on the strength of an AI-detection score. The score is presented as a measurement. It is closer to a statistical guess with a confidence attached, produced by a model trained on text that resembles the text it was trained on. That does not make detectors pointless. It makes them easy to misuse, and the misuse follows a remarkably consistent pattern.

A laptop at night displaying a document with softly highlighted sentences and a confidence meter in amber and violet, reading glasses and coffee in the foreground
Practical principle: a detector score tells you how much a passage resembles a training set. It cannot tell you who wrote it.

What a detector is actually measuring

Most detectors look at two things. The first is perplexity, a measure of how predictable each next word is. The second is burstiness, the variation in sentence length and rhythm. Machine-generated prose tends to be smoother and more evenly paced than human writing, so a model trained on labelled examples learns to separate the two distributions.

Notice what is missing from that description: any evidence about the author. A detector never sees your keystrokes, your drafts, or your notes. It sees a string of characters and estimates how similar that string looks to text it was trained to label. Everything that follows is a consequence of that gap.

Two practical effects follow immediately. Editing changes the score, because editing changes predictability. And different detectors, trained on different data with different thresholds, disagree with each other about the same paragraph.

The false-positive problem is not evenly distributed

The most consequential finding in this literature is not about average accuracy. It is about who the errors land on. In a widely cited Stanford study, essays written by non-native English speakers as part of a language test were misclassified as machine-generated at a mean rate of roughly 61 percent across seven detectors. The same detectors performed far better on essays from native speakers.

The mechanism is not mysterious. Second-language writing, and any writing produced under tight formulaic constraints, tends to be more regular: predictable connectives, uniform sentence structure, a narrow vocabulary. Those are exactly the signals the classifiers treat as machine-like. So are legal boilerplate, lab reports, translated text, and the careful, flat prose that many people produce when they are trying to be precise.

That distribution matters because the people most likely to be wrongly accused are often the least equipped to appeal: students far from home, staff writing in a second language, freelancers with no union and no lawyer.

The better commercial tools are better, with one important asymmetry

It would be unfair to describe detectors as uniformly bad. An independent working paper from the University of Chicago Booth School of Business, published in August 2025, tested several commercial detectors and found that the best of them kept false-positive rates at or below roughly one percent at their operating thresholds on the passages tested. That is a real improvement over the state of the art a few years earlier, when some tools produced false positives in the double digits.

The catch is the asymmetry. The same research found false negatives varying enormously between products, in one case ranging from about ten to forty percent depending on which model wrote the text. Read that combination carefully. A high score is more informative than a low one, but the error that damages a person's life is still the false positive, and confidence in the tool is not confidence in any individual result. Many of these studies also rest on fairly small samples, which is worth remembering when a vendor quotes a single number.

Editing moves the number more than people expect

Researchers at the University of Maryland found in early 2025 that minimal polishing of machine-written text with a general-purpose model shifted detection rates anywhere from about ten to seventy-five percent, depending on the detector. Reviews of paraphrasing tools report similar movement in the other direction, with some detectors dropping sharply once text is rewritten.

Both halves of that matter. It means a detector is not a reliable gate, because a light edit can beat it. And it means a human who used a grammar checker, a translation tool, or a paraphrasing aid for legitimate reasons can be pushed over a threshold without doing anything dishonest.

If you are the one being accused

Do not start by rewriting. Start by asking questions, politely and in writing, because the process is usually less rigorous than the letter suggests.

If you are the one setting the policy

The same research that complicates enforcement also points to a defensible way to use these tools. Treat a detector as a screening signal that can trigger a human conversation, never as a verdict. Publish the tool, the threshold, and the appeal route, so that people can respond to the actual standard rather than guessing at it. Measure the false-positive rate on your own population before you rely on it, especially if a large share of your students or staff write in a second language.

Require corroborating evidence before any sanction, log decisions so similar cases are treated similarly, and give people a genuine chance to explain their process. Where the stakes are high, assessment design is a better lever than detection: in-class writing, oral defence, and assignments tied to specific local sources are harder to fake and impossible to false-positive.

Where the numbers come from, and what they do not cover

Almost every figure in this area comes with a caveat. Studies use different thresholds, different models, different lengths of text, and different definitions of what counts as AI-assisted. Vendors publish their own benchmarks, which are worth reading and worth discounting. Detector behaviour drifts as models are updated, so a number from eighteen months ago is history rather than guidance. Treat every specific percentage here as an estimate with a wide error bar, and treat the direction of the findings as the durable part.

Regional notes for the US, Canada, UK, and Australia

In the United States there is no federal rule requiring or banning these tools in classrooms or workplaces. Decisions are made by institutions, and ordinary discrimination and consumer-protection law still applies to how those decisions are made and explained.

In Canada, privacy law constrains what student or employee text can be sent to a third-party detector and on what basis, which is why some institutions require a privacy assessment before adopting one.

In the UK, universities and exam boards publish their own misconduct policies, and a detector score on its own is generally not treated as sufficient proof of anything. In Australia, higher-education providers set their own rules but are expected to give students a reasonable chance to respond before a finding is made.

The common thread is that the policy, not the product, decides what a score means. That is good news: the policy is the part you can fix.

The bottom line

Detectors are better than they were, and still not a lie detector. The best commercial tools keep false positives low on the passages they were tested on, while missing a meaningful share of edited machine text and treating some honest human writing as suspicious. Use them to start a conversation, never to end one. If a score is about to cost someone a place, a job, or a reputation, the burden should be on the evidence around it, not on a number that nobody can fully explain.

Questions readers ask

Can an AI detector prove that someone used ChatGPT?

No. Detectors estimate how similar a passage looks to text they were trained to label as machine-generated. They do not observe how the text was produced, and published research documents both false positives on human writing and substantial false negatives on machine text, especially after editing or paraphrasing. A score can justify a closer look; it cannot establish authorship.

Why do AI detectors flag writing by non-native English speakers?

Because second-language writing often has the statistical profile detectors associate with machines: more predictable word choices, more uniform sentence structure, and a narrower vocabulary. A widely cited Stanford study found essays from non-native writers were misclassified at a mean rate above sixty percent across seven detectors, while native-speaker essays were recognised far more reliably.

Is a low AI score proof that a text is human-written?

No. Detector performance is asymmetric: false positives are relatively low for the best tools, but false negatives vary widely between products and rise sharply after light editing or paraphrasing. A low score means the text does not resemble the detector's picture of machine writing. It is not evidence about how the text was made.

This article provides general information, not legal, financial, security, or professional advice. Product terms, availability, privacy practices, and local requirements can change. Check the current information from the relevant provider and seek qualified local advice for high-impact decisions. IntelGuard may earn a commission from future affiliate links, at no additional cost to readers. Read our affiliate disclosure.