Can We Really Trust AI Detectors

Most people, including myself, believe they can easily spot AI-generated writing. Some pay attention to signs like the em dash. Then there are patterns and stylistic devices that are effective when used moderately, but when they're repeated to the point of fatigue, you just know it's ChatGPT.

People who work with textual content all the time typically have a better nose for AI, but even they can't be sure. That's what created the ground for the AI detection market. But those detectors only seem useful until you write a piece and then get accused of submitting AI content as your own because your editor, teacher, or client ran an AI detection tool.

According to Penn State University research, most people can't tell the difference between AI-generated and human-written content. On the other hand, those who do are precisely the same people who use it the most and can circumvent it by careful prompting and post-editing. A Cornell University paper says that People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text.

How Accurate Are AI Detection Tools?

It makes sense to employ algorithms when we can't trust human judgment. But it looks like we can't trust them either. Most detectors are accurate in more than 50%, but that's far from reliable.

AI checkers basically measure "perplexity" in a text. They look for simple, predictable language, low complexity, limited vocabulary, repetitive phrasing. That's what AI tends to produce, but, for example, people who use English as a second language use similar patterns in writing. A recent Stanford study shows that the average false-positive rate for TOEFL essays is 61.3%.

Universities that initially embraced Turnitin's AI writing detection feature started turning it off, to avoid false accusations. MIT, Yale, Berkeley, and many others (see the full list here) banned AI detectors altogether. Others no longer provide them to the staff and advise teachers against using them. But the problem is still not resolved, as there are those that still trust them, and many are drawn by the promise of efficiency those tools provide.

Meanwhile, they keep marketing their products. Despite the massive false-positive rate, certain AI detectors promise higher accuracy. But there's a massive issue with those claims.

Grammarly claims to be 99.99% accurate. Copyleaks claims it can recognize human writing with 99.97% reliability and detect AI output with 99.20% reliability.

Originality.ai's homepage shows a comparison table based on six published studies. Their scores range from 85 to 100 across those studies. Sounds impressive, until you look closer. Most competitors are marked "not included" in most of these studies. GTPZero appears in all six, but scores between 45 and 84. Tools like Copyleaks and Writer only show up in one or two. Different tools were tested on different datasets, with different methods. So the table doesn't actually compare them head-to-head. It just shows Originality.ai scored well in the studies it entered.

Detectors Testing Their Own Tools

Many of the accuracy figures circulating online come directly from the companies selling the detectors. That's a significant conflict of interest. A company that profits from convincing teachers, publishers, and employers that AI detection works has every incentive to publish favorable results and to test their tools under conditions that make them look good.

The most common version of this is testing detectors on clean, unedited AI text. Feed a detector a raw ChatGPT response, untouched, and it will definitely catch it. Any detector can flag those. But that's not how AI writing gets used in practice. Most people edit, paraphrase, or lightly rewrite LLM output before using it. Even minor edits can change those results, and skilled AI users know how to trick the detectors.

And the companies selling detectors know that, as well. They know exactly what kinds of edits are needed to bypass the algorithm.

Selling the Problem and the Solution

Many of the same companies that sell AI detectors also sell tools designed to make LLM writing sound more human. Originality.ai, one of the better-known detector products, markets both. Although, under recent pressure, the humanizer tool is no longer listed in their main offer list, and they have since added a disclaimer stating that the detector flags humanized text, just as it does raw outputs.

Can You Really Distinguish Human Writing From AI Output?

The deeper problem is that AI detection market was built on the premise that AI writing is reliably distinguishable from human writing. But models improve all the time, and that premise gets weaker day by day.

The commercial incentives point in the wrong direction. Detection companies profit from appearing to solve a problem they can't fully solve, and in some cases, from selling the very tools that make their detectors fail.

Company-funded accuracy claims mean little without independent, peer-reviewed benchmark to back them up. Until those exist, treat any detection percentage the way you'd treat a drug trial funded by the manufacturer.