How AI Detection Tools Work — and What They Actually Detect

The two core signals: perplexity and burstiness

Most AI detection tools rely on two core statistical signals derived from language model research.

Perplexity

Perplexity measures how "surprising" a piece of text is to a language model. When an AI writes text, it naturally chooses high-probability, low-surprise word sequences — the words that statistically fit best in context. This produces low-perplexity text. Human writers, by contrast, make unexpected word choices, use unusual structures, and take creative risks — producing higher-perplexity text overall.

Detection tools measure this: low perplexity suggests AI authorship; high perplexity suggests human authorship.

Burstiness

Burstiness refers to variation in sentence length and complexity. Human writing is "bursty" — we write long complex sentences, then short punchy ones, then medium ones. AI writing tends to be uniform: each sentence is roughly the same length and complexity, creating low burstiness. Detectors measure this variation (or lack of it) as an additional signal.

The major AI detection tools compared

Note: Accuracy figures are self-reported or from limited testing. Real-world performance on edited or humanized text is significantly lower for all tools.

The false positive problem

This is the most important and least-discussed issue with AI detection: false positives are common and consequential.

Research has consistently shown that certain types of human writing score very high on AI detection tools — not because they were AI-generated, but because they happen to share stylistic traits with AI output:

• Non-native English speakers, who write more simply and predictably
• Students with formal academic training, who use structured, predictable essay formats
• Technical writers, whose clarity-focused style resembles AI output
• Writers in certain genres (how-to guides, product descriptions) that naturally follow predictable patterns

A 2023 study found that GPTZero flagged over 50% of essays written by non-native English speakers as AI-generated. This is a serious problem when detection results are used to make academic misconduct decisions.

"AI detectors don't detect AI — they detect writing that resembles AI. That's a crucial distinction, and most institutions aren't making it clearly enough."

What makes text harder to detect?

Several factors consistently make AI-generated text harder for detection tools to catch:

• Sentence length variation — Adding short and long sentences disrupts the low-burstiness signal
• Vocabulary specificity — Using concrete, specific words raises perplexity
• First-person voice — Personal anecdotes and opinions are harder to attribute to AI
• Editing — Even light human editing meaningfully reduces detection scores
• Humanizer tools — Tools like Humanizor restructure the text at the level that detectors measure

The arms race problem

AI detection is fundamentally an arms race that detection tools are losing. As language models improve, their output naturally becomes harder to distinguish from human writing. Each new model generation produces more varied, nuanced text — and the detection tools trained on older model outputs are less accurate on new ones.

OpenAI released and then quietly retired its own AI classifier in 2023, noting it was "not reliable enough." Anthropic has not released a detection tool, citing similar concerns about accuracy.

What this means for writers

If you're a writer using AI tools for any purpose — whether that's drafting, editing, research, or brainstorming — the most important things to know are:

1. Detection tools are imperfect and produce false positives, especially for non-native speakers
2. No tool can prove that text is AI-generated — only that it shares statistical properties with AI output
3. Editing your AI drafts meaningfully reduces detection scores and, more importantly, improves writing quality
4. Humanizer tools change the statistical properties that detectors measure — primarily perplexity and burstiness