Jev for AI text detection

01One question, no training

A dedicated AI-text detector is a classifier: someone collected a corpus, labelled it, trained a model, and shipped the weights. The reported approach here deletes every step after the first. You ask the decision model the question directly and use the returned probability as the score. No dataset, no training run, no retraining when the underlying generators change.

That is a real structural advantage and it is worth being precise about why. A trained detector encodes the statistical fingerprint of the models that existed when its corpus was collected. When the next generation of models changes those statistics — and editing tools change them too, without any model change at all — the detector degrades and the only fix is another labelled corpus. A zero-shot question does not have that failure mode, though it has others.

Zero-shot does not mean the wording is free. “Was this written by an AI model?” and “Is this text machine-generated rather than human-written?” are not the same question, and the probability that comes back moves with the phrasing, the amount of context you include, and whether you ask for a choice or a score. That is an extra variable a trained detector does not have, and it means any number you read about this approach is a number about a particular prompt. If you are going to rely on it, the prompt is part of your configuration and should be versioned like one.

questions = {
  written_by_model: { type: 'choice', criteria: { yes, no } },
}
// the probability attached to 'yes' is the score; no training corpus involved

02The number that matters is the false-positive rate

Reported headline accuracy for these experiments sits in the low nineties. That number is close to useless on its own, because the two error types have completely different consequences. Flagging a model’s text as human is an annoyance. Flagging a person’s writing as machine-generated is an accusation, and in any setting where the verdict has consequences — academic, editorial, hiring — it is the error that ends the experiment.

The builds that report the useful number are the ones that report both. One measured 94 percent accuracy with one percent of human samples flagged; that false-positive figure is the one to carry forward, and it should be re-measured on your own domain before you use the system on anything that matters. Detector behaviour varies sharply between short product copy, long essays, and text that has been edited by a person after being drafted by a model.

03Cost and latency, and why they are not the point

The reported economics are striking: one build measured the decision-model approach at roughly a six-thousandth of the cost of a dedicated detector while being five to ten times faster, and another at about three hundredths of a millisecond per text at a fraction of a cent per thousand. Those are real, and they matter if you are screening a corpus rather than checking one document.

But they are not the reason to prefer this approach, and treating them as the reason leads to a bad system. A cheap detector that is wrong in a way that harms someone is worse than an expensive one that is right. The cost figures are what make it feasible to screen at volume; the accuracy figures are what decide whether you are allowed to.

04Failure modes, in the order they bite

Detection quality is not a property of the detector alone. The same system will perform differently on marketing copy, on a research paper, and on a forum post, because the question “was this written by a model” is easier to answer when the text is repetitive and harder when it is not. A single aggregate number hides that variance, and the variance is where the false accusations live.

Heavy editing moves text across the line in both directions. A model-drafted paragraph that a person rewrote is genuinely mixed authorship, and any binary answer is wrong about part of it. If your use case cannot tolerate that ambiguity, detection is the wrong tool rather than a tool in need of a better threshold.

The arms race runs one way. Every improvement in the detector invites a change in the generator, and the change is cheap for the generator side. Building a workflow that depends on detection staying accurate is building on a moving surface; building one where detection is one weak signal among several is not.

And the last one is procedural: a probability is not a verdict. Reporting “eighty-five percent likely” to a person who has to make a decision is a different act from using a threshold internally, and the two should not share a UI.

05Where this sits in the wider pattern

This is the same closed-question shape as the document classification page — one input, one label, a probability you threshold — with the label set reduced to two. The difference is that here the output has consequences for a person, which is why this page spends most of its length on error asymmetry rather than on speed.