The trick: Lab Not Field
The 96% accurate deepfake detector is 96% accurate on the deepfakes it was shown.
On new ones it is closer to a coin flip.
deepfake detectors hit around 96% accuracy, so synthetic video can be caught reliably. Intel's FakeCatcher, marketed at a 96% accuracy rate, is the number everyone quotes.
Before you read on. Your call?
TRUE, BUT
78% in the wild
that figure is a controlled-lab result. The DeepFake-Eval-2024 study found leading commercial detectors around 78% on in-the-wild deepfakes, NIST evaluations put accuracy below 40% when the generator differs from the training software, and the Vector Institute named the pattern the Generalization Illusion, where benchmark scores stay high while real-world detection quietly declines. Real capability, badly oversold.
There’s more to this story.
Membership opens the full investigation, the strongest counterargument and what to do with what you’ve learned.
Start your free month →First membership: 30 days free, then A$89 a year. One introductory trial per customer. Card required; renews annually until cancelled. Cancel before the trial ends to avoid the first charge. Already a member? Sign in
Couldn't check your access. That's on us.
The trick has a name
We call it Lab Not Field: it works in the test conditions, not the deployed ones. You'll see it again. Learn to spot it →
Receipts
- Supports newsroom.intel.com:
the company has developed FakeCatcher, a technology that can detect fake videos with a 96% accuracy rate
- Refutes helpnetsecurity.com:
The Vector Institute team calls the result the Generalization Illusion. Benchmark scores stay high. Real-world detection performance quietly declines.
- Context realitydefender.com:
the DeepFake-Eval-2024 study found that leading commercial detectors achieved approximately 78% accuracy on in-the-wild deepfakes, a significant drop from published benchmark numbers
Open the Receipts Pack → What each source proves, every figure traced, and what would change our verdict.