The “Good Enough” Classical NLP Principle

The Principle

When classical NLP techniques achieve 90%+ of LLM-based performance, the classical approach often wins on total value delivered.

Why This Matters

The current AI landscape defaults to “use the biggest model.” But production systems optimize for total value: performance × reliability × cost × speed. A technique that’s 94% as accurate but 10× cheaper and 100× faster changes the math entirely.

Dependency parsing is decades old. It’s well-understood, deterministic, and runs on CPU. LLM-based extraction is newer, impressive, and expensive. For knowledge graph construction, the old technique does nearly as well.

This pattern likely generalizes: for many NLP tasks, the gap between classical and modern approaches is smaller than marketing suggests.

When to Apply

Use classical techniques as the default when:

• The performance gap is single digits (< 10%)
• Scale matters (thousands of documents, continuous processing)
• Determinism matters (same input → same output)
• Cost scales linearly with usage

Use LLM-based approaches when:

• The task genuinely requires reasoning, not just pattern matching
• Performance gaps are larger and consequential
• Scale is limited and cost isn’t a primary constraint
• You need handling of truly ambiguous cases

Exceptions

• Tasks requiring genuine language understanding, not just structure extraction
• Domains where 6% accuracy difference has significant real-world impact
• One-time processing where cost amortizes across many downstream uses

The principle isn’t “never use LLMs,” it’s “prove you need them before paying for them.”

Related: 05-atom—the-94-percent-threshold, 06-atom—construction-bottleneck-problem, 05-atom—demos-deployment-ethics-gap