Can Transformers Learn Causality? Part 3: What This Means For Deployment and Practice
Last time I covered the research on whether transformers can learn causal structure:
- Part 1: The theoretical and empirical case that they can
- Part 2: The reality check — their world models are less coherent than they appear
Now the practical question: what do you do with this information?
Three Principles for Deployment
1. Don't Trust Benchmark Performance
A model that scores well on your test set might have a completely incoherent world model underneath.
The Myhill-Nerode approach — testing for consistency across equivalent situations — is a better probe than accuracy alone. If your model succeeds on task A but fails on logically equivalent task A', something is wrong even if A is the one you care about.
Concrete practice: Before deploying, construct equivalent reformulations of your key tasks. If performance varies wildly, the model is pattern-matching, not understanding.
2. Expect Brittleness at the Edges
Even models that genuinely learned something will be fragile when deployment differs from training.
This is especially true for causal reasoning. Small changes in the underlying structure can invalidate learned patterns. The model learned that A causes B in your training data — but maybe that was only true because of context C, which won't be present in production.
Concrete practice: Map the assumptions embedded in your training data. Which correlations might be context-dependent? Test those explicitly.
3. Verify Causal Claims Empirically
If your application depends on the model understanding cause and effect — rather than just correlating inputs with outputs — you need to test this directly.
Concrete practice: Construct scenarios where correlation and causation diverge. If smoking correlates with lung cancer in your data, does the model "know" that stopping smoking reduces risk? Or does it just predict outcomes from correlated features?
The Honest Summary
Transformers can learn causality — sometimes, partially, fragilely.
That's more than nothing. It's why they work so well on so many tasks.
But it's less than we might want. The gap between correlation and causation isn't fully bridged. The Lucas Critique isn't fully escaped.
For practitioners, this means: treat LLM outputs as sophisticated pattern matching until proven otherwise. They might be doing something deeper. But you can't assume it. And in domains where the difference matters — medicine, policy, law, finance — that assumption could be costly.
What's Next
The field is moving fast. Research directions that could change this picture:
- Causal representation learning: Training models to explicitly represent causal structure
- Better evaluation: Beyond accuracy to coherence, consistency, counterfactual reasoning
- Architecture changes: Modifications that make causal learning more robust
For now, we're somewhere in between "just correlations" and "true understanding." Knowing exactly where matters for what we can responsibly build.
Full paper list: