In-Context Learning in Transformers: Correlation vs Causation

In a16z podcast, Columbia professor Vishal Misra argues that transformers mainly learn correlation, not causation—an obstacle to true intelligence and AGI. He explains how LLMs generate text by predicting the next token from probability distributions, and how prompt context can sharply change outputs. Misra highlights in-context learning as a key mechanism: when an LLM receives examples in the prompt, it can solve problems in real time. He says this behavior resembles Bayesian updating, where new evidence shifts belief and the model’s next-token probabilities update in a mathematically predictable way. The discussion also covers why token-space is modeled sparsely (many token combinations are effectively nonsensical), improving efficiency by filtering irrelevant combinations. Misra notes practical extensions such as domain-specific languages (DSLs) that convert natural-language questions into structured database queries. To evaluate machine-learning architectures more rigorously, Misra proposes a “Bayesian wind tunnel,” a controlled testing framework to compare transformers with other model types (e.g., MAMBA, LSTMs, MLPs). Overall, he frames progress toward AGI as moving beyond pattern matching toward causal understanding and continuous post-training learning—while using in-context learning as a bridge.