"Model LLMs as a sparse matrix to grasp them The speaker, a Columbia computer science professor, developed a mathematical abstraction to understand how large language models actually work. He describes LLMs as a giant matrix where every row is a possible prompt and every column is a probability over the next token: "given a prompt, they construct a distribution of probabilities of the next token." Because arbitrary token combinations are gibberish, the matrix is extremely sparse, and "what all these LLMs are doing is coming up with a compressed representation of this matrix." Anyone trying to reason about AI behavior — from engineers to product managers — can use this mental model to move from vague intuitions to precise questions about what a model can and cannot do. Source: Vishal Misra, No Priors (Martin Casado), 'How LLMs Actually Work: Bayesian Inference, Causality, and the Path to AGI'"