What are large language models?
A large language model, usually shortened to LLM, is a neural network trained on an enormous amount of text to predict what word or word-fragment comes next in a sequence, and through learning to do that prediction well across a vast and varied body of text, it develops the ability to generate coherent writing, answer questions, follow instructions, and reason through problems, despite never being explicitly programmed with rules for any of those specific tasks. Everything else this collection covers about language models, reasoning models, small language models, multimodal models, open source models, is a variation or specialization built on top of this same fundamental idea: a model that learned to predict text so well, across so much text, that the prediction itself became a genuinely useful general-purpose capability.
Why “predicting the next word” produces something that looks like understanding
It’s genuinely surprising, on first encounter, that a model trained on nothing more than predicting the next piece of text in a sequence ends up capable of things that look like reasoning, creativity, and even a kind of understanding. The explanation is that predicting the next word well, across an enormous and varied enough body of text, turns out to require the model to implicitly learn an enormous amount about how the world works, how facts relate to each other, how arguments are structured, how a story develops, since accurately predicting what comes next in a passage about physics, or a legal contract, or a piece of code, genuinely depends on the model having internalized real patterns about physics, law, and programming, not just surface-level statistics about which words tend to follow which others.
This is the core insight that makes the whole approach work: next-word prediction isn’t a narrow, shallow task when it’s applied across a broad enough range of text, it’s a task that, done well, requires the kind of deep pattern recognition that ends up looking a great deal like genuine understanding, even though the model’s actual training objective never explicitly asked for understanding at all, only for accurate prediction.
What “large” in large language model actually refers to
The “large” in large language model refers primarily to the number of parameters, the individual adjustable values inside the model that get tuned during training, and secondarily to the enormous volume of text the model was trained on to tune those parameters. Both of these scaled up dramatically over the years leading to today’s most capable models, and this scaling turned out to matter enormously, larger models trained on more text consistently demonstrated meaningfully better capability than smaller ones trained on less, a relationship researchers now understand well enough to predict fairly reliably in advance of training a specific model.
This scaling relationship is exactly why the field pursued larger and larger models for a considerable stretch of time, and it’s also why the emergence of small language models, covered in this collection’s dedicated article on that topic, represents a genuinely notable development, better training techniques and more careful data curation have found ways to extract more capability out of a smaller parameter count than the original scaling relationship alone would have predicted, without abandoning the underlying next-word prediction approach that makes the whole thing work in the first place.
How training actually happens, in broad strokes
Training a large language model happens in stages. The first and by far most computationally expensive stage, pretraining, exposes the model to an enormous, broad corpus of text and trains it purely on the next-word prediction task described above, with no specific task or behavior being targeted beyond that general prediction ability. This produces a model with broad, general capability but not necessarily behavior that’s well-suited to being directly useful in a conversational or instruction-following context.
A further stage of training, often involving human feedback on the model’s responses, shapes that broad, general capability into behavior that’s useful and appropriately calibrated, following instructions reliably, declining genuinely harmful requests, communicating in a way that’s helpful rather than merely technically correct. This is why a model fresh out of pure pretraining behaves quite differently from the same model after this further stage of training, the underlying knowledge and capability were already present after pretraining, but the model’s ability to apply that capability usefully and appropriately came from this additional, more targeted training that follows it.
Why scale alone stopped being the whole story
For a considerable period, the dominant strategy for improving language model capability was straightforward, train a larger model on more text, and capability would predictably improve. This relationship still holds broadly, but the field has increasingly recognized that scale alone isn’t the only lever available, the reasoning models covered in this collection’s dedicated article demonstrate that applying more computation at the moment a specific question is being answered, not just during the model’s original training, can meaningfully improve accuracy on hard problems, and the progress in small language models demonstrates that careful data curation can substitute for some of what raw scale used to provide.
This broadening of what actually drives capability improvement, model size, training data quality, additional training stages, inference-time computation, is part of why the field of language models has diversified so considerably beyond a single race toward ever-larger models, different levers now offer genuinely different tradeoffs, and the right combination depends on what a specific application needs, connecting directly to the broader discussion of model selection covered elsewhere in this collection.
Why large language models don’t actually “know” things the way a database does
A large language model’s knowledge isn’t stored anywhere as an explicit, lookup-able fact the way a traditional database stores a record, it’s distributed across the model’s parameters as patterns learned during training, which is why a model can produce a fluent, confident-sounding answer that’s nonetheless factually wrong, the model isn’t retrieving a stored fact and misreporting it, it’s generating a plausible continuation based on learned patterns, and those patterns don’t come with a built-in guarantee of factual accuracy. This distinction matters enormously for understanding the hallucination problem covered throughout this collection’s broader discussion of AI reliability and verification, it isn’t a bug that occasionally slips through, it’s a direct consequence of how the underlying prediction mechanism actually works.
Understanding this also clarifies why techniques like retrieval-augmented generation and tool calling, both covered in their own dedicated articles in this collection, matter so much for building reliable applications on top of large language models, they give the model access to actual, verifiable information at the moment of generating a response, compensating for the fact that the model’s own internal knowledge, however impressive, was never designed to function as a reliable, lookup-able source of truth on its own.
Why understanding this foundation matters for everything else in this collection
Nearly every other topic covered in this collection, agentic systems, tool calling, retrieval-augmented generation, model serving, inference optimization, ultimately traces back to working with or around the fundamental nature of a large language model as described here, a system that generates plausible, pattern-based continuations of text rather than retrieving verified facts or executing explicit logical rules. The considerable engineering discipline covered throughout this collection, verification, testing, careful tool design, thoughtful architecture, exists specifically because this underlying nature creates both remarkable capability and genuine, predictable limitations that need to be actively designed around rather than assumed away.
Understanding large language models at this foundational level is what makes the rest of this collection’s more specific, more applied discussions make sense, every technique for improving reliability, managing cost, or extending capability is ultimately a response to some specific consequence of how these models actually work underneath, not an arbitrary best practice disconnected from the underlying mechanism it’s addressing.
Common mistakes people make around large language models
1. Assuming a large language model retrieves and reports stored facts the way a database does, rather than generating plausible, pattern-based continuations that carry no built-in guarantee of accuracy.
2. Treating scale, more parameters, more training data, as the only lever for improving capability, missing the growing range of other approaches that now offer genuinely different tradeoffs.
3. Expecting a model fresh out of pretraining to behave usefully in a conversational context, without recognizing that further, more targeted training is what shapes that broad capability into appropriate, useful behavior.
4. Underestimating how directly the rest of this collection’s practices, verification, tool calling, retrieval, trace back to specific, predictable consequences of how large language models actually generate text.
5. Dismissing the “just predicting the next word” description as proof that these models can’t do anything genuinely sophisticated, missing why that prediction task, done well at scale, produces capability that goes considerably beyond what the simple description suggests.
What connects these mistakes is underestimating both the genuine capability and the genuine limitations that follow directly from how large language models actually work, understanding the mechanism clearly, prediction based on learned patterns, not stored facts or explicit rules, is what makes it possible to use these models effectively rather than either overestimating their reliability or underestimating their real capability.
The deeper point about large language models is that an enormous amount of what looks like reasoning, knowledge, and understanding in a modern AI system emerges from a training objective that never explicitly asked for any of those things, only for accurate prediction across a vast body of text, and grasping that origin is the foundation for understanding everything else this collection covers about how to build reliable, capable systems on top of models that work this way.