PART 4 · MODULE 2Under the hood
How LLMs Actually Work
The internals of large language models
Loading module...
The internals of large language models
In Part 3, we explored Transformers — the architecture behind modern AI. Now let's see how that architecture becomes an LLM. At its core, an LLM plays one game over and over: given these words, predict the next one.
Before an LLM can help you, it spends months reading. Books, Wikipedia, websites, code repositories — billions of documents. It doesn't memorize them. Instead it learns the patterns of how language works.
Trillions of words from books, articles, forums, and code. A single training run can process more text than a person could read in a thousand lifetimes.
The model learns grammar, facts, reasoning patterns, coding conventions, and even humor — all from predicting the next word.
When you send a prompt, the LLM doesn't write the whole answer at once. It predicts one token, appends it to the input, and predicts again. This loop repeats hundreds or thousands of times until the response is complete.
THE LLM GENERATION LOOP
At each step, the model doesn't just pick one word. It assigns probabilities to every possible next token. The temperature setting controls how adventurous the model is when choosing.
TEMPERATURE: CONTROLLING CREATIVITY
Always picks the most likely word. Good for facts and code.
"The capital of France is Paris."
"The capital of France is Paris."
"The capital of France is Paris."
Mix of predictable and surprising. Good for general writing.
"The capital of France is Paris."
"France's capital city is Paris."
"Paris serves as France's capital."
More random and surprising. Good for brainstorming and poetry.
"Paris, the luminous heart of France..."
"Ah, France — its crown jewel is Paris!"
"The beating pulse of France: Paris."
A model's size is measured in parameters— the adjustable values it learns during training. More parameters means more capacity to capture subtle patterns.
7B
Good at simple tasks, fast and cheap to run.
70B
Strong reasoning, handles complex instructions well.
400B+
State-of-the-art quality, requires massive compute.
The context window is how many tokens the model can "see" at once. A bigger window means it can consider more of the conversation or document. Early models had 2,000 tokens. Today's models can handle over 1,000,000.
Raw pre-trained models are like know-it-all interns — they have knowledge but don't know how to present it helpfully. RLHF(Reinforcement Learning from Human Feedback) fixes this. Human reviewers rank the model's outputs, and the model learns to prefer responses that people find helpful, honest, and safe.
LLMs predict the most likely next token based on patterns — they are not looking up facts in a database. When the patterns suggest a plausible-sounding answer that happens to be wrong, we call it a hallucination. The model is confident because the words sound right, even if the facts are not.
LLMs are incredibly sophisticated pattern-completion machines. They predict the most likely next token, billions of times, to produce text that reads as if a human wrote it.