Large Language Models

LLMs explained simply and quickly

Large language models generate responses by predicting one token at a time, based on patterns they’ve learned from massive datasets. This explainer covers their core components: data, architecture, training, and how transformer models use context to simulate conversation.

EOS and maximum length tokens tell an LLM when to stop generating a response

Tokens are words or subwords (e.g., prefixes) that a large language model processes to generate a response. During training, the model is taught that if it ever generates a special token called end-of-sequence, its response is complete.

LLMs are the backbone of AI tools that process and generate natural language

Large language models identify patterns in massive datasets of books, code, and unlabeled text from which to generate coherent responses. GPT-3 was trained on about 45 terabytes of data and uses 175 billion parameters—each parameter being a value the model adjusts as it learns.

Find out how important you are to AI tools when they generate text responses with this tool, which estimates the weights of people's names.

Tokens—numbers assigned to words and subwords—are the basic units of data that AI tools process to generate outputs that mimic language without understanding the text itself.

Transformer architecture was the breakthrough that made AI chatbots possible

It changed how text was translated from a sequential, "one word at a time" method to one where every word in a text is processed in parallel. Advances in positional encoding and self-attention also helped models recognize context and word order better.

Visualize how transformer architecture generates an LLM's outputs

Transformer architecture is the neural network framework upon which large language models predict and generate text based on user inputs. This interactive lets you explore the "thinking" a GPT does to generate responses based on your input.

LLMs are trained via modeling, imitation learning, and reinforcement learning

Training large language models begins with pretraining the model to predict the next word in a sequence based on finding patterns in massive amounts of text. Patterns are then fine-tuned to model human-written dialogue and to align responses with user preferences.

Temperature is a value that controls how predictable or creative an LLM's output is

In large language models, temperature adjusts the randomness in text generation by modifying how often words with lower probabilities are selected next in the sequence. Lower temperatures produce more obvious outputs while higher temperatures produce more creative responses.

Unlike generative AI systems, such as chatbots that do not take follow-up actions after generating their output, agentic AI systems proactively take steps to achieve goals without continuous user input.