Generative AI & LLMs • Module A: The Transformer RevolutionLesson 1: How Transformers and Attention Work
Every modern Large Language Model—including ChatGPT, Claude, and Gemini—is built on a single, revolutionary neural network architecture: the Transformer. Let's demystify how it works.
Historically, text processing relied on Recurrent Neural Networks (RNNs). RNNs processed text word-by-word sequentially. This meant they were slow to train, struggled to remember long-range dependencies, and could not run in parallel. Each word had to wait for the previous word to finish processing—imagine reading a book where you can only look at one letter at a time.
Published in the famous 2017 paper "Attention Is All You Need", the Transformer replaced sequential processing with the Attention Mechanism. Instead of reading word-by-word, a Transformer processes all words simultaneously. Through Self-Attention, the model determines how much "attention" each word in a sentence should pay to every other word. For example, in the sentence "The bank of the river", the word "bank" pays attention to "river" to determine its meaning, rather than "money".
Click any word to see its attention distribution. Each cell shows how much the Query word (row) attends to each Key word (column).
Notice how "it" attends most strongly to "chicken" — the model learns coreference resolution, understanding that "it" refers back to "chicken", not "road".
Mathematically, Self-Attention works like a database lookup. For every token (word/sub-word), the network projects three vector representations, namely the Query, Key, and Value vectors:
The model computes dot products of the Query vector of one word with the Key vectors of all other words, applies a softmax function to get attention weights, and multiplies these weights by the Value vectors.
See how a token is decomposed into Query, Key, and Value vectors, and how dot product scores are computed and normalized via softmax.
Answer the following multiple choice question about the attention equation:
What does the term Q · KT compute?
In the next lesson, we will see how we can guide these attention mechanisms to generate highly specific text outputs using prompt engineering!