Transformers are the architecture behind modern LLMs, using attention to weigh how each token relates to every other token in the input. Knowing that output is sampled one token at a time explains why any text in the context can steer the model and why the same prompt can produce different answers.
Read in this order
- VideoTransformers, the tech behind LLMs (3Blue1Brown)Start here: a visual explanation of attention and next-token prediction.
- ArticleThe Illustrated Transformer (Jay Alammar)A diagram-driven walkthrough of each part of the transformer, step by step.
- PaperAttention Is All You NeedThe original paper, if you want the exact architecture.