Decoder-Only Architecture

A decoder-only architecture is a transformer variant that uses causal (left-to-right) self-attention: each token can attend only to tokens that precede it in the sequence. This design makes the model naturally suited to autoregressive text generation. GPT, LLaMA, Claude, and most leading large language models use decoder-only architectures. The encoder-decoder design used in the original transformer is more common in translation and summarization models where the full input is available upfront.