Transformer

The transformer is a neural network architecture built around self-attention rather than recurrence. It processes all tokens in a sequence simultaneously, letting each position attend to every other position in parallel. Introduced in the 2017 'Attention Is All You Need' paper, the transformer became the dominant architecture for language modeling, machine translation, and most modern large-scale AI systems because it scales efficiently with data and compute.