Multi-Head Attention

Multi-head attention runs several self-attention operations in parallel, each with its own learned projection matrices for queries, keys, and values. Each 'head' can specialize in attending to different relationship types — syntactic structure, coreference, or long-range context. The outputs of all heads are concatenated and projected back to the model dimension. Using multiple heads consistently outperforms a single larger attention operation of equivalent parameter count.