Overfitting

Overfitting occurs when a model learns patterns specific to its training data so thoroughly that it fails to generalize to new examples. In classical NLP, overfitting is common when vocabulary-sized feature vectors have far more dimensions than training examples. Countermeasures include regularization (L1/L2 penalties), dropout, data augmentation, and cross-validation to detect the gap between training and validation performance before deploying a model.