Model Parameters

Also known as: weights

Model parameters are the learned numerical values — weights and biases — stored in the neural network after training. For transformer-based language models, parameters include the token embedding matrix, the query/key/value projection matrices in every attention head, feed-forward layer weights, and layer normalization scales. Parameter count is a common proxy for model capacity: larger models can represent more complex patterns but require more memory and compute to run.