Glossary

destroying the learned representations

a phenomenon sometimes called "catastrophic forgetting." The new classification head, initialized randomly, needs larger updates to learn meaningful weights quickly. Using **discriminative learning rates** (smaller for pretrained, larger for new layers) allows the pretrained features to adapt gently

Learn More

Related Terms