Challenges:

**Underflow:** Small gradient values become zero in fp16 (minimum positive value: ~6e-8). - **Overflow:** Large values exceed fp16 range (max: 65504). - **Loss of precision:** Accumulated rounding errors can affect training.