attention weights would not form a distribution over positions. 4. **Gradient flow** would be altered --- softmax provides a natural gradient that encourages competition between positions.