Use a held-out test set when:

Your dataset is large enough that you can afford it - You want a final, honest evaluation after all model development decisions are made