The paired t-test on cross-validation scores has a known problem: the folds are not independent (they share training data), which violates the t-test's independence assumption. This makes the test slightly anti-conservative. A correction called the "corrected resampled t-test" (Nadeau and Bengio, 20