Permutation Testing for Machine Learning Model Validation using Sklearn

Have you ever had a model perform slightly better than chance and wondered if this was significant Here I ll show you one way of answering this question. Permutation testing is a great tool to use in machine learning in order to express if a model performance is greater than chance. Even though the theoretical boundary for chance is 50 for a two balanced class problem, when you are dealing with a small dataset it might not be the case. If you get 51, or even up to 60 accuracy you might still have