Theoretical Deep Learning, 2: Worst case bounds. Part 3

We present a different approach to bound testtrain risk difference. This approach naturally leads us to the notion of Rademacher complexity. We upperbound the latter using covering numbers. These covering numbers appear to be computable for deep ReLU nets with upperbounded weight norms. Combining this, we obtain bound on testtrain risk difference which depends on Lipschitz constant of the learned network. Find all relevant info on github page: Our opensource framework