Towards demystifying over parameterization in deep Soltanolkotabi Workshop 3 CEB T1 2019

Mahdi Soltanolkotabi (USC) , Towards demystifying overparameterization in deep learning. Many modern learning models including deep neural networks are trained in an overparameterized regime where the parameters of the model exceed the size of the training dataset. Training these models involve highly nonconvex landscapes and it is not clear how methods such as (stochastic) gradient descent provably find globally optimal models. Furthermore, due to their overparameterized nature these neur