Neural Networks with Quadratic VC Dimension

摘要

This paper shows that neural networks which use continuous activation functions have VC dimension at least as large as the square of the number of weightsw. This results settles a long-standing open question, namely whether the well-knownO(w log w) bound, known for hard-threshold nets, also held for more general sigmoidal nets. Implications for the number of samples needed for valid generalization are discussed.