2 comments

[ 5.0 ms ] story [ 12.5 ms ] thread
I ran some experiments to firm up my geometric intuition about ReLU networks.
Without looking the answer, what is your intuition about the size of the VC-dimension of ReLU networks as a function of a number of weights and layers?

Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks https://arxiv.org/abs/1703.02930