If the paper would not have used this clickbait title, it would probably go unnoticed and noone would pay attention? Sad truth about sensation-oriented modern lives :(
It is true that MLPs are classic, but the regularizaton methods that apparently make a big empirical difference at this paper are new concepts (data augmentation, skip connections/residual blocks, dropout, batch norm,…
Batch norm has an advantage for iterative methods on mini-batches, while XGB uses the full training set. Using batch norm on the full training set is equivalent to Z-normalizing the features, which has no effect at all…
I agree with your point, e.g. data augmentation can be added, but thats pretty much it. All the other regularization techniques they use are neural network specific and cannot be applied to gradient-boosted trees. What…
After reading the paper I believe they do both aspects that you mention: i) they give XGboost and all the baselines up to 4 CPU days of hyperparameter optimization time on 20 CPU core servers per dataset, same compute…
If the paper would not have used this clickbait title, it would probably go unnoticed and noone would pay attention? Sad truth about sensation-oriented modern lives :(
It is true that MLPs are classic, but the regularizaton methods that apparently make a big empirical difference at this paper are new concepts (data augmentation, skip connections/residual blocks, dropout, batch norm,…
Batch norm has an advantage for iterative methods on mini-batches, while XGB uses the full training set. Using batch norm on the full training set is equivalent to Z-normalizing the features, which has no effect at all…
I agree with your point, e.g. data augmentation can be added, but thats pretty much it. All the other regularization techniques they use are neural network specific and cannot be applied to gradient-boosted trees. What…
After reading the paper I believe they do both aspects that you mention: i) they give XGboost and all the baselines up to 4 CPU days of hyperparameter optimization time on 20 CPU core servers per dataset, same compute…