Why Do Tree Based-Models Outperform Neural Nets on Tabular Data?

[dhlee]

Eco-friendly approach is for our future

[ThethDoctor]

Fascinating - I actually found this out myself while experimenting on a dataset which was funny enough about actual trees!

[wryltxw]

Great format. Compact and highly informative.

[edufantini]

Tree-based models are really awesome but, from my experience with my research, they're behind NN-based models (specially ResNets) if you need good OOD predictions. Neural Nets are better at generalization.
Oh, and great video btw. Loved the format!

[xTagmanx]

Awesome first short, really great video for the format

[mrmosaic]

One of the most useful shorts on the YouTube I've seen.

[mgostIH]

Meta-learning approaches like TabPFN will dominate, I want to see a future where every task can be solved by few shot transformers!

[SherriMSDRML-qmpe]

Thank you thank you 🤠🐎👣🌍🧭🧲🧠 thank you thank you

[isaac]

Algorithm led me to this shorts. This is such miracle

[j.]

I don’t even know what the hell this means but I like it!

[stkyriakoulisdr]

Please when you have a question as a title, answer the question!!

[haluk]

Easy, DL overfits a model with huge number of freedoms. Tabular data has some correlation. Hence its nature is smooth. So using DL to learn tabular data is to use a construction machinery to open a walnut.

[Houshalter]

Neural nets have a huge advantage though. They can do unsupervised learning. Which can greatly improve performance on supervised tasks with limited data. Since it models all of the data available instead or just one variable.

Another note is that in the early days of deep learning, people often cut neurons out of the neural network. And used them as features for decision trees.

Also I'm skeptical trees could beat a bayesian neural network. BNNs are slow, complicated and a giant pain to get working. So you rarely hear about them. But perhaps better approximations of Bayes could still beat the trees.

[bayestraat]

Aight mate im subbing your channel for more

[yw]

Love those 'laser' graphs. Hope there's template

[piotr]

mayby effective dimension of tabular data (number of latent variables) so very small - like in mapping from (x, y) to RGB for whole image - then image is blur

[username]

does anyone know where do we check these model comparisons for tabular data sets ? or any easiest and better way to make models for classifications of tabular sets

[BederikStorm]

Trees split at each node based on a single feature. Thus, you don't need to normalize inputs, to deal with mixture of categorical and numerical data, etc. Trees just are good with that
Maybe, they are good with LLM, we just didn't create a good algorithm to grow such tree

[_mk]

I think you should try making more youtube Shots.
In some cases, this can serve as a pointer to more detailed videos.

[radhikadesai]

But designing neural nets is an art. How can u say that the neural net u designed was the best design for that model ?