You already made a faulty assumption — that we're interested in "classifying the data" in the first …
You already made a faulty assumption — that we're interested in "classifying the data" in the first place. Maybe we already know everything about the dataset. For example, if it's line-of-business customer data gradually built up by a sales team, then the brains of the salespeople have likely already done all the "implicit classification" needed to generate good questions about the dataset. And this is, by far, the usual scenario for Business Intelligence questions: someone with "business-domain knowledge", e.g. an executive, has formed an intuitional hypothesis about the data based on their personal experience; and so they ask someone with "data-domain knowledge", e.g. a business analyst or data scientist, to test that hypothesis. It's actually rare, in my experience, to have a tabular-data dataset that someone is motivated to understand, that doesn't also "come with" a set of people who can already act as (good!) models trained on that dataset, to aid them in that understanding. …
Original discussionWhy do tree-based models still outperform deep learning on tabular data? (2022)111 source comments · at import