The conversation

Business & incomeExperience excerpt · English

You already made a faulty assumption — that we're interested in "classifying the data" in the first …

You already made a faulty assumption — that we're interested in "classifying the data" in the first place. Maybe we already know everything about the dataset. For example, if it's line-of-business customer data gradually built up by a sales team, then the brains of the salespeople have likely already done all the "implicit classification" needed to generate good questions about the dataset. And this is, by far, the usual scenario for Business Intelligence questions: someone with "business-domain knowledge", e.g. an executive, has formed an intuitional hypothesis about the data based on their personal experience; and so they ask someone with "data-domain knowledge", e.g. a business analyst or data scientist, to test that hypothesis. It's actually rare, in my experience, to have a tabular-data dataset that someone is motivated to understand, that doesn't also "come with" a set of people who can already act as (good!) models trained on that dataset, to aid them in that understanding. …

Original discussionWhy do tree-based models still outperform deep learning on tabular data? (2022)111 source comments · at import
3

What was this responding to?

Parent context · original excerpt

DI

> ask your questions in the form of SQL queries How do you know which questions to ask? This is what ML is good at, finding the right questions which classify the data.

Full original

Community replies 0

No community replies yet. Add your perspective.

Replies from the source 3

Direct replies available in this import. Original-language excerpts.

LE

I'm really not clear on why you're arguing against this. A proper data warehouse tackles the known unknowns, i.e. supervised learning. But you can glean new insights using unsupervised learning, like the textbook example of Target knowing a woman is pregnant based on sales data. https://www.forbes.com/sites/kashmirhill/2012/02/16/how-targ...

Full original
LE

I might be misunderstanding your point, but there's use cases that have repeatedly come up for me in multiple businesses, below being some examples, without getting too specific: - identify latent features of customers via their behavioral data, to be used for profiling customers or recommending products to them - within a large amount of customer behavioral data, identify potentially fraudulent behavior - identify causes of seasonality (e.g. temporal patterns) in the data in order to improve forecasting (sales, traffic, whatever) In those cases part of the investigation is to initially take a hands-off (unsupervised) approach, so that we can compare our initial top-down hypotheses with actual patterns in the data. In both of those cases there's considerable (and sometimes adversarial) noise in the data.

Full original
IT

> You already made a faulty assumption — that we're interested in "classifying the data" in the first place. It's not clear what your point is. If you're not interested in the predictions that tree-based models provide, do not use tree-based models on your tabular data. A predictive model and a SQL query are not the same thing.

Full original
Continue on Hacker News

Respect different perspectives. Share concrete experiences.

Sign in to post, reply, repost, and quote.