The conversation

l33tmanHNSource author
Lessons learnedExperience excerpt · English

I started such an alternative project just before GPT-3 was released, it was really promising (lots …

I started such an alternative project just before GPT-3 was released, it was really promising (lots of neuroscience inspired solutions, pretty different to Transformers) but I had to put it on hold because the investors I approached seemed like they would only invest in LLM-stuff. Now a few years later I'm trying to approach investors again, only to find now they want to invest in companies USING LLMs to create value and still don't seem interested in new foundational types of models... :/ I guess it makes sense, there is still tons of value to be created just by using the current LLMs for stuff, though maybe the low hanging fruits are already picked, who knows. I heard John Carmack talk a lot about his alternative (also neuroscience-inspired) ideas and it sounded just like my project, the main difference being that he's able to self-fund :) I guess funding an "outsider" non-LLM AI project now requires finding someone like Carmack to get on board - I still don't think traditional investors are that disappointed yet that they want to risk money on other types of projects..

Original discussionGPT-5: Key characteristics, pricing and system card295 source comments · at import
2

What was this responding to?

Parent context · original excerpt

GO
godelskiHN

> It does sort of give me the vibe that the pure scaling maximalism really is dying off though I think the big question is if/when investors will start giving money to those who have been predicting this (with evidence) and trying other avenues. Really though, why put all your eggs in one basket? That's what I've been confused about for awhile. Why fund yet another LLMs to AGI startup. Space is saturated with big players and has been for years. Even if LLMs could get there that doesn't mean something else won't get there faster and for less. It also seems you'd want a backup in order to avoid popping the bubble. Technology S-Curves and all that still apply to AI Though I'm similarly biased, but so is everyone I know with a strong math and/or science background (I even mentioned it in my thesis more than a few times lol). Scaling is all you need just doesn't check out

Full original

Community replies 0

No community replies yet. Add your perspective.

Replies from the source 2

Direct replies available in this import. Original-language excerpts.

GO
godelskiHN

> I guess funding an "outsider" non-LLM AI project now requires finding someone like Carmack to get on board And I think this is a big problem. Especially since these investments tend to be a lot cheaper than the existing ones. Hell, there's stuff in my PhD I tabled and several models I made that I'm confident I could have doubled performance with less than a million dollars worth of compute. My methods could already compete while requiring less compute, so why not give them a chance to scale? I've seen this happen to hundreds of methods. If "scale is all you need" then shouldn't the belief that any of those methods would also scale?

Full original
AN
andrepdHN

Markets are very good and squeezing profit out of sometimes ludicrous ventures, not good at all for foundational research. It's the problem of having organised our economic life in this way, or rather, exclusively this way.

Full original
Continue on Hacker News

Respect different perspectives. Share concrete experiences.

Sign in to post, reply, repost, and quote.