The conversation

latchkeyHNSource author
Business & incomeExperience excerpt · English

On Dec 6th, AMD released the MI300x

On Dec 6th, AMD released the MI300x. This is effectively their entry into the AIA (ai accelerator is the new GPU) market. The previous generation (MI250) cards were good, but this new release leapfrogs everything. These cards offer 192gb of ram and pretty great performance. 8 cards in a chassis. Great for training and inference. Sure, we all know that the AMD software needs work, but I think the long bet here is on Lisa. Reminds me of the Mac vs. Windows days. If NVIDIA owns this much of the market, then I'd want to work to decentralize my AI business off a single point of failure. So, I've been working to launch a business to buy up as many of these chips as I can and make them available as bare metal rentals, even with IPMI access. This is something that hasn't been done before. Usually these sorts of AMD cards end up in super computers (ie: Frontier) and/or only accessible to a few people. Azure is catching on with their recent product, but that is a big cloud... I believe people will still want private clouds. …

Original discussionAnalysts estimate Nvidia owns 98% of the data center GPU market83 source comments · at import
4

Community replies 0

No community replies yet. Add your perspective.

Replies from the source 4

Direct replies available in this import. Original-language excerpts.

FR
fragmedeHN

that's awesome! what support are you giving for the rewriting that has to happen for things to run on ROCm or whatever instead of CUDA?

Full original
RE
renewiltordHN

Exciting news. How's software story on this? Currently we are on CUDA, etc. We have nice big AMD Epycs (1GB of cache!) but use Nvidia for GPU. Mi300x at $40k would be tempting, but hard to justify with porting effort required.

Full original
SA
sandworm101HN

>> We are interested in feedback on what people want and can be agile enough to customize our purchases for your needs. I would like you to not by pallets of these chips so that I and others may be able to get our hands on a couple for our own personal projects.

Full original
KE
KennyBlankenHN

"needs work" is an understatement. ROCM only just came out for windows(!) and so far everyone in the LLM and SD scene says that ROCM on linux is a pain in the ass to get working, requires a lot of hackery and override environment variables, is buggy, and also only supports a small handful of cards - whereas CUDA runs on nearly everything NVIDIA makes, from laptop GPUs to datacenter compute cards. And there's very little support on the third party software side at the moment. That "no consumer hardware support" sounds stupid, but people getting into ML/AI, grad students, etc want to be able to mess around and develop/prototype on local hardware they already own. You cannot do that with ROCM, because they only support a very, very small number of consumer-level cards - and very oddly, it's a mix of the very high end and low end, nothing in the middle. AMD is also massively behind hardware-…

Full original
Continue on Hacker News

Respect different perspectives. Share concrete experiences.

Sign in to post, reply, repost, and quote.