The right mindset for hosting cost is to think of it in FTEs (full time employee cost for a year)

You should keep in mind that for a lot of things you can use a servicing contract, rather than hiring full-time employees. It's typically going to cost significantly less; it can make a lot of sense for small companies, especially.

Your calculation assumes that an FTE is needed to maintain a few beefy servers. Once they are up and running that employee is spending at most a few hours a month on them. Maybe even a few hours every six months. OTOH you are specifically ignoring that you'll require mostly the same time from a cloud trained person if you're all-in on AWS. I expect the marginal cost of one employee over the other is zero.

To be fair, I think people are vastly over estimating the work they would have and the power they would need. Yes, if you have to massively scale up, then it'll take some work, but most of it is one-time work. You do it, and when it runs, you only have a fraction of work over the next months to maintain it. And with fraction, I mean below 5%. And keep in mind that >99% of startups who think of "yeah we need this and that cloud, because we need to scale" will never scale. Instead they are happily locking themselves into a cloud service. And if they actually scale at some point, this service will be massively more expensive.

> At scale (like comma.ai), it's probably cheaper. But until then it's a long term cost optimization with really high upfront capital expenditure and risk. Which means it doesn't make much sense for the majority of startup companies until they become late stage and their hosting cost actually becomes a big cost burden. You rent a dataspace, which is OPEX not CAPEX, and you just lease the servers, which turns big CAPEX into monthly OPEX bill Running your own DC is "we have two dozen racks of servers" endeavour, but even just renting DC space and buying servers is much cheaper than getting same level of performance from the cloud. > This flips when you start getting into the multiple FTEs per month in cost for just the hosting. At that point you probably have additional cost measured in 5-10 FTE in staffing anyway to babysit all of that. So now you can talk about trading off some hosting F…

> it doesn't make much sense for the majority of startup companies until they become late stage Here's what TFA says about this: > Cloud companies generally make onboarding very easy, and offboarding very difficult. If you are not vigilant you will sleepwalk into a situation of high cloud costs and no way out. and I think they're right. Be careful how you start because you may be stuck in the initial situation for a long time.

> But until then it's a long term cost optimization with really high upfront capital expenditure and risk. The upfront capex does not need to be that high, unless you're running your own AI models. Other than leasing new ones, as a sibling comment stated, you can buy used. You can get a solid Dell 2U with a full service contract (3 years) for ~$5-10K depending on CPU / memory / storage configuration. Or if you don't mind going older - because honestly, most webapps aren't doing anything compute-heavy - you can drop that to < $1K/node. Replacement parts for those are cheap, so buy an extra of everything.

On the software side... depending on your business model, you can factor in a lot of the cost structures into your structure. Especially for say B2B arrangements. Cloud integrations, for example, allow you to simply use a different database instance altogether per customer, while you can share services that utilize a given db connection. But actually setting up and managing that type of database infrastructure yourself may be much more resource intensive from a head count perspective. I mention this, because having completely separate databases is an abstraction that cloud operations have already solved... while you can choose other options, such as more complex data models to otherwise isolate or share resources how does this complexity affect your services down-stream and the overall data complexities across one or all clients. Harder still, if your data/service is centered around b2b …

>At scale (like comma.ai), it's probably cheaper. But until then it's a long term cost optimization with really high upfront capital expenditure and risk. The issue with comma.ai is that the company is HEAVILY burdened with Geohotz ideals, despite him no longer even being on the board. I used to be very much into his streams and he rants about it plenty. A large reason of why they run their own datacenter is that they ideologically refuse to give money to AWS or Google (but I guess Microsoft passes their non woke test). Which is quite hilarious to me because they live in a very "woke" state and complain about power costs in the blog post. They could easily move to Wyoming or Montana and with low humidity and colder air in the winter run their servers more optimally.