The conversation

klodolphHNSource author
Business & incomeExperience excerpt · English

The problems I’m talking about arise because when you use an S3-compatible API on your NFS system, …

… The problems I’m talking about arise because when you use an S3-compatible API on your NFS system, it’s often true that you’re rolling the dice with three different vendors—you have the storage appliance vendor, you have the vendor for the software talking to S3, and you have Amazon who wrote the S3 client libraries. It’s kind of a nightmare of compatibility problems in my experience. Amazon changes how the S3 client library works, the change wasn’t tested against the storage vendor’s implementation, and boom, things stop working. But your first call is to the application vendor, and they are completely unfamiliar with your storage appliance. :-(

Original discussionI Just Want Simple S3127 source comments · at import
2

What was this responding to?

Parent context · original excerpt

ES
estebarbHN

Personally I would suggest that the "easiest S3" would be simply using NFS. You can get replication with RAID. S3 is simple for the users, not the operators. For replicating something like S3 you need to manage a lot of parts and take a lot of decisions. The design space is huge: Replication: RAID, distributed copies, distributed erasure codes... Coordination: centralized, centralized with backup, decentralized, logic in client... How to handle huge files: nope, client concats them, a coordinator node concats them... How will be the network: local networking, wan, a mix. Slow or fast? Nature of storage: 24/7 or sporadically connected. How to handle network partitions, pick CAP sides... Just for instance: network topology. In your own DC you may say each connection has the same cost. In AWS you may want connections to stay in the same AZ, use certain IPs for certain source-destination to …

Full original

Community replies 0

No community replies yet. Add your perspective.

Replies from the source 2

Direct replies available in this import. Original-language excerpts.

TH
themafiaHN

> but it’s just not good enough in most NFS systems. NFS is just an interface. At the end of the day it's on top of an FS. It's entirely possible and sometimes done in practice to replicate the underlying store served by NFS. As you would expect there are several means of doing this from the simple to the truly "high-availability."

Full original
KA
KaiserProHN

Why would you use S3 on top of NFS? I mean you can, it would simplfy the locking somewhat. But if you are doing file sharing for apps inside a network you manage, just use NFS, and maybe worry about the locking later.

Full original
Continue on Hacker News

Respect different perspectives. Share concrete experiences.

Sign in to post, reply, repost, and quote.