Sadly I never got to work on this when I was at Apple (interviewed for it though!), but hearing about this a few years ago sort of made me realize something that should have been obvious: there’s not really a difference between a database and a file system.

Fundamentally they do the same thing, and are sort of just optimizations for particular problem-sets. A database is great for data that has proper indexes, a file system is great for much more arbitrary data [1].

If you’re a clever enough engineer, you can define a file system in terms of a database, as evidenced by iCloud. Personally, I have used this knowledge to use Cassandra to store blobs of video for HLS streams. This buys me a lot of Cassandra’s distributed niceties, at the cost of having to sort of reinvent some file system stuff.

[1] I realize that this is very simplified; I am just speaking extremely high level.

Previously:

"FoundationDB: A Distributed Key-Value Store" [https://news.ycombinator.com/item?id=36572658]

"FoundationDB Record Layer" [https://news.ycombinator.com/item?id=18906341]

"Apple Acquires FoundationDB [https://news.ycombinator.com/item?id=9259986]

"How FoundationDB works and why it works" [https://news.ycombinator.com/item?id=37552085]

I leveraged FoundationDB and RecordLayer to build a transactional catalog system for all our data services at a previous company, and it was honestly just an amazing piece of software. Adding gRPC into the mix for the serving layer felt so natural since schemas / records are defined using Protobuf with RecordLayer.

The only real downside is that the onramp for running FoundationDB at scale is quite a bit higher than a traditional distributed database.

We gave up on iCloud for file sync, it’s broken on dozens of devices trying to “optimize” storage even when asked not to. Imagine having 4Tb (size doesn’t matter) mostly empty hard drives and not being “allowed” to keep a file copy offline, because iCloud knows better…

Now Apple is asking all file sync products like Dropbox to do the same, see Fileprovider API, breaking those as well. Really annoying

I wish they'd build iCloud to store my Time Machine backups.

With iCloud Apple indeed handles well update conflicts in Apple Notes. I have tried to set up Obsidian or any other Markdown-based notetaking system, the sync is so often and I had to give up. Apple Notes does handle this pretty well. So I finally moved to Apple Notes.

On an unrelated note, having the original title edited by the system after being submitted without the OP being noticed really annoys me, especially when the title starts with How, Why and other terms. It just made it a little weird to read, and sometimes it breaks the meaning. I once submitted a story and had some people complaining about the title being somehow misleading. When I noticed this, it was too late to edit the title.

In the HN guidelines, you read: "Otherwise, please use the original title, unless it is misleading or linkbait; don't editorialize."

I hope this will be taken into consideration.

SQLite & HCTree

Given that FoundationDB is built on top of SQLite, I wonder if that team is eyeing the HCTree engine for it.

It's still in experimental mode but provides literally 10x improvement on read/writes to SQLite.

Given Apple size & scale of iCloud, that seems like a massive win for them if that engine for SQLite can mature to production stability.

https://sqlite.org/hctree/doc/hctree/doc/hctree/threadtest.w...

This reminds me of years back when I worked in banking. I vaguely recall there was a report system called Hyperion(an?) (IBM?). The system generated a new database for every single report it made. I thought that was kinda crazy at the time but I guess it was ahead of the times.

Someone feel free to correct my memory if needed, I was not the primary person for this system or anything so I could be totally wrong.

…and still struggles to sync one file from my laptop to my phone.

Very cool. This is the architecture that inevitably results from when you start with boxed, native, desktop software and incrementally move towards cloud based storage and collaboration. You have to be really good at doing schema changes and version migrations, because they're happening at fantastic scale without administrator intervention: not when you launch, but when each individual customer chooses to use the next version.

Quite different from a SaaS-first approach where it actually makes sense to do "customer id column"-based multi-tenancy and one-migration-at-a-time schema changes that I think most of us at less-than-Apple scales are familiar with.

How would you handle schema migrations in a system like this?

CouchDB implements a DB per user approach. Personally, I've found it much easier to use than an SQL DB for web apps I've made, but I've heard others who've always used SQL say they were frustrated with it.

Great! If only I could manage which of my files stay local, and which get offloaded to iCloud I might be impressed. But it seems that iCloud likes to offload recently used files, apps and photos to make room for my massive library of old photos. It frequently makes my iPhone unusable unless I'm on wifi, and then I still have to wait for everything I want to use to re-download from iCloud.

This sounds a lot like AWS Aurora, which (I'm simplifying here) is a database interface on top of a distributed file store, (S3).

If only large downloads from Apple like iCloud pulls didn’t time out and you weren’t trapped in these databases as a result

Does anybody other than Apple use FoundationDB in production?

…and still can't show which 114 images in my iCloud photo library cannot be synced.

Phone says this since many years and iOS updates back: 10365 photos synced, Mac says: 10251 photos synced.

2005 - We need 1 database

2010 - We need 2 databases

2015 - We need 500 databases

2020 - We need billions of databases

2025 - Prediction: We need 1 database.