The satire in the title is reminiscent of how Firebase was born.

We were previously working on a chat system called Envolve (https://www.envolve.com), that was 'Facebook Chat for any website'. A game that was using us for in-game chat created channels, used display: none on them, and passed game state through the chat.

We scratched our head, asked them why, and learned they wanted to focus on the frontend, not to deal with realtime message passing.

This led us to create a 'headless version' of our chat infra (re-written in Scala) that became the Firebase Realtime Database.

Very funny.

Also, you can actually make it cost competitive if the object you store is the last n milliseconds of packets instead of one packet each. So, instead of incurring two API calls per packet, you incur two API calls per minimum buffering time. If S3 is zero-rated for any ingress/egress then you get “infinite” bandwidth for 4.22$ * 3 (for the active case) = 12.66$ a day if you are willing to accept 500 ms minimum latency, or ~600$ a day at a more reasonable 10 ms. If you are saturating even just a 1 Gb link for a whole day that is ~10,000 GB which would be ~700$ via the blessed channel, so you could very well come out ahead.

You could do even better if you out-of-band signal the readiness so you do not need to poll while idle. Then you only incur a cost while actively transmitting so as long as you average 1 Gb/s on the channel you should be coming out even or ahead with minimal latency impact.

I once used a MySQL database as a replacement for a message queue. This was the easiest solution to implement since all the servers were already connected to the database anyways. A server would write a new row to the table and all the servers would remember the last row they had already seen. Occasionally the table is cleared. I'm sure there are some race conditions in the system but its only purpose is to send Discord notifications when someone breaks a highscore in a video game, so its not really critical. It's still working that way today.

I know of a Famous Large Company that used email as their message queue to synchronize the data across two of their large systems.

It's a perfectly apt message queue, just a bit heavyweight. But if it's "light enough", it comes "for free" with many OSes.

I was hoping someone would notice this by arguing about my pricing estimates in the arguments, but nobody did so I'm gonna spoil the surprise: if you have JavaScript enabled your browser will render slightly different prices if you view this post from Hacker News. This will persist to when you visit from not-Hacker News too, just so you don't notice the difference. The prices range from 0.8x to 5x the ones I figured out in the AWS calculator.

If you want to see how I did it, view source and search "gaslight".

I love this! Reminds me of Harder Drive by Tom 7:

https://youtube.com/watch?v=JcJSW7Rprio

It's better to go in unspoiled so I won't reveal any details.

I always thought it would be fun to host a Rube Goldberg competition for systems engineering. Whoever could accomplish a simple task with the most ridiculous system would win.

Something like: read the 54th line of this file hosted at xx address.

Then a submission could look like:

    1. FTP a file to a server with the address.
    2. the server reads the file and spins up a VM.
    3. The VM polls an endpoint to download code to pull the address.
    4. The code downloads the file and splits it into a single file per line.
    5. A script loads all the files into an array and accesses array[53] to get the answer.

As a junior dev I used folder pairs (to_process and processed) as a way to move messages between loosely coupled systems with file system watchers picking up new files in the to_process folder. Very light weight, got the job done, and was told it was in production for over a decade (even after better best practices came into use).

Wow, this reminds me of https://www.youtube.com/watch?v=JcJSW7Rprio (Harder Drive: Hard drives we didn't want or need - suckerpinch)

Impractical, yet possible ways to store data is an exciting satire genre for me now.

I initially expected a horrible amount of flamewar tinder, but it is an Overton Window expanding Art Project and I love it!

The Cardio rate control library is cool, https://pkg.go.dev/within.website/x/cardio this library would be a great problem for using a neural net to model a PID loop.

https://www.semanticscholar.org/paper/A-Design-of-FPGA-Based...

https://www.semanticscholar.org/paper/Self-Tuning-Neural-Net...

This, this is what the internet used to be like.

ok, what the good parts of the internet used to be like.

I also love that its highlights an AWS dark pattern. more, more I say.

> The bytes are stored in the cloud, which is slightly slower to read from than it would be to read data out of the heap.

Given latency and bandwidth differences, that's like saying that it's slightly slower to transport water by driving standard, 1-gallon jugs between the US east and west coasts than it is to transport it a few miles using a tanker truck.

The only thing worse would be this “on the blockchain.”

On second thought it seems likely that some charlatan has already years ago raised a couple of millions in an ICO for such a project. “We’re making the entire Internet web3-compatible!”

What could be worse than IPv6? this

Someone submit it as "IPv8" immediately.

> What if there was a way you could reduce that cost for your own services by up to 700%?

How can something be reduced over 100% What is it that they actually mean here?

This is the best worst thing in software that I have ever seen. I love it so much.

This is astoundingly well-written and informative. It's the sort of thing I come here for. It's also engaging and charming in a way rarely seen in tech writing.

I like that they started out by talking about $0.07/G cost and went through the whole exercise before pointing out what immediately came to mind for me when they started pushing bytes in and out of S3.

"anything stored in that pointer to memory you got back from malloc() is stored in an area of ram called "the heap", which is moderately slower to access than it is to access the stack." Is this true, or a myth? Ignoring the allocation cost and access patterns making cache misses more likely, surely memory is just memory

This is the first hn headline that I can remember actually making me laugh out loud. Bravo.

>"Access to S3 is zero-rated in many cases with S3, however the real advantage comes when you are using this cross-region."

Or cross-Country... as in two Countries that are geoblocked from one another...

>"This lets you have a worker in us-east-1 communicate with another worker in us-west-1 without having to incur the high bandwidth cost per gigabyte when using Managed NAT Gateway."

Simplified algorithm for bypassing geoblock via the above method:

1) Select a cloud storage provider (could be any cloud storage provider or shared persistent storage platform; doesn't necessarily need to be Amazon/S3) that works or does business in two countries, Country A and Country B, where normal IP traffic is geoblocked between Country A and Country B.

2) Use shared cloud storage objects/buckets/rows (call them whatever you will, "keyed persistent storage discrete thingies" for lack of a better term!) as the article suggests, to emulate IP traffic between user A in Country A and user B in blocked country B...

3) Combined with a P2P or other front end app that knows how to use this method of communication (along with code stubs, such that people could customize it to their own cloud storage provider or platform) if/when normal country-to-country IP communication is blocked for whatever reason (zombie apocalypse? <g>) could make a powerful future P2P communications tool, for lawful purposes...

Anyway, great article!

I just had a flashback of creating a forwarding-buffer-queue on memcached, a complete abomination but it was able to drastically reduce the load on the endpoints that saved game progress in a casual game.

It updated or created an entry for each call. But if a buffer was already in the queue, it would write the most recent one into the existing slot. This had the effect of reducing the load by the multiple of the update rate. So if you had clients sending save game payloads every 30s and your queue depth is 2.5mins, then your write rate to disk is 1/5. I think.

But memcached wasn't Redis, this was pre-Redis, and memcached could have evicted any of those keys at any time. We gave it lots of space, it never GCd, we never needing to fix it. The game slide from above the fold and wasn't fun enough to be viable long term.

One of my proudest Scotty Engineering moments.

Such dark works from such adorable creatures. The nature of furry.

The title reminds me that, in a previous life, our "architecture" team implemented a service bus using Lotus Notes email.

Given that Corey Quinn is involved, I’m surprised that route 53 isn’t included somewhere (or maybe it is, I’m still reading through it)

i used Couchdb as "RPC" pseudo-HTTP request-response transport layer, essentialy two queues, both ways. Especialy useful for mobile devices when connectivity is a random thing. Hugely Simplifed both client and server - Let the db handle all the multiplication, connections, switch on and off, timeouts, retries, delays, repeats, etc.

>> Read time in minutes: 40

I wonder if this part is satire, if so, how come? I definitely feel it should be much less.

I think we can safely expand it even further: Anything can be a anything if you use it wrongly enough.

Too late, already deployed to prod :)

The title of this article is what malware authors say every day =)

At some point you can also discover it is Turing complete.

meanwhile in the minds of ZeroMQ devs...

Could someone explain this please?

I clicked the site and wow...

Tl;dr: S3-based tun/tap virtual network device.

Been there, done that

DigitalOcean had a nice blog post on this "From 15,000 database connections to under 100: DigitalOcean's tale of tech debt" https://www.digitalocean.com/blog/from-15-000-database-conne... they used MySQL as a message queue and migrated from it.

Sadly, they fundamentally misunderstood the "everything is a file" paradigm of *nix. It's not that everything is an extent of octets, it's that everything has a directory entry so C programs can use the open call to create a file handle. It might be more appropriate to say "Everything looks like a file in the file system and most every operation on a thing represented by a directory entry goes through a filehandle."