Nubmq: A high performant key-value store engine built from first principles

by nubskr on 3/15/2025, 8:54:23 AM with 18 comments
not able to post the link in url for some reason, here it is: https://github.com/nubskr/nubmq

  • by ForTheKidz on 3/15/2025, 10:26:14 AM

    The image here appears to be upside down (or rather rotated 180): https://github.com/nubskr/nubmq/blob/master/assets/architect...

    It's not clear from the readme physically where the data is stored, nor where in the storage process the "congestion" is coming from.

    I'm surprised there's no range scan. Range scans enable a whole swathe of functionality that make kv stores punch above their weight. I suppose that's more rocksdb/dynamo/bigtable/cassandra than redis/memcached, though.

  • by avinassh on 3/15/2025, 10:55:52 AM

    This looks great, how are you doing the benchmarks? It claims to be way faster than Redis. Can you also measure it against Microsoft Garnet? Whats the secret sauce for beating in latency?

    Redis:

        Write Latency ~1.1ms
    Read Latency ~700µs
    Max Throughput ~85,000 ops/sec
    Nubmq:

        Write Latency 900µs
    Read Latency 500µs
    Max Throughput 115,809 ops/sec
    also, 700µs for Redis reads sounds high to me. Running against memtier_bench also would be great - https://github.com/RedisLabs/memtier_benchmark

  • by pmjoe on 3/15/2025, 2:17:15 PM

    Is Redis using all cores or you're comparing a multi threaded implementation in Go with a single threaded one in Redis?

  • by fnordian on 3/15/2025, 12:17:39 PM

    What first principles were used to build this?

  • by solo_jinwoo on 3/16/2025, 6:33:16 PM

    How to contribute? Could you add contribute.md ?

  • by andsoitis on 3/15/2025, 10:26:17 AM

    are you using it in production yet?