glomers

my solutions to fly.io's distributed systems challenges.

Published

04 January 2025

Wrapped up distributed systems (eecs491) in college and loved it. There, I built a replicated, sharded key-value store with Paxos, and now am diving deeper into dist-sys patterns through Fly.io's challenges.

What's Here

a series of dist-sys challenges, each one building on the last:

flowchart LR
    echo["echo"]
    uid["unique ids"]
    broadcast["broadcast"]
    counter["CRDT counter"]
    log["Kafka-style log"]
    txn["transactions"]

    echo --> uid --> broadcast --> counter --> log --> txn

    classDef blue fill:#dbeafe,stroke:#3b82f6,color:#1e293b
    classDef green fill:#dcfce7,stroke:#22c55e,color:#1e293b
    classDef amber fill:#fef3c7,stroke:#d97706,color:#1e293b
    classDef slate fill:#f1f5f9,stroke:#64748b,color:#1e293b
    class echo,uid,broadcast blue
    class counter green
    class log amber
    class txn slate

The series starts with message plumbing and ends with the annoying parts: replication, ordering, caching, and consistency.

  1. Echo - Getting our feet wet with the framework
  2. Unique ID Generation - Making IDs that are actually unique across nodes
  3. Broadcast - Making nodes talk to each other reliably
  4. (3a) Single-Node Broadcast
  5. (3b) Multi-Node Broadcast
  6. (3c) Fault-Tolerant Broadcast
  7. (3d) Efficient Broadcast
  8. Part I
  9. Part II
  10. Grow-Only Counter - Building a CRDT (fancy distributed counter)
  11. Kafka-Style Log - Implementing a distributed log with some Kafka-like properties
  12. (5a) Single-Node Kafka-Style Log
  13. (5b) Multi-Node Kafka-Style Log
  14. (5c) Efficient Multi-Node Kafka-Style Log
  15. Transaction System - Building increasingly consistent distributed transactions
  16. (6a) Single-Node Transactions
  17. (6b) Read Uncommitted Transactions
  18. (6c) Read Committed Transactions

Each challenge has its own write-up explaining the key ideas and tradeoffs.

Final Thoughts

Finished all challenges! Started with simple echo servers, worked through message passing patterns, built CRDTs, replicated logs, and wrapped up with increasingly sophisticated distributed transaction systems - all while handling partitions and maintaining high availability. Still no silver bullets in distributed systems, but now I have a solid arsenal of patterns.

Setup

You'll need:

- Download from Maelstrom's releases page

  • Go 1.20+
  • Maelstrom 0.2.3 (the testing framework)
  • Needs Java (OpenJDK)

Execute ./run.sh in a challenge directory to run the tests.