glomers

Wrapped up distributed systems (eecs491) in college and loved it. There, I built a replicated, sharded key-value store with Paxos, and now am diving deeper into dist-sys patterns through Fly.io's challenges.

What's Here

a series of dist-sys challenges, each one building on the last:

1. Echo - Getting our feet wet with the framework

2. Unique ID Generation - Making IDs that are actually unique across nodes

3. Broadcast - Making nodes talk to each other reliably

   1. (3a) Single-Node Broadcast

   2. (3b) Multi-Node Broadcast

   3. (3c) Fault-Tolerant Broadcast

   4. (3d) Efficient Broadcast

      1. Part I

      2. Part II

4. Grow-Only Counter - Building a CRDT (fancy distributed counter)

5. Kafka-Style Log - Implementing a distributed log with some Kafka-like properties

   1. (5a) Single-Node Kafka-Style Log

   2. (5b) Multi-Node Kafka-Style Log

   3. (5c) Efficient Multi-Node Kafka-Style Log

6. Transaction System - Building increasingly consistent distributed transactions

   1. (6a) Single-Node Transactions

   2. (6b) Read Uncommitted Transactions

   3. (6c) Read Committed Transactions

Each challenge has its own write-up explaining the key ideas and tradeoffs.

Final Thoughts

Finished all challenges! Started with simple echo servers, worked through message passing patterns, built CRDTs, replicated logs, and wrapped up with increasingly sophisticated distributed transaction systems - all while handling partitions and maintaining high availability. Still no silver bullets in distributed systems, but now I have a solid arsenal of patterns.

Setup

You'll need:

• Go 1.20+

• Maelstrom 0.2.3 (the testing framework)

• Needs Java (OpenJDK)

  • Download from Maelstrom's releases page

Execute ./run.sh in a challenge directory to run the tests.