A fabulous reference resource from the Jepsen and Antithesis teams. A glossary of key concepts related to distributed systems.

Recent research on intra-datacenter routing topologies:

We design and deploy in production the first flat datacenter networks. Our design, called RNG, is based on quasi-random graphs. While the cost and fault-tolerance benefits of such topologies have been long known, their practical realization has been hampered by a lack of scalable routing and cabling approaches. RNG has a new distributed routing protocol that exploits the properties of random graphs to find a large number of edge disjoint paths between pairs of endpoints. It uses a novel passive optical device that internally shuffles cables, which makes its cabling complexity similar to that of fat trees. We show that RNG matches or exceeds the performance of fat trees for a range of traffic patterns, despite being up to 45% cheaper. RNG is now the default datacenter network for most workloads at Amazon.

Phil Eaton on orienting oneself when beginning the study of distributed systems:

Distributed systems is simply the study of interactions between processes. Every two interacting processes form a distributed system, whether they are on the same host or not. Distributed systems create new challenges (compared to single-process systems) in terms of correctness (i.e. consistency), reliability, and performance (i.e. latency and throughput).

Phil’s recommendations are to find a couple of friends and work together on reading Martin Kleppmann’s book Designing Data-Intensive Applications and working through MIT’s 6.5840: Distributed Systems course.

Broad high-level remarks on consensus in distributed systems from Graydon Hoare, originator of the Rust programming language and several other influential software projects.

A fascinating from-scratch construction of something approaching CRDTs starting from a much simpler idea, “server reconciliation”. You, yes you, can implement this!

Collaborative text editing is arguably the hardest feature to implement in a collaborative app. […] Or rather, that’s what I thought until recently. This blog post describes an alternative, straightforward approach to collaborative text editing, without Conflict-free Replicated Data Types (CRDTs) or Operational Transformation (OT).

An examination of several approaches to handling overload, flow-control, and backoff, with examples drawn from Erlang/OTP programming.

[…] there is a rich zoo of overload problems and potential solutions […] ranging far beyond a simple queue boundary for mailboxes. By reasoning and looking at your system, it becomes interesting to figure out which approach is best: a small system with some relatively infrequent load-spikes may be fine with a queue or a pull-based mechanism. An all-synchronous system may be fine with simple counter-based protection for most tricky cases. […]

A collection of 100 impossibility results and some discussion of commonalities and differences among them:

This talk is about impossibility results in the area of distributed computing. In this category, I include not just results that say that a particular task cannot be accomplished, but also lower bound results, which say that a task cannot be accomplished within a certain bound on cost.

You may ask yourself why impossibility proofs are important: my answer is that if you don’t know what’s impossible, you run the risk of wasting a lot of time trying to achieve the unachievable!