Introduction

What cairn is

cairn is a distributed key-value database built from scratch in Rust. It is a portfolio flagship, not a product — the goal is to demonstrate the ability to architect a large, genuinely hard system end to end: a custom storage engine, a real consensus protocol, multi-key transactions, and a sharded cluster with a control plane on top.

The complexity here is intrinsic to the problem — consensus, storage-engine internals, transaction isolation, and shard placement — not a function of traffic volume. The bar is a working, benchmarked, adversarially-tested system with an architecture writeup that explains why each tradeoff was made, not just what was built.

Guarantees

The test suite exists to prove these hold, not just assert them:

• Linearizable reads and writes within a single key's Raft group.
• Snapshot isolation for multi-key transactions via MVCC.
• Durability — an acknowledged write survives a crash of the acking node (WAL fsync + Raft commit on a majority).
• Availability under a minority of node failures per group (Raft quorum).

Cross-shard transactions are explicitly out of the guarantee set for the first cut — see Roadmap and the design decisions for how that's fenced.

The storage engine and the Raft consensus core are both built and independently proven — the engine against a BTreeMap reference model, the consensus core against four safety invariants under a deterministic, fault-injecting simulation (see Raft Consensus). The guarantees above are what holds once they're wired into one running system and chaos-tested end to end, which is the work still ahead — see Roadmap for exactly what's left.

Non-goals

• Serving third-party or production traffic. This runs locally or on a homelab.
• SQL. No parser, planner, or relational layer — the interface is a typed KV + transaction API.
• Multi-region / geo-replication, auth, or wire-compatibility with an existing database.

How the docs are organized

• Architecture — the layered system design and how a write flows through it.
• LSM Storage Engine — internals of the Phase 1 engine: WAL, memtable, SSTables, bloom filters, compaction.
• Raft Consensus — the Phase 1 consensus core: log store, transport, election and replication, read-index linearizable reads, and the safety-invariant simulation that proves it.
• Roadmap — phased delivery plan with honest status.
• Design Decisions — ADR-style writeups of the tradeoffs that shaped the system.
• Benchmarks — real numbers from the shipped engine, with methodology.

Source

cairn is open source. The code, specs, and implementation plans this site is generated from live at github.com/uptonm/cairn.