etcd Internals on Random Tech Excursions

Mon, 01 Jan 0001 00:00:00 +0000

Chapter 1: gRPC Key-Value API and clientv3 #

Welcome! If you are new to etcd, you are in the right place. etcd is a distributed, reliable key-value store. That means it stores simple key -> value pairs and makes them available to many machines.

Imagine you are building a small web app. You want all of your servers to display the same dashboard background color. You could edit a config file on every server, but that gets messy. Instead, you store one key called dashboard/color in etcd. Your Go application uses the official Go client, clientv3, to save and read that value.

Mon, 01 Jan 0001 00:00:00 +0000

Chapter 10: Robustness Testing Framework #

Welcome back! In the previous chapter, Multi-Module Go Repository, you learned how etcd organizes its code into separate Go modules. That keeps dependencies clean and makes the project easier to maintain.

But a clean codebase is not enough. How do we know etcd’s distributed promises really hold? How do we prove that a successful write is still there after a crash, or that watches never travel back in time? That is exactly what the Robustness Testing Framework is for.

Mon, 01 Jan 0001 00:00:00 +0000

Chapter 2: etcdctl / etcdutl Command-Line Tools #

Welcome back! In the previous chapter, gRPC Key-Value API and clientv3, we wrote a small Go program to save and read a key. That is a great way to understand how applications talk to etcd. But sometimes you just want to type one command and see what is stored in the cluster. You do not want to write a Go program every time.

Mon, 01 Jan 0001 00:00:00 +0000

Chapter 3: Raft Consensus Engine #

Welcome back! In the previous chapter, etcdctl / etcdutl Command-Line Tools, you learned how to talk to a running etcd cluster using command-line tools. You ran etcdctl put, saw OK, and read the value back.

But have you ever wondered: how do multiple etcd servers agree on that value?

If you have only one etcd server, it can just write the value to disk. But etcd is built to run as a cluster of multiple servers. If those servers all wrote independently, they would quickly disagree. One server might have "blue", another might have "green", and a third might not have the key at all.

Mon, 01 Jan 0001 00:00:00 +0000

Chapter 4: Cluster Membership and Peer Communication #

In the last chapter, Raft Consensus Engine, you learned how etcd members use Raft to agree on a shared log of decisions. But what exactly is an “etcd member”? How do members join, leave, and talk to each other? That is what this chapter is about.

Think of an etcd cluster as a partner board. Each member is a partner with a name and a vote. The board makes decisions by voting. New partners can join, old partners can leave, and each partner has a phone number so they can be reached. The phone number is called a peer URL.

Mon, 01 Jan 0001 00:00:00 +0000

Chapter 5: WAL and Backend Snapshot Storage #

In the previous chapter, Cluster Membership and Peer Communication, you learned how etcd members join a cluster and talk to each other on port 2380. But what happens when an etcd member crashes and restarts? How does it remember all of the keys and values you wrote?

The answer is etcd’s storage layer. It is made of two important pieces that work together:

Mon, 01 Jan 0001 00:00:00 +0000

Chapter 6: Data Integrity and Corruption Detection #

In the last chapter, WAL and Backend Snapshot Storage, you saw how etcd stores every Raft entry in a Write-Ahead Log (WAL) and how it applies those entries to a backend database. But how does etcd know that the backend database has not been corrupted? How can you prove that a snapshot backup is valid? What happens when one etcd member secretly has different data than the others?

Mon, 01 Jan 0001 00:00:00 +0000

Chapter 7: Security and Trust Boundaries #

Welcome back! In the previous chapter, Data Integrity and Corruption Detection, we saw how etcd uses hashes and consistent indexes to detect when data on disk is corrupted. Now we ask an even earlier question: should that data have been allowed to reach etcd in the first place?

That is the job of Security and Trust Boundaries.

Mon, 01 Jan 0001 00:00:00 +0000

Chapter 8: Feature Gates and Release Stages #

Welcome back! In the previous chapter, Security and Trust Boundaries, we learned how etcd protects the two main “gates” into a cluster: port 2379 for clients and port 2380 for peers. mTLS makes sure only trusted actors can get inside.

But there is another kind of gate inside etcd itself: a feature gate. It controls which new features are visible, active, and trustworthy. Feature gates are how etcd manages code that is still experimental while keeping stable users safe.

Mon, 01 Jan 0001 00:00:00 +0000

Chapter 9: Multi-Module Go Repository #

Welcome back! In the previous chapter, Feature Gates and Release Stages, we saw how etcd introduces new features carefully, hiding them behind gates until they are ready. But where does the code for those features actually live? If you open the etcd repository, you won’t find one giant Go module. Instead, you’ll find a monorepo with several Go modules — and each one has a special job.