Chapter 1: gRPC Key-Value API and clientv3 #

Welcome! If you are new to etcd, you are in the right place. etcd is a distributed, reliable key-value store. That means it stores simple key -> value pairs and makes them available to many machines.

Imagine you are building a small web app. You want all of your servers to display the same dashboard background color. You could edit a config file on every server, but that gets messy. Instead, you store one key called dashboard/color in etcd. Your Go application uses the official Go client, clientv3, to save and read that value.

In this chapter, we will look at the front door of etcd: the gRPC Key-Value API and the official Go client, clientv3.

Why Do We Need This Abstraction? #

etcd’s contract with applications is a gRPC API on port 2379. That API defines operations like:

• Put — store a value under a key.
• Get — read the value stored under a key.
• Delete — remove a key.

But nobody wants to manually build gRPC connections, serialize protobuf messages, handle retries, and manage deadlines every time. That is where clientv3 comes in.

Here is a simple way to think about it:

• The API defines the forms you need to fill out.
• gRPC transports those forms over the network.
• clientv3 is the helpful clerk who fills out the forms for you and hands you the result.

So when you call cli.Put(...), you are really saying:
“Please fill out a PutRequest form, send it to etcd over gRPC, wait for the response, and tell me what happened.”

Key Concepts #

1. The Key-Value API #

The etcd Key-Value API is a set of operations for working with keys and values. A key can be something like dashboard/color, and a value can be something like "blue".

This is similar to a coat check:

• You hand over your coat and receive a ticket.
• Later, you hand over the ticket and get your coat back.

In etcd, the ticket is the key, and the coat is the value.

2. gRPC #

gRPC is a modern remote procedure call framework. It lets one program call a function on another program as if it were local. etcd uses gRPC on port 2379 for client requests.

gRPC also uses Protocol Buffers (protobuf) to serialize data. Protobuf is a compact, structured way to turn Go structs into bytes and back.

A simplified version of the API definition looks like this:

service KV {
  rpc Range(RangeRequest) returns (RangeResponse);
  rpc Put(PutRequest) returns (PutResponse);
}

This is the contract. etcd promises: “If you send me a PutRequest, I will try to store your key-value pair and return a PutResponse.”

3. clientv3 #

clientv3 is the official Go client for etcd. It wraps the gRPC API so you can write simple Go code.

Under the hood, clientv3 takes care of:

• Creating and managing gRPC connections.
• Setting timeouts and deadlines.
• Retrying requests when appropriate.
• Serializing requests and responses.
• Handling multiple endpoints.

4. The api Module #

The shared protobuf messages live in the api module: go.etcd.io/etcd/api/v3.

This is important because both clients and servers use the same definitions. Since protobuf messages are designed to evolve carefully, the client and server can change over time without breaking the wire protocol.

For example, a proto message for a Put request looks a bit like this:

message PutRequest {
  bytes key = 1;
  bytes value = 2;
}

The field numbers 1 and 2 are part of the contract. That means older clients and newer servers can usually still understand each other.

etcd is also organized as a multi-module Go repository. You can learn more in the Multi-Module Go Repository chapter.

Solving Our Use Case: Save and Read a Color #

Let’s use clientv3 to store and retrieve dashboard/color = "blue".

First, make sure etcd is running. Open a terminal and run:

etcd

This starts etcd listening on port 2379 for client requests.

Now, install the Go client:

go get go.etcd.io/etcd/client/v3

Next, create a client connection in Go:

cli, err := clientv3.New(clientv3.Config{
    Endpoints: []string{"localhost:2379"},
})
if err != nil {
    panic(err)
}
defer cli.Close()

This creates a client that will talk to etcd on localhost:2379. If etcd is not running, the client will fail to connect and err will tell you what went wrong.

Now let’s store the color:

ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()

resp, err := cli.Put(ctx, "dashboard/color", "blue")
if err != nil {
    panic(err)
}
fmt.Printf("write succeeded at revision %d\n", resp.Header.Revision)

Here, we tell clientv3 to send a Put request. The context gives the request a 2-second deadline so it cannot hang forever. If the write succeeds, the response includes a revision, which is a version number for the store.

The output might look like:

write succeeded at revision 42

Your actual revision number will be different. What matters is that etcd now has the key-value pair.

Now let’s read it back:

getResp, err := cli.Get(ctx, "dashboard/color")
if err != nil {
    panic(err)
}
fmt.Printf("value=%s\n", getResp.Kvs[0].Value)

The output will be:

value=blue

That is the whole use case: write a value, read a value.

If the key did not exist, getResp.Kvs would be empty. In a real application, you should check len(getResp.Kvs) first.

What Happens Under the Hood? #

When you call cli.Put(...), a lot happens behind the scenes. Here is a simplified view:

Step by step:

1. Your Go application calls cli.Put(...).
2. clientv3 creates a protobuf message called PutRequest, containing the key and value.
3. The gRPC library sends that message to the etcd server on port 2379.
4. The etcd server accepts the request and stores the key-value pair in its replicated key-value store.
5. The server returns a PutResponse with metadata such as the new revision.
6. clientv3 decodes the response and returns it to your program.

The “store the key-value pair” step is actually very interesting. etcd uses the Raft consensus algorithm to make sure a majority of cluster members agree on the data. You will learn more in the Raft Consensus Engine chapter.

A Quick Look at the Code #

If you want to explore the code in the etcd repository, here are good starting places:

• client/v3/kv.go — defines the KV interface with methods like Put, Get, and Delete.
• api/etcdserverpb/rpc.proto — defines the gRPC service and protobuf messages.
• server/etcdserver/api/v3rpc/ — contains the server-side gRPC handlers.

Inside clientv3, a Put call eventually turns into something like this pseudo-code:

// Conceptual: clientv3.Put is a wrapper around the gRPC client stub.
_, err := kvClient.Put(ctx, &pb.PutRequest{
    Key:   []byte("dashboard/color"),
    Value: []byte("blue"),
})

On the server side, the gRPC handler receives the PutRequest and writes it into etcd’s storage. Again, this is a simplified pseudo-code version:

// Conceptual: server receives a PutRequest and applies it.
func (s *KVServer) Put(ctx context.Context, req *pb.PutRequest) (*pb.PutResponse, error) {
    rev := s.store.Put(req.Key, req.Value)
    return &pb.PutResponse{
        Header: &pb.ResponseHeader{Revision: rev},
    }, nil
}

Do not worry if these details feel abstract. The important idea is that clientv3 handles the messy parts of gRPC, while the server code handles the actual storage.

The storage layer also writes to a Write-Ahead Log (WAL) and periodically creates snapshots so etcd can recover after crashes. Those details are covered in WAL and Backend Snapshot Storage.

Conclusion #

In this chapter, you learned that:

• etcd exposes a gRPC Key-Value API on port 2379.
• The API defines operations like Put and Get.
• clientv3 is the official Go client that makes talking to etcd easy.
• The api module holds the shared protobuf definitions.
• A simple Put and Get can be done in just a few lines of Go.

Now that you know how applications talk to etcd, the next chapter shows you how humans talk to etcd from the terminal.

Continue to etcdctl / etcdutl Command-Line Tools.

Generated by AI Codebase Knowledge Builder