Chapter 4: Update Encoding / Decoding #

Welcome back! In the last chapter, Transaction, we saw how Yjs groups changes into batches and produces one update. But what is that update? It’s a binary message that can be sent over a network. This chapter is about how Yjs packs changes into that message, and how another Yjs document unpacks them.

Imagine you and a friend both have a suitcase full of shared notes. When you edit a note, you don’t send your friend the entire suitcase. You fold just the changed notes into a small package, send it over, and your friend unfolds it and puts it into their own suitcase. That is exactly what Yjs does with updates.

The problem: how do changes travel? #

Let’s look at a simple example.

Alice and Bob are working on a shared profile map. Alice changes her copy:

const aliceDoc = new Y.Doc()
const profile = aliceDoc.getMap('profile')
profile.set('name', 'Ada')

Bob has his own empty Doc:

const bobDoc = new Y.Doc()

How does Bob’s Doc learn that Alice’s profile now says 'Ada'? Alice’s Doc must encode the change into bytes, send those bytes to Bob, and Bob’s Doc must decode them.

Here is the first half:

const update = Y.encodeStateAsUpdate(aliceDoc)
console.log(update)
// Uint8Array(22) [0, 1, 2, ...]

update is a compact binary Uint8Array. It is not a JavaScript object. It is not JSON. It is a packed suitcase.

Here is the second half:

Y.applyUpdate(bobDoc, update)
console.log(bobDoc.getMap('profile').get('name'))
// 'Ada'

Bob’s Doc successfully unpacked Alice’s message. The profile is now synchronized.

In the rest of this chapter, we’ll open the suitcase and see how Yjs does this.

Key concept: an update is a binary message #

An update is a sequence of bytes that describes a change to a Yjs document. It does not contain JavaScript objects or text strings directly. It contains encoded structs — the small building blocks that Yjs uses to remember shared data.

You will see updates in two common places:

• doc.on('update', update => ...)
  The update event fires when a local or remote change is applied.
• Y.encodeStateAsUpdate(doc)
  This manually encodes the whole document state into one update.

Both produce a binary Uint8Array.

Why binary instead of JSON? Binary is smaller and faster. For a shared typing app, every keystroke might produce an update. You want those messages to be as small as possible.

Key concept: encoders and decoders #

An encoder writes data into a binary format. A decoder reads it back.

Think of an encoder as a friend who is very good at folding clothes into a tiny suitcase. They have a system:

• Write down the number of shirts.
• Write down the type of each shirt.
• Write down the order of the shirts.

A decoder is another friend who receives the suitcase and knows exactly how to unfold everything using that same system.

In Yjs, the important encoders and decoders live in:

• src/utils/UpdateEncoder.js
• src/utils/UpdateDecoder.js

UpdateEncoderV1 and UpdateEncoderV2 are the two main encoders. They both do the same job, but they use different “folding systems”.

Key concept: Update format V1 and V2 #

Yjs has two update formats:

• V1 is the original format. It is simpler and compatible with older Yjs versions.
• V2 is a newer format. It is more compact, which means smaller updates.

You don’t need to know every byte difference, but it is helpful to know the functions:

// V1
const updateV1 = Y.encodeStateAsUpdate(doc)
Y.applyUpdate(bobDoc, updateV1)

// V2
const updateV2 = Y.encodeStateAsUpdateV2(doc)
Y.applyUpdateV2(bobDoc, updateV2)

V2 is like a better folding method: it squeezes repeated patterns out of the suitcase. This makes V2 updates smaller, especially when the same client names or type names appear many times.

Key concept: ID-set encoders and decoders #

Sometimes Yjs doesn’t need to send actual content. It just needs to say which IDs are in a document.

An ID in Yjs is a pair: { client, clock }. For example, “client 7, clock 3” might identify the 4th item created by device 7. You’ll learn more about IDs in ID.

Sending a list of every single ID would be big. Instead, Yjs packs ranges of IDs. Imagine saying:

“I already have items 0 to 99 from client 7.”

That’s much shorter than listing 100 IDs.

Special classes do this job:

• IdSetEncoderV1 / IdSetEncoderV2
• IdSetDecoderV1 / IdSetDecoderV2

They are used when Yjs needs to describe a set of ID ranges, for example when working with snapshots and state vectors. We will see snapshots in the next chapter, Snapshot.

How to use updates in your app #

You usually don’t create UpdateEncoder objects yourself. Yjs does that for you.

Imagine Alice’s app sends updates over WebSocket:

aliceDoc.on('update', update => {
  websocket.send(update)
})

Bob’s app receives the update:

websocket.onmessage = event => {
  Y.applyUpdate(bobDoc, event.data)
}

That’s it. The update event gives you already-encoded bytes. Y.applyUpdate decodes them for you.

You can also send only what Bob is missing. This is useful when Bob joins after Alice has been editing for a while.

const missingUpdate = Y.encodeStateAsUpdate(aliceDoc, Y.encodeStateVector(bobDoc))
Y.applyUpdate(bobDoc, missingUpdate)

First, Y.encodeStateVector(bobDoc) creates a compact description of what Bob already knows. Then Y.encodeStateAsUpdate(aliceDoc, stateVector) encodes only the changes Alice has that Bob doesn’t.

That is a little like checking what is already in Bob’s suitcase before packing new clothes. This is one reason ID-set encoders matter.

What happens under the hood? #

Let’s trace what happens when Alice makes a change and Bob applies it.

1. Alice calls profile.set('name', 'Ada').
2. Alice’s Doc starts a transaction.
3. The transaction creates a struct and stores it in Alice’s StructStore.
4. When the transaction finishes, Yjs uses an UpdateEncoder to write that struct into a binary Uint8Array.
5. Alice sends the Uint8Array to Bob.
6. Bob’s Doc calls Y.applyUpdate.
7. Y.applyUpdate creates an UpdateDecoder.
8. The decoder reads the structs from the bytes.
9. The decoded structs are added to Bob’s StructStore.
10. Bob’s shared types update, and Bob’s observers fire.

Here is a simple picture:

The UpdateEncoder is like the packing machine. The UpdateDecoder is the unpacking machine.

A peek at the encoder source code #

Let’s look at a very simplified version of UpdateEncoderV1.

The real code in src/utils/UpdateEncoder.js is longer, but the idea is simple: it has methods for writing numbers, strings, and whole structs.

// simplified from src/utils/UpdateEncoder.js
class UpdateEncoderV1 {
  constructor() {
    this.encoder = createEncoder()
  }

  writeVarUint(value) {
    writeVarUint(this.encoder, value)
  }

  writeStruct(struct) {
    this.writeVarUint(struct.id.client)
    this.writeVarUint(struct.id.clock)
    // ... write the content
  }
}

writeVarUint writes an integer in a compact way. The encoder is building up a byte array. Each method adds more bytes to the suitcase.

The decoder is the mirror image:

// simplified from src/utils/UpdateDecoder.js
class UpdateDecoderV1 {
  constructor(update) {
    this.decoder = createDecoder(update)
  }

  readVarUint() {
    return readVarUint(this.decoder)
  }

  readStruct() {
    const client = this.readVarUint()
    const clock = this.readVarUint()
    // ... read the content
    return createStruct(client, clock)
  }
}

Every write has a matching read. That is what makes encoding and decoding work.

A peek at applyUpdate #

In Yjs source code, the function that decodes incoming updates is in src/utils/applyUpdate.js.

Here is a very simplified version:

// simplified from src/utils/applyUpdate.js
export const applyUpdate = (doc, update, origin) => {
  const decoder = new UpdateDecoderV1(createDecoder(update))
  readUpdate(decoder, doc, origin)
}

readUpdate is the part that reads the version, structs, and delete sets from the bytes.

For V2 updates, Yjs has a separate function:

// simplified from src/utils/applyUpdate.js
export const applyUpdateV2 = (doc, update, origin) => {
  const decoder = new UpdateDecoderV2(createDecoder(update))
  readUpdate(decoder, doc, origin)
}

If you mix them up, the decoder might try to read V2 bytes with a V1 decoder. That’s why Yjs provides separate functions for each format.

What about ID-set encoders? #

ID-set encoders are specialized suitcases for lists of ID ranges.

Imagine Yjs wants to describe this set of IDs:

• client 1, clocks 0 to 9
• client 2, clocks 0 to 4

That’s 15 individual IDs. Instead of writing all 15, an ID-set encoder can write:

// simplified idea: write range for each client
writeClientRange(1, 0, 9)
writeClientRange(2, 0, 4)

The matching decoder reads those ranges back:

// simplified idea
const client = readClient()
const start = readStart()
const end = readEnd()

This is much more compact. Snapshot formats use this idea to capture “all IDs at this point in time.”

Common beginner questions #

Do I need to understand byte formats to use Yjs? #

No. You can build complete apps with doc.on('update') and Y.applyUpdate. But knowing that updates are binary helps you debug network messages and understand why V2 updates are smaller.

Why is the update a Uint8Array and not a string? #

A Uint8Array is a list of byte values. It is exactly what you need when sending binary data over a network or storing it in a file.

Can I store an update in a database? #

Yes. You can store the Uint8Array as a binary blob. When you load the blob, you can apply it to a fresh Doc.

Conclusion #

Now you know how Yjs packs and unpacks changes.

• An update is a binary Uint8Array.
• An encoder turns Yjs structs into bytes.
• A decoder turns bytes back into Yjs structs.
• Yjs has V1 and V2 update formats.
• V2 is more compact.
• ID-set encoders and decoders compactly describe ranges of IDs.
• You don’t need to handle byte-level details in your own app; Yjs handles it for you.

The next natural question is: how can we capture a moment in a document’s history? That is what snapshots are for.

Continue to: Snapshot

Generated by AI Codebase Knowledge Builder