HLD Diagram Discipline Cheat Sheet #

Status: Archive candidate. Keep as historical reference; prefer system-design-core-index.md and the core notes for day-to-day use.

Prefer the infra-specific or core-index-guided notes instead of treating this as primary.

Use this note to draw cleaner interview diagrams.

The main rule:

Draw services around responsibilities for critical paths, then attach the stores and async pipelines they need.

Do not start from random boxes like User Service, DB, Cache, Kafka.

1. Start from paths, not boxes #

Before drawing anything, identify:

1. the 2-4 most important user paths
2. any hidden system paths that matter for correctness

Examples:

1. hold seat
2. buy seat
3. view seat map
4. expire stale hold

Why:

1. paths tell you what the system must do
2. services exist to own responsibilities on those paths

2. Group paths into service responsibilities #

Turn paths into a small set of responsibilities.

Examples:

1. Inventory Service
2. Booking Service
3. Search Service
4. Feed Service
5. Notification Worker
6. Scheduler / Expiry Worker

Good box test:

1. which path hits this box?
2. what truth does it own?
3. is it sync or async?

If you cannot answer those, the box is probably fake.

3. Draw the sync path first #

Always draw the request-response path before async machinery.

Typical order:

1. Client
2. API / App Service
3. Core Domain Service
4. Primary Store

Example:

1. Client
2. Booking API
3. Inventory/Booking Service
4. Primary DB

Why:

1. interviews care first about correctness on the main path
2. async details are easier once the sync path is clear

4. Put source-of-truth stores directly under the owning service #

For each service, attach the canonical store it owns.

Examples:

1. Follow Service -> Relation Store
2. Booking Service -> Booking DB
3. Inventory Service -> Inventory DB
4. Document Service -> Document Store

Rule:

1. source truth should be obvious in the diagram
2. derived stores should be drawn later and separately

5. Add async edges only after the source truth path is clear #

Now add:

1. outbox
2. queue
3. event bus
4. worker
5. scheduler
6. fanout pipeline

Typical flow:

1. source write commits
2. event or outbox record is produced
3. worker consumes event
4. worker updates projections / sends notifications / triggers side effects

This is where:

1. feed fanout
2. search indexing
3. counter maintenance
4. expiry scanning
5. notification delivery

usually appear.

6. Draw derived stores separately from source truth #

Keep these in a separate visual lane:

1. cache
2. search index
3. feed store
4. leaderboard store
5. analytics store
6. dashboard store

Reason:

1. source truth and projection should not be visually conflated
2. this makes consistency tradeoffs easier to explain

Good pattern:

1. source truth in middle / left
2. queue in middle
3. projections on right

7. Add external systems last #

Only after the internal flow is stable, add:

1. payment provider
2. email provider
3. push notification provider
4. CDN
5. object storage
6. auth provider

Why:

1. external systems are side effects or dependencies
2. they are rarely the starting point of the design

8. Keep service boxes responsibility-oriented #

Good service names:

1. Inventory Service
2. Booking Service
3. Feed Service
4. Search Indexer
5. Notification Service
6. Scheduler

Bad service names:

1. Everything Service
2. Database Service
3. User Service if it owns unrelated responsibilities
4. one service box per entity with no behavior

Use verbs or responsibility nouns, not schema nouns.

9. Use one diagram lane per role #

A clean interview diagram usually has 4 lanes:

1. clients / entrypoints
2. synchronous domain services
3. source-of-truth stores
4. async workers + projections + external systems

This keeps the board readable.

10. Show ownership, not every network hop #

You do not need to draw:

1. service mesh
2. load balancer internals
3. every internal RPC
4. every cache invalidation call

Only show what matters for:

1. correctness
2. performance
3. failure handling
4. scaling

11. Use arrows to show semantics, not decoration #

Different arrows should imply something:

1. sync request/response
2. async event or queue
3. projection update
4. external side effect

If speaking, say it clearly:

1. this write is synchronous
2. this projection update is asynchronous
3. this external notification is retried

12. Annotate API paths directly on the diagram #

Do not leave the main arrows unnamed.

Every important sync path should be annotated with:

1. method + endpoint or operation name
2. short intent
3. optional consistency or idempotency note

Good examples:

1. POST /holds [idem key]
2. POST /payments/{id}/capture
3. GET /feed/home [eventual]
4. PUT /docs/{id}/ops [versioned]

If you do not want full HTTP detail, use operation labels:

1. create_hold
2. confirm_booking
3. append_message
4. search_nearby

Best practice:

1. put the path label above the arrow
2. put the consistency / idempotency hint in brackets
3. label only the 2-4 paths that actually matter

Do not annotate every edge.

The goal is:

1. make the critical request paths visible
2. tie APIs to boxes
3. make later deep dives easier

13. Annotate stores and boxes with ownership and truth #

Each important box should answer one of these visually:

1. source of truth
2. derived projection
3. cache
4. async worker
5. external side effect

Useful suffixes:

1. Booking DB [truth]
2. Feed Store [projection]
3. Search Index [projection]
4. Redis Cache [cache]
5. Notification Worker [async]
6. PSP [external]

Useful box notes:

1. partitioned by user_id
2. strong write path
3. eventual read model
4. idempotent consumer

This prevents the classic interview confusion:

1. what is authoritative?
2. what can be stale?
3. what can be rebuilt?

14. Annotate the mechanism on the diagram #

For mechanism-bearing archetypes, the coordination primitive is the most important thing to make visible. Without it, an auction diagram and a CRUD diagram look identical.

Add a small annotation next to the relevant store or arrow:

Rule: if the mechanism is the load-bearing part of the design, it must appear on the diagram. A box labeled Booking Service -> DB with no annotation says nothing about how double booking is prevented.

15. Draw the read path explicitly #

The write path and read path are often different shapes. Draw them separately if they diverge.

Cache hit/miss path #

Only draw the miss path if it matters for the design:

Client -> Cache [HIT: return] [MISS: -> DB -> populate cache -> return]

Read replica #

Label which queries go to replica vs primary:

Write path -> Primary DB
Read path  -> Read Replica [bounded lag]

Fan-out on write vs fan-out on read #

These produce fundamentally different diagram shapes:

Fan-out on write:

Write Service -> DB -> Event Bus -> Fanout Worker -> Feed Store (per user)
Read path: Client -> Feed Store [precomputed]

Fan-out on read:

Write Service -> DB
Read path: Client -> Feed Service -> scatter-gather(Follow Store + Post Store) -> merge -> return

State which model you are using and why. Fan-out on write is cheaper to read, expensive to write, and amplifies under high-follower-count users. Fan-out on read is cheaper to write but expensive to read at high follower counts.

Scatter-gather #

When a query must fan out across N shards:

Client -> Query Service -> [Shard 1, Shard 2, ... Shard N] -> merge -> return

Label the merge step and state its latency dependency.

16. Write failure modes next to the path where they occur #

Do not keep all failure talk verbal.

Annotate the diagram with short failure tags near the boxes or edges that own the risk.

Good annotation style:

1. F1 duplicate hold
2. F2 stale cache read
3. F3 retry after timeout
4. F4 webhook side effect succeeds but ack lost

Then write a one-line mitigation nearby:

1. idem key + unique constraint
2. TTL + invalidate on source write
3. retry with backoff
4. outbox + reconciliation

Good places to mark failures:

1. before commit
2. after commit before publish
3. during external side effect
4. during async projection update
5. during lease / hold expiry

This makes the diagram defendable under questioning.

A strong pattern is:

1. F1 near the edge
2. M1 near the mitigation box

Example:

1. Booking API -> Booking DB : POST /book
2. F1 duplicate submit
3. M1 idem key

17. Annotate scalability bottlenecks where they originate #

Do not say only “this scales horizontally.”

Mark the stress point on the diagram.

Use short scale tags:

1. S1 hot inventory row
2. S2 fanout amplification
3. S3 hot search shard
4. S4 projection lag
5. S5 cold-start pressure
6. S6 origin read miss storm

Then add the first mitigation beside it:

1. bucket by event_id
2. partition by user_id
3. precompute feed
4. bounded freshness SLA
5. warm pool
6. request coalescing

Useful targeted annotations:

Best rule:

1. annotate the bottleneck on the edge or box where load concentrates
2. annotate the mitigation on the box that absorbs or spreads the load

Never annotate a box with scale numbers just to seem thorough. Only annotate when the number explains a design choice.

18. Use a small, explicit diagram legend #

If the diagram is non-trivial, spend 10 seconds defining notation.

Suggested legend:

1. solid arrow = sync request
2. dashed arrow = async event / queue
3. [truth] = source of truth
4. [projection] = derived read model
5. F# = failure mode
6. S# = scale hotspot

This prevents the interviewer from guessing what your arrows mean.

19. Canonical drawing sequence #

Use this order every time:

1. write down main paths
2. draw client
3. draw main synchronous service
4. draw primary store
5. draw second core service if the path truly splits responsibilities
6. annotate the mechanism on the write path
7. draw async queue/outbox
8. draw workers
9. draw projection stores
10. draw read path if it diverges from write path
11. draw external systems
12. add scale annotation on the bottleneck component only
13. circle the part you will deep dive into

This sequence prevents messy diagrams.

20. Default skeletons - product systems #

CRUD / entity system #

Client -> API Service -> Primary DB
                     -> Cache [read path]
                     -> Search Indexer -> Search Index [async]

Social / relation system #

Client -> Relation Service -> Relation Store [forward + reverse index]
                          -> Counter Worker [async] -> Counter Store
                          -> Feed/Recommendation Service [read path: scatter-gather]

Workflow / transaction system #

Client -> Workflow Service -> Primary DB [CAS on status+version]
                          -> Outbox [inside DB] -> Queue -> Worker -> External Provider
                                                         -> Reconciliation Job

Inventory / hold / booking system #

Client -> Inventory Service -> Inventory DB [CAS on availability]
                           -> Hold Store [Redis SETNX + TTL] -> Expiry Worker
                           -> Booking DB [guarded confirm]

Search / feed / ranking system #

Write Service -> Source DB -> Event Bus -> Indexer/Fanout Worker -> Projection Store / Search Index
                                                                -> Feed Store (fan-out on write)
Query Service <- Projection Store [read path]

Collaborative / realtime system #

Client A -+
Client B -+-> Operation Coordinator [per document] -> Operation Log DB [append]
Client C -+         -> broadcast
              Connected Clients [WebSocket]
              Snapshot Store [periodic; rebuilt from log]

File sync system #

Client -> Sync Service -> Namespace DB [CAS on version] -> Conflict Object [on version mismatch]
                      -> Block Store [S3; content-addressed by SHA-256]
                      -> Sync Cursor [derived; rebuilt from namespace history]
Delta sync read path: Client -> Sync Service -> Namespace DB [delta from last_version]

Matching / assignment system #

Request State DB [guarded transitions] <-> Assignment Service
Candidate Pool [Redis; eligibility index] <-> Assignment Service
Assignment Record [Lease: SETNX + TTL + fencing token]
Execution State DB [state machine] <- Worker

Crawler / frontier system #

Seed URLs -> Frontier Store [PostgreSQL; dedup by canonical URL]
Worker Pool -> claim via FOR UPDATE SKIP LOCKED -> fetch -> parse
           -> discovered URLs -> Frontier Store [dedup]
           -> Result Store [content-addressed]

Critical transaction with saga #

Client -> Orchestrator Service -> Local DB [CAS on status+version] + Outbox
                              -> Service A [idempotent step] <- compensation -+
                              -> Service B [idempotent step] <- compensation -+
                              -> External Provider                            |
Compensation Worker [on failure: replay compensations in reverse order] ------+

21. Default skeletons - infrastructure systems #

Messaging / streaming system #

Producer -> Topic/Partition [append log; partitioned by key]
         <- Consumer Group [pull; offset per partition per group]
         -> Offset Store [committed offsets]
         -> Dead Letter Queue [unprocessable messages]
Admin API -> Topic Metadata [partition count, retention, replication]

Key-value / cache infrastructure #

Client -> Router [consistent hash on key] -> Partition Node [primary]
                                         -> Replica Nodes [async replication]
Eviction Worker [LRU/TTL sweep per node]
Control Plane [partition map; rebalance on node join/leave]

Rate limiter #

Client -> Rate Limiter Service -> Counter Store [Redis: SETNX + sliding window or token bucket]
                              -> Policy Store [limits per key/tenant]
[ALLOW: pass through] [DENY: 429 + retry-after header]

Coordination / consensus store #

Client -> Leader Node [Raft; all writes go to leader]
        -> Follower Nodes [replicated; reads allowed with staleness flag]
Watch Registration -> Leader -> Watch Fanout [notify all watchers on key change]
Lease Store [TTL leases; auto-delete keys on lease expiry]

CDN / edge delivery #

Origin -> Origin Shield [single PoP; buffers all edges from direct origin hits]
       -> Regional PoP [mid-tier cache]
       -> Edge PoP [serves client]
Client -> Edge PoP [HIT: return] [MISS: -> Regional -> Shield -> Origin -> populate down]
Purge API -> Origin -> propagate invalidation to Shield -> Regional -> Edge [bounded lag]

Control plane + data plane #

Admin API -> Control Plane Store [versioned configs; CAS on version]
          -> Propagation Layer [push delta to agents or agents long-poll]
          -> Agent [local snapshot; applied_version tracked]
Data Plane [serves traffic using local snapshot; never calls control plane on hot path]
Health Reports -> Control Plane [agents report applied_version + health signals]
Rollout Controller [advance % or rollback based on health signals]

22. Diagram evolution over the interview #

Do not draw everything at once. Evolve the diagram in three passes:

Pass 1 - baseline (first 10 minutes): Draw the sync write path only. One client, one service, one primary store. Annotate the mechanism. This establishes correctness before complexity.

Pass 2 - read path and async (next 10 minutes): Add the read path if it diverges. Add outbox, queue, workers, projection stores. Label async boundaries explicitly.

Pass 3 - deep dive expansion (remaining time): Expand only the part the interviewer wants to go deeper on. Add scale annotations, replicas, caches, external systems. Do not expand the parts not being discussed.

Rule: every box added in pass 2 and pass 3 must be justified by a path or a failure mode. If you cannot state why a box was added, remove it.

23. When to split a service #

A common failure is either splitting too early or too late.

Split a monolith into two services only when at least one of these is true:

1. different correctness scopes
2. different scaling axes
3. different failure tolerance
4. different ownership

Do not split because:

1. the names are different
2. you want to seem thorough
3. microservices feel more modern

When you do split, state which of the four reasons applies. This turns a diagram decision into a derived choice.

24. Questions to ask for every box #

For each box in your diagram, be able to answer:

1. which path uses this box?
2. what data does it own or serve?
3. is it source truth or projection?
4. is it sync or async?
5. why can this not be merged into another box at this scale?

If you cannot answer these, simplify the diagram.

25. What to say while drawing #

Use lines like:

1. I'll start with the main write path.
2. This service owns the canonical booking state.
3. After the source write commits, I publish to an async pipeline.
4. These read-heavy queries come from projections, not the source store.
5. This worker exists because freshness can lag but correctness cannot.
6. I'm annotating CAS here because this is where the exclusivity invariant is enforced.
7. I'll split this into two services because the read path and write path have different scaling requirements.
8. I'm adding this box now because the deep dive is on fanout, not because every design needs it.

This makes the diagram feel intentional and derived, not assembled from memory.

26. Common mistakes #

1. drawing Kafka, Redis, and Elasticsearch before naming the path
2. drawing one box per entity with no behavior
3. mixing source truth and projections in the same store box
4. drawing too many microservices too early without stating why
5. failing to show async boundaries
6. failing to show which store is canonical
7. drawing infrastructure instead of responsibilities
8. no mechanism annotation on CAS or lease paths
9. drawing the write path and read path as the same arrow
10. adding scale annotations everywhere instead of only on the bottleneck
11. splitting services speculatively

27. Interview one-liner #

I draw HLDs in three passes: sync write path first with mechanism annotated, then read path and async pipeline, then deep dive expansion on the bottleneck. Every box must answer which path uses it, what truth it owns, and why it cannot be merged. Service splits require an explicit reason.