API Gateway Analysis Note #

This note captures the full step-by-step analysis for an API gateway: route config, auth/policy config, backend health, effective gateway state, and versioned snapshot propagation to serving nodes.

Step 1 — Normalize #

Assume the baseline prompt is:

• design an API gateway
• clients send API requests to one endpoint
• gateway authenticates, authorizes, rate-limits, and routes to backend services
• policies can change over time
• system scales across nodes

Notes on normalization:

Important choices:

• request routing is read source
  • the hot path mostly reads current routing state and makes a routing decision
• route config and policy config are overwrite state
• backend health is overwrite state
  • current health is the main truth
• effective gateway state update is a control-plane recompute transition
• config propagation is async process
  • control-plane to data-plane dissemination

This is clearly a:

• Control Plane + Data Plane system

Step 2 — Critical Path Selection #

Critical paths:

• P1 request handling
• P2 update route config
• P3 update policy config
• P4 record health
• P5 update effective gateway state
• P6 propagate config snapshot

Step 3 — Primary State Extraction #

Minimal primary set:

• RouteConfig
• PolicyConfig
• HealthState
• GatewayRoutingState
• ConfigSnapshot

Step 4 — Hard Invariants #

What matters most:

1. Route only under valid policy and health #

The core safety property is:

• gateway must not send requests using stale/invalid route, auth, or backend-eligibility state beyond the allowed bound

2. Effective gateway state is derived from authoritative inputs #

Route config, policy config, and health feed the hot-path serving state.

3. Propagation lag is bounded #

This is the key control-plane/data-plane correctness interface.

Step 5 — Execution Context #

For the API gateway baseline:

Derived:

• bounded_staleness_allowed = true
• exclusive_claim_required = false
• guarded_by_current_state = true

This pushes us toward:

• authoritative control plane
• versioned snapshots to data-plane nodes
• hot path reads from local or near-local routing snapshot

Step 6 — Deterministic Mechanism Selection #

Hot request path P1 is a read path, not a write-shape path.

Step 7 — Read Model / Source of Truth #

Important point:

For the hot path:

• data-plane node reads local ConfigSnapshot
• not control-plane state per request

So the serving read path is:

• local materialized view / config snapshot

Step 8 — Failure Handling #

What matters most:

1. Versioned snapshot propagation #

Serving nodes must reject older snapshots and move monotonically forward by config version.

2. Bounded stale routing #

The hot path is usually eventually updated, not strongly synchronized with every control-plane change.

3. Health flapping #

Health-state overwrite policy must handle noisy observations without causing excessive routing churn.

Step 9 — Scale Adjustments #

What scales well:

The system scales well if:

• the hot path is local snapshot read + policy/routing evaluation
• control plane is narrow and versioned
• propagation is incremental

What fails first:

Usually:

• health-flap storms
• config propagation bursts
• giant route/policy tables

Canonical design conclusion:

• primary truth:
  • RouteConfig
  • PolicyConfig
  • HealthState
  • GatewayRoutingState
  • ConfigSnapshot
• hot path:
  • local snapshot read + auth/policy check + backend selection
• control plane:
  • versioned config + recomputed effective state + snapshot publication

Concrete Substrate #

• control plane in Go/Java
• authoritative config/health store in etcd or similar strongly consistent store
• snapshot propagation via watch streams/pub-sub
• data-plane fleet using Envoy-like or custom proxy processes
• local snapshot cache per node for hot path

Operation Layer #

1. HandleRequest(listener, request) #

• entry point: data-plane node
• authoritative decider: local ConfigSnapshot
• transition: none on source truth
• response: proxied backend response or policy rejection

2. PutRouteConfig(route_id, config, expected_version?) #

• entry point: control-plane API
• authoritative decider: config store
• transition: overwrite RouteConfig, bump version

3. PutPolicy(policy_scope, config, expected_version?) #

• same shape as route config

4. ReportHealth(backend_id, health, observation_revision) #

• entry point: control plane
• authoritative decider: health-state owner
• transition: overwrite HealthState

5. internal recompute #

• recompute GatewayRoutingState from config + policy + health

6. snapshot propagation #

• push/pull latest ConfigSnapshot(version) to data-plane nodes

Entry Point vs Decider vs Responder #

Concrete HLD #

Main components:

• control-plane API
  • route config
  • policy config
  • health ingestion
• control-plane state store
  • route, policy, health, effective routing versions
• routing recompute worker
  • derives effective gateway state
• snapshot distribution layer
  • pushes/pulls versioned config to serving nodes
• data-plane gateway fleet
  • handles client traffic using local snapshots

Short interview version #

“I’d design the API gateway as a control-plane/data-plane system. Control plane owns route config, auth/policy config, backend health, and effective gateway state. Data-plane nodes don’t query control plane per request; they use versioned local snapshots to authenticate, enforce policy, and route to healthy backends. Config and health are updated in the control plane, effective state is recomputed there, and snapshots are propagated incrementally to the serving fleet.”