Feature Flag / A-B Testing Delivery Analysis Note #

This note captures the full step-by-step analysis for a feature flag and A/B testing delivery system: flag config, targeting rules, experiment allocation, config propagation, local evaluation, and optional exposure logging.

Step 1 — Normalize #

Assume the baseline prompt is:

• design a feature flag / A-B testing delivery system
• applications evaluate flags locally or through a low-latency service
• admins update flags, rollout percentages, and targeting rules
• experiments assign users to variants deterministically
• system scales across many services/clients

Notes on normalization:

• evaluation is a read path against current effective evaluation state
• flag and targeting config are overwrite-state control-plane objects
• effective evaluation state is a control-plane derived object
• config propagation is async dissemination
• exposure logging is append-only, but secondary unless explicitly required for billing/analytics correctness

This is primarily a:

• Control Plane + Data Plane system

with optional secondary analytics/event-stream behavior.

Step 2 — Critical Path Selection #

Critical paths:

• P1 evaluate flag
• P2 update flag config
• P3 update targeting policy
• P4 compute effective evaluation state
• P5 propagate config snapshot

Optional/secondary:

• P6 exposure logging

Step 3 — Primary State Extraction #

Minimal primary set:

• FlagConfig
• TargetingPolicy
• EvaluationState
• ConfigSnapshot

Important modeling choices:

EvaluationState #

This is worth making explicit because:

• it is the effective compiled form used by the hot path
• it can include normalized targeting rules, percentage rollout config, variant weights, kill-switch flags, prerequisites, etc.

ConfigSnapshot #

Like other control-plane/data-plane systems:

• local SDKs or edge nodes should evaluate from a versioned snapshot
• not query control plane for every request

ExposureEvent #

Usually secondary in baseline correctness:

• important for experiment analytics
• not needed to decide variant selection if assignment is deterministic from config + context

Step 4 — Hard Invariants #

If experiment assignment is deterministic, add:

What matters most:

• clients must not evaluate against arbitrarily stale config
• deterministic bucketing must be stable for same subject and config version
• config versions must move monotonically
• effective state must be a faithful compiled form of authoritative rules

Step 5 — Execution Context #

Derived:

• bounded_staleness_allowed = true
• exclusive_claim_required = false
• guarded_by_current_state = true

This implies:

• authoritative control plane
• versioned snapshot propagation to evaluators
• local evaluation on hot path

Step 6 — Deterministic Mechanism Selection #

Hot path P1 is a read path.

Why these fit:

• flag config and targeting are versioned current-state config
• effective evaluation state is a recomputed current view
• snapshots are versioned overwrite propagation
• exposure logging, if included, is append-only

Step 7 — Read Model / Source of Truth #

Important point:

For the hot evaluation path:

• SDK/edge/service reads local ConfigSnapshot
• not control-plane config source on every request

That is the key control-plane/data-plane split for feature-flag delivery.

Step 8 — Failure Handling #

What matters most:

• evaluators reject older config versions
• same subject + config version yields stable assignment
• stale snapshots are bounded
• exposure logging, if used, is idempotent enough for analytics

Step 9 — Scale Adjustments #

What scales well:

• hot path is local evaluation from snapshot
• control plane is narrow and versioned
• propagation is incremental

What fails first:

• config fanout storms
• large rule trees
• exposure analytics overload if tied too closely to the hot path

Canonical design conclusion:

• archetype: Control Plane + Data Plane
• primary truth:
  • FlagConfig
  • TargetingPolicy
  • EvaluationState
  • ConfigSnapshot
• hot path:
  • local snapshot read + deterministic evaluation/bucketing
• control plane:
  • authoritative config + compiled effective state + snapshot publication

Concrete Substrate #

• control plane in Go/Java
• authoritative config store in etcd, Postgres, or another strongly consistent config DB
• config distribution via watch streams / streaming SDK updates / CDN-backed snapshot pull
• local evaluators in SDKs, edge nodes, or a low-latency gateway service
• optional exposure event pipeline via Kafka/PubSub/ClickHouse path

Operation Layer #

1. Evaluate(flag_id, context, subject_key)

• entry point: SDK / edge evaluator / local client library
• authoritative decider: local ConfigSnapshot
• transition: none on source truth
• response: {enabled, variant, config_version, reason}

2. PutFlagConfig(flag_id, config, expected_version?)

• entry point: control-plane API
• authoritative decider: config store owner
• transition: overwrite FlagConfig

3. PutTargetingPolicy(scope, config, expected_version?)

• entry point: control-plane API
• authoritative decider: policy store owner
• transition: overwrite TargetingPolicy

4. internal recompute

• recompute EvaluationState from flag config + policy

5. snapshot propagation

• publish latest ConfigSnapshot(version) to SDKs/edge nodes

6. RecordExposure(flag_id, subject_key, variant, config_version, request_id?)

• entry point: async event pipeline
• authoritative decider: analytics/event pipeline, not hot-path evaluator
• transition: append ExposureEvent

Entry Point vs Decider vs Responder #

Concrete HLD #

Main components:

• control-plane API
• flag + policy state store
• effective-state compiler/recompute worker
• snapshot distribution layer
• SDK / edge evaluators
• optional exposure analytics pipeline

Short interview version #

“I’d design the feature-flag and A/B testing system as a control-plane/data-plane setup. Control plane stores flag definitions and targeting rules, compiles them into an effective evaluation state, and publishes versioned snapshots to SDKs or edge evaluators. The hot path does not query control plane; it evaluates locally from the snapshot and uses deterministic bucketing so the same user gets the same variant for a given config version. Exposure logging is asynchronous and separate from the evaluation hot path.”