DynamoDB Realizations for Core Archetypes Cheat Sheet #

This note captures the DynamoDB-oriented realizations we discussed for key core archetypes:

1. relation / tagging
2. future constraint + claimable run
3. frontier + claimable run
4. time-bounded exclusive allocation
5. matching / assignment
6. auction / competitive window
7. critical transaction process

Each important action is shown in two layers:

• (abstract: write shape, base mechanism)
• (concrete: DynamoDB / queue / Redis realization)

This is a study note for derivation, not final spoken interview delivery.

1. Relation / Tagging #

Canonical entities #

• TagDefinitionState
• ContentTagUniqueState
• ContentTagByTagIndexState
• TagPopularityView

Recommended storage split #

• tags
  • canonical tag dictionary / metadata
• content_tag_unique
  • canonical edge truth
• content_tag_by_tag
  • bucketed read index for hot tags

Tag metadata #

• create tag -> TagDefinitionState

  • (abstract: create target, insert/create)
  • (concrete: DynamoDB PutItem on tags with PK=(tenant_id, tag_name_norm), conditional on attribute_not_exists(PK), generated tag_id as ULID/KSUID)

• read tag metadata -> TagDefinitionState

  • (abstract: read source)
  • (concrete: DynamoDB GetItem on tags by tenant_id + tag_name_norm or tag_id index)

Attach tag #

• attach tag to content -> ContentTagUniqueState

  • (abstract: state transition, guarded add/remove)
  • (concrete: DynamoDB PutItem on content_tag_unique with PK=(tenant_id, content_id, tag_id), ConditionExpression attribute_not_exists(PK))

• update reverse index by tag -> ContentTagByTagIndexState

  • (abstract: create target, bucketed relation index write)
  • (concrete: DynamoDB PutItem on content_tag_by_tag with PK=(tenant_id, tag_id_bucket), SK=(created_at DESC, content_id), bucket=hash(tag_id, content_id) mod B)

• publish relation change -> TagEvent

  • (abstract: create target, projection/update event)
  • (concrete: DynamoDB Streams event or explicit event publish to Kafka/Kinesis)

Detach tag #

• detach tag from content -> ContentTagUniqueState

  • (abstract: state transition, guarded add/remove)
  • (concrete: DynamoDB DeleteItem on content_tag_unique by PK=(tenant_id, content_id, tag_id))

• remove reverse index entry -> ContentTagByTagIndexState

  • (abstract: state transition, index maintenance)
  • (concrete: DynamoDB DeleteItem on content_tag_by_tag using tag bucket + content key)

Read content by tag #

• view content by tag -> ContentTagByTagIndexState
  • (abstract: read projection)
  • (concrete: parallel DynamoDB Query across all B buckets for tag_id, then k-way merge by created_at/content_id)

Popular tags #

• update popularity counts -> TagPopularityView
  • (abstract: overwrite, counter/projection update)
  • (concrete: DynamoDB Streams/Lambda or Kafka/Flink updates Redis sorted sets or aggregate table keyed by (tenant_id, tag_id, window))

Important notes #

1. content_tag_unique is the source truth
2. content_tag_by_tag is a repairable read index
3. bucketization spreads writes for hot tags but increases read fanout

2. Future Constraint + Claimable Run #

Example: job scheduler

Canonical entities #

• JobDefinitionState
• ExecutionState
• RunnableExecutionState

Create job #

• create job -> JobDefinitionState
  • (abstract: create target, insert/create)
  • (concrete: DynamoDB PutItem into Jobs table with job_id, schedule, parameters, user_id)

Materialize execution instance #

• create execution -> ExecutionState
  • (abstract: create target, insert/create future-constrained run)
  • (concrete: DynamoDB PutItem into Executions table with PK=time_bucket, SK=execution_time#job_id, status=PENDING, attempt=0)

Scan due work #

• discover near-term executions -> ExecutionState
  • (abstract: read source on due-time index)
  • (concrete: DynamoDB Query on current and next time_bucket partitions for PENDING executions within lookahead window)

Materialize claimable run #

• enqueue execution -> RunnableExecutionState
  • (abstract: create target, materialize claimable run)
  • (concrete: SendMessage to SQS with DelaySeconds and payload={execution_id, job_id, execution_time})

Claim run #

• claim execution -> RunnableExecutionState
  • (abstract: state transition, claim/lease)
  • (concrete: SQS ReceiveMessage; visibility timeout acts as lease)

Mark in progress #

• mark execution in progress -> ExecutionState
  • (abstract: state transition, guarded conditional update)
  • (concrete: DynamoDB UpdateItem set status=IN_PROGRESS, increment attempt, condition status IN {PENDING, RETRYING})

Complete or fail #

• mark completed -> ExecutionState

  • (abstract: state transition, guarded conditional update)
  • (concrete: DynamoDB UpdateItem set status=COMPLETED, completed_at=now, then SQS DeleteMessage)

• mark retrying/failed -> ExecutionState

  • (abstract: state transition, guarded conditional update)
  • (concrete: DynamoDB UpdateItem set status=RETRYING or FAILED; if retrying, SendMessage to SQS with DelaySeconds based on backoff)

Important notes #

1. future constraint lives in ExecutionState.execution_time
2. claim is realized by SQS visibility timeout, not by DynamoDB itself
3. ExecutionState adds workflow/lifecycle semantics on top of the future-constrained run

3. Frontier + Claimable Run #

Example: web crawler

Canonical entities #

• URLMetadataState
• URLFrontierState
• URLFetchRunState
• URLParseRunState
• DomainPolicyState
• URLArtifactState

Add seed URL to frontier #

• add seed URL -> URLFrontierState
  • (abstract: create target, frontier append)
  • (concrete: DynamoDB PutItem into URLMetadata table with pk=url_hash, status=DISCOVERED, depth=0; then SendMessage to SQS frontier queue)

Dedup discovered child URL #

• discover child URL -> URLFrontierState
  • (abstract: create target, deduped frontier expansion)
  • (concrete: DynamoDB PutItem with ConditionExpression attribute_not_exists(pk); on success, SendMessage to SQS frontier queue)

Claim frontier item for fetch #

• claim URL for fetch -> URLFetchRunState
  • (abstract: state transition, claim/lease)
  • (concrete: SQS ReceiveMessage from frontier queue; visibility timeout acts as claim lease)

Mark fetching #

• mark URL fetching -> URLMetadataState
  • (abstract: state transition, guarded conditional update)
  • (concrete: DynamoDB UpdateItem set status=FETCHING, increment fetch_attempts, condition status IN {DISCOVERED, RETRYING})

Persist fetched artifact #

• store raw HTML -> URLArtifactState
  • (abstract: create target, durable artifact write)
  • (concrete: PutObject to S3 with key=url_hash/raw.html)

Mark fetch complete #

• mark fetched -> URLMetadataState
  • (abstract: state transition, guarded conditional update)
  • (concrete: DynamoDB UpdateItem set status=FETCHED, raw_blob_key=..., fetched_at=now)

Materialize next-stage parse run #

• enqueue parse work -> URLParseRunState
  • (abstract: create target, next-stage claimable run)
  • (concrete: SendMessage to SQS parse queue with url_hash/blob_key payload)

Ack fetch claim #

• release fetch claim -> URLFetchRunState
  • (abstract: release claim / finalize run)
  • (concrete: SQS DeleteMessage on frontier queue receipt handle)

Claim parse run #

• claim parse run -> URLParseRunState
  • (abstract: state transition, claim/lease)
  • (concrete: SQS ReceiveMessage from parse queue; visibility timeout acts as lease)

Persist extracted text #

• store parsed text -> URLArtifactState
  • (abstract: create target, durable artifact write)
  • (concrete: PutObject to S3 with key=url_hash/text.txt or text.json)

Mark parsed #

• mark parsed -> URLMetadataState
  • (abstract: state transition, guarded conditional update)
  • (concrete: DynamoDB UpdateItem set status=PARSED, text_blob_key=..., parsed_at=now)

Expand frontier with linked URLs #

• enqueue extracted links -> URLFrontierState
  • (abstract: create target, deduped frontier expansion)
  • (concrete: for each extracted URL, conditional DynamoDB PutItem into URLMetadata; on success, SendMessage to frontier queue)

Reclaim stalled work #

• reclaim stalled fetch/parse run -> URLFetchRunState / URLParseRunState
  • (abstract: claim lease expiry, run becomes reclaimable)
  • (concrete: SQS visibility timeout expires; message becomes visible again for another worker)

Important notes #

1. the frontier is the set of discovered-but-not-yet-fully-processed URLs
2. claim is realized by queue visibility timeout, not DynamoDB itself
3. DynamoDB stores durable progress truth; SQS provides claimable execution

4. Time-Bounded Exclusive Allocation #

Example: Ticketmaster seat booking

Canonical entities #

• TicketInventoryState
• TicketHoldState
• BookingState

Reserve seat / create hold #

• create temporary hold -> TicketHoldState + TicketInventoryState

  • (abstract: state transition, guarded exclusive hold)
  • (concrete with pure DDB: conditional UpdateItem or TransactWriteItems changing ticket AVAILABLE -> HELD, setting hold_expires_at and booking_id, and creating BookingState with status=IN_PROGRESS)

• create temporary hold -> TicketHoldState + BookingState

  • (abstract: state transition, temporary lease + workflow create)
  • (concrete with DDB + Redis: Redis SET ticket:{ticket_id} {booking_id} NX EX 600, then DynamoDB PutItem for BookingState status=IN_PROGRESS)

Reject competing reserver #

• reject second reserve -> TicketInventoryState / TicketHoldState
  • (abstract: fail competing hold)
  • (concrete with pure DDB: conditional write fails because status is HELD or SOLD and hold_expires_at is valid)
  • (concrete with DDB + Redis: Redis SET NX fails because hold key exists)

Read availability #

• view seat availability -> TicketInventoryState
  • (abstract: read source)
  • (concrete with pure DDB: GetItem/Query on ticket item; business logic interprets expired HELD rows using hold_expires_at)
  • (concrete with DDB + Redis: read durable ticket state from DDB, optionally overlay current Redis hold state for freshness)

Expire hold #

• release stale hold -> TicketHoldState + TicketInventoryState
  • (abstract: state transition, timed release / expiry)
  • (concrete with pure DDB: sweeper queries expired HELD items and conditionally updates HELD -> AVAILABLE; BookingState -> EXPIRED)
  • (concrete with DDB + Redis: Redis TTL expires automatically; sweeper marks stale IN_PROGRESS BookingState as EXPIRED)

Confirm purchase #

• convert hold to committed booking -> BookingState + TicketInventoryState
  • (abstract: state transition, guarded confirm/commit)
  • (concrete with pure DDB: TransactWriteItems checks ticket HELD, booking_id matches, hold_expires_at >= now, then updates ticket HELD -> SOLD and booking IN_PROGRESS -> CONFIRMED)
  • (concrete with DDB + Redis: DynamoDB conditional update / transaction confirms booking and sets ticket SOLD after revalidating booking state; Redis hold may be deleted early or allowed to expire)

Important notes #

1. do not rely on DynamoDB TTL alone for business expiry
2. always keep an explicit hold_expires_at
3. with DDB + Redis, Redis is the lease layer, DynamoDB is the durable commit layer

5. Matching / Assignment #

Example: Uber

Canonical entities #

• RideRequestState
• DriverAssignmentState
• TripExecutionState
• DriverLocationState
• CandidateSetView

Create request #

• request ride -> RideRequestState
  • (abstract: create request, create target)
  • (concrete: DynamoDB PutItem into RideRequests with ride_id, rider_id, pickup, destination, fare_id, status=OPEN)

Compute candidate set #

• find nearby drivers -> CandidateSetView
  • (abstract: read projection)
  • (concrete: query geo index such as Redis GEO / H3-backed location service for nearby AVAILABLE drivers)

Propose assignment #

• propose driver -> DriverAssignmentState
  • (abstract: state transition, guarded assignment/claim)
  • (concrete: DynamoDB PutItem/UpdateItem creating assignment row/item with status=PROPOSED, driver_id, ride_id, offer_expires_at; optionally acquire Redis lock on driver_id with TTL to block simultaneous offers)

Accept assignment #

• accept driver assignment -> DriverAssignmentState + RideRequestState
  • (abstract: state transition, guarded conditional update)
  • (concrete: DynamoDB conditional UpdateItem or TransactWriteItems moving assignment PROPOSED -> ACCEPTED and ride OPEN/MATCHING -> MATCHED)

Reject assignment #

• reject driver assignment -> DriverAssignmentState
  • (abstract: state transition, guarded conditional update)
  • (concrete: DynamoDB UpdateItem PROPOSED -> REJECTED, then matching service advances to next candidate)

Expire assignment #

• expire driver offer -> DriverAssignmentState + RideRequestState
  • (abstract: state transition, timeout/reassignment)
  • (concrete: timeout worker scans offer_expires_at, sets assignment PROPOSED -> EXPIRED, keeps ride in MATCHING/open for next proposal)

Start trip execution #

• start trip -> TripExecutionState
  • (abstract: state transition, workflow start)
  • (concrete: DynamoDB UpdateItem or PutItem marking trip DRIVER_EN_ROUTE / ARRIVING and binding driver_id to ride_id)

Complete trip execution #

• advance/complete trip -> TripExecutionState
  • (abstract: state transition, guarded workflow transition)
  • (concrete: DynamoDB conditional updates through ARRIVED / PICKED_UP / IN_PROGRESS / COMPLETED)

Important notes #

1. clean matching archetype is not “fan out to many and take first”
2. canonical flow is:
   • create request
   • compute candidate set
   • propose assignment
   • accept/reject/expire
   • then start execution

6. Auction / Competitive Window #

Example: online auction

Canonical entities #

• AuctionState
• BidState
• BestBidState
• AuctionWinnerState

Create auction #

• create auction -> AuctionState
  • (abstract: create target, create competitive window subject)
  • (concrete: DynamoDB PutItem into Auctions table with auction_id, item_id, starting_price, end_time, status=OPEN)

Read current auction #

• view auction -> AuctionState
  • (abstract: read source)
  • (concrete: DynamoDB GetItem on Auctions table by auction_id)

Place bid #

• place bid -> BidState
  • (abstract: append, append competitive child event)
  • (concrete: DynamoDB PutItem into Bids table with PK=auction_id, SK=bid_time#bid_id, amount, bidder_id, created_at)

Validate against current best #

• read current best -> BestBidState

  • (abstract: read source on current best state)
  • (concrete: DynamoDB GetItem on BestBid table by auction_id)

• read auction status -> AuctionState

  • (abstract: read source on competitive window state)
  • (concrete: DynamoDB GetItem on Auctions table to verify status=OPEN and end_time not passed)

Update current highest bid #

• update current best -> BestBidState
  • (abstract: overwrite, guarded best-state update)
  • (concrete: conditional UpdateItem on BestBid table for auction_id, only if new amount > current highest and auction is still OPEN)

Reject invalid lower bid #

• reject invalid bid -> BidState / BestBidState
  • (abstract: fail guarded best-state update while preserving append history if desired)
  • (concrete: conditional update on BestBid fails because amount <= current highest; bid row may still be stored in Bids as REJECTED if full audit is desired)

Broadcast new highest bid #

• publish highest-bid update -> BestBidState
  • (abstract: create projection/fanout side effect)
  • (concrete: DynamoDB Streams + Lambda or pub/sub publishes the new highest bid to websocket/SSE subscribers)

Close auction #

• close auction -> AuctionState
  • (abstract: state transition, guarded close of competitive window)
  • (concrete: conditional UpdateItem on Auctions table changing status OPEN -> CLOSED only if end_time <= now and status still OPEN)

Commit winner #

• commit winner -> AuctionWinnerState
  • (abstract: create target or overwrite, commit competitive result)
  • (concrete: DynamoDB PutItem or UpdateItem into AuctionWinner table with auction_id, winning_bid_id, bidder_id, winning_amount after close succeeds and BestBidState is read)

Read auction history #

• view bid history -> BidState
  • (abstract: read source or read projection)
  • (concrete: DynamoDB Query on Bids table by auction_id ordered by bid_time#bid_id)

Important notes #

1. append-only BidState preserves auditability
2. BestBidState should be explicit rather than derived ad hoc on every read
3. AuctionState and AuctionWinnerState make the competitive window and final result explicit

7. Critical Transaction Process #

Example: payment processing system

Canonical entities #

• PaymentIntentState
• TransactionState
• ExternalAuthorizationAttemptState
• WebhookDeliveryState
• ReconciliationState

Create payment intent #

• create payment intent -> PaymentIntentState
  • (abstract: create target, create transaction workflow root)
  • (concrete: DynamoDB PutItem into PaymentIntents table with payment_intent_id, merchant_id, amount, currency, status=CREATED, created_at)

Start charge attempt #

• create charge attempt -> TransactionState
  • (abstract: create target, create money-movement attempt)
  • (concrete: DynamoDB PutItem into Transactions table with transaction_id, payment_intent_id, type=CHARGE, status=PENDING, amount, currency, created_at)

Persist idempotency key #

• record idempotency key -> IdempotencyState
  • (abstract: create target, dedup request intent)
  • (concrete: DynamoDB PutItem into Idempotency table with idempotency_key -> payment_intent_id / transaction_id, ConditionExpression attribute_not_exists(idempotency_key))

Move intent into processing #

• mark processing -> PaymentIntentState
  • (abstract: state transition, guarded workflow transition)
  • (concrete: DynamoDB UpdateItem on PaymentIntentState setting status=PROCESSING, condition status IN {CREATED, RETRYABLE})

Send authorization to external network #

• send authorization -> ExternalAuthorizationAttemptState
  • (abstract: state transition, external side effect / network attempt)
  • (concrete: optional DynamoDB PutItem into ExternalAuthorizationAttempts table with attempt_id, transaction_id, status=SENT; then call external card network / processor API)

Record immediate network response #

• record response -> TransactionState + PaymentIntentState
  • (abstract: state transition, guarded conditional update of workflow and attempt state)
  • (concrete: DynamoDB UpdateItem on TransactionState to AUTHORIZED / DECLINED / PENDING_NETWORK_RESULT; UpdateItem on PaymentIntentState to SUCCEEDED / FAILED / PENDING depending on response semantics)

Handle timeout / unknown outcome #

• mark uncertain outcome -> TransactionState + PaymentIntentState
  • (abstract: state transition, move into uncertain/pending state instead of assuming failure)
  • (concrete: DynamoDB UpdateItem on TransactionState and PaymentIntentState to PENDING_NETWORK_RESULT or REQUIRES_RECONCILIATION when external outcome is unknown)

Emit durable downstream event #

• emit event -> PaymentEventState
  • (abstract: create target, durable async event)
  • (concrete: DynamoDB Streams or outbox table emits changes for async consumers like webhook delivery and analytics)

Reconcile uncertain result #

• reconcile payment -> ReconciliationState + TransactionState + PaymentIntentState
  • (abstract: state transition, reconciliation against external source of truth)
  • (concrete: reconciliation worker reads PENDING/UNKNOWN transactions, queries external processor status, then conditionally updates DynamoDB TransactionState and PaymentIntentState)

Notify merchant #

• deliver webhook -> WebhookDeliveryState
  • (abstract: create target / state transition, async external notification)
  • (concrete: DynamoDB PutItem into WebhookDelivery table with status=PENDING; worker sends webhook and updates DELIVERED / RETRYING / FAILED)

Read payment status #

• view payment status -> PaymentIntentState
  • (abstract: read source)
  • (concrete: DynamoDB GetItem on PaymentIntents table by payment_intent_id)

Retry merchant request safely #

• retry payment request -> IdempotencyState + PaymentIntentState
  • (abstract: retry-safe dedup and result reuse)
  • (concrete: lookup idempotency_key in DynamoDB Idempotency table; if found, return existing payment_intent_id / transaction status instead of creating a second charge)

Important notes #

1. PaymentIntentState is the workflow root
2. TransactionState captures money-movement attempts
3. uncertain external outcomes should become explicit pending/reconciliation states
4. idempotency and reconciliation are first-class parts of the design

8. Shared Mutable Subject #

Example: Google Docs style collaborative editor

Canonical entities #

• DocumentState
• DocumentOperationLog
• DocumentVersionState
• CursorPresenceState
• DocumentSnapshotState

Create document #

• create document -> DocumentState
  • (abstract: create target, create shared mutable subject)
  • (concrete: DynamoDB PutItem into Documents table with doc_id, title, content_snapshot="", version=0, created_at)

Load document #

• load document -> DocumentState + DocumentVersionState
  • (abstract: read source, load current subject state)
  • (concrete: DynamoDB GetItem on Documents table for doc_id to load latest snapshot/version before joining the edit session)

Submit edit #

• submit edit -> DocumentOperationLog
  • (abstract: append, append operation against shared subject)
  • (concrete: client sends insert/delete op with base_version over websocket to the Document Service, which prepares a canonical op record for DynamoDB)

Transform concurrent edit #

• transform concurrent edit -> DocumentOperationLog + DocumentVersionState
  • (abstract: state transition, rewrite operation against concurrent operations before apply)
  • (concrete: authoritative document server runs OT using the incoming op and the document's unapplied/concurrent ops before assigning canonical order)

Advance canonical version #

• advance canonical version -> DocumentVersionState
  • (abstract: state transition, guarded ordered apply to shared subject)
  • (concrete: DynamoDB UpdateItem on Documents table setting latest_version = latest_version + 1, ConditionExpression latest_version = expected_version)

Persist canonical operation #

• persist canonical operation -> DocumentOperationLog
  • (abstract: append, persist ordered operation history)
  • (concrete: DynamoDB PutItem into DocumentOperations table with PK=doc_id, SK=version#op_id, transformed operation payload, editor_id, base_version)

Update current document state #

• update current document state -> DocumentState
  • (abstract: overwrite/state transition, materialize latest shared state)
  • (concrete: document owner updates in-memory active document state and optionally updates snapshot metadata/pointer in DynamoDB)

Broadcast remote change #

• broadcast remote change -> RealtimeDeliveryState
  • (abstract: create realtime fanout side effect)
  • (concrete: document server pushes the canonical transformed op to all websocket clients connected to that doc)

Rebase local pending edits #

• rebase local pending edits -> ClientReplicaState
  • (abstract: transform local replica against canonical remote operation)
  • (concrete: client OT engine applies received op and rebases any unacked local edits)

Update cursor / presence #

• update cursor / presence -> CursorPresenceState
  • (abstract: overwrite, update ephemeral collaborator state)
  • (concrete: websocket server stores latest cursor position in memory or Redis and broadcasts it to other active editors)

Remove presence on disconnect #

• remove presence -> CursorPresenceState
  • (abstract: state transition, delete ephemeral collaborator state)
  • (concrete: websocket disconnect handler removes editor presence from in-memory map and broadcasts user-left event)

Snapshot / compact #

• snapshot / compact -> DocumentSnapshotState
  • (abstract: create target, materialize compacted subject state from operation history)
  • (concrete: background job writes a compacted snapshot of document content plus snapshot_version, then old ops are checkpointed or pruned by retention policy)

Recover after restart #

• recover document session -> DocumentState + DocumentOperationLog + DocumentSnapshotState
  • (abstract: rebuild active mutable subject from durable truth)
  • (concrete: load latest snapshot from DynamoDB, query and replay subsequent operations, then resume serving reconnecting editors)

Retry edit safely #

• retry edit -> DocumentOperationLog
  • (abstract: retry-safe append with operation dedup)
  • (concrete: operation carries op_id; DynamoDB PutItem into DocumentOperations or a dedup table uses ConditionExpression attribute_not_exists(op_id) to avoid double append on retry)

Important notes #

1. the real correctness mechanism is authoritative ordered apply per document, not DynamoDB alone
2. DynamoDB provides durable metadata, version state, and operation log persistence
3. cursor and presence are ephemeral and should not be modeled as durable document truth
4. snapshotting is necessary to avoid replaying an unbounded operation log on every cold load

9. Versioned Namespace + Immutable Content-Addressed Units #

Example: Dropbox style file storage and sync

Canonical entities #

• NamespaceEntryState
• FileHeadState
• FileVersionState
• ContentChunk
• ChunkManifestState
• ShareRelationState

Create file entry #

• create file entry -> NamespaceEntryState
  • (abstract: create target, create mutable namespace entry)
  • (concrete: DynamoDB PutItem into Files table with file_id, owner_id, filename/path, current_version_id=null, status=CREATED)

Start upload session #

• start upload session -> UploadSessionState
  • (abstract: create target, create pending version/upload workflow)
  • (concrete: DynamoDB PutItem into UploadSessions table with upload_id, file_id, status=UPLOADING, expected_size, file_fingerprint if known)

Fingerprint file #

• fingerprint file -> FileFingerprintState
  • (abstract: derive immutable content identity candidate)
  • (concrete: client computes whole-file hash/fingerprint and backend checks DynamoDB fingerprint index for dedup/resume lookup)

Chunk file #

• chunk file -> ContentChunk
  • (abstract: derive immutable content-addressed units)
  • (concrete: client splits file into chunks, preferably content-defined chunks, and computes a content hash for each chunk)

Dedup existing chunks #

• dedup existing chunks -> ContentChunk
  • (abstract: read source on immutable content store to avoid re-upload)
  • (concrete: backend checks DynamoDB chunk index for existing chunk hashes and returns only missing chunks for upload)

Upload missing chunk #

• upload missing chunk -> ContentChunk
  • (abstract: create target, add immutable content unit)
  • (concrete: client uploads chunk directly to blob storage using presigned URL; backend records chunk_hash -> blob_key/size/reference metadata in DynamoDB)

Build chunk manifest #

• build chunk manifest -> ChunkManifestState
  • (abstract: create target, materialize immutable file-content composition)
  • (concrete: DynamoDB PutItem into ChunkManifests table listing ordered chunk hashes / blob references for this file version)

Create new file version #

• create new file version -> FileVersionState
  • (abstract: create target, create immutable version node)
  • (concrete: DynamoDB PutItem into FileVersions table with version_id, file_id, manifest_id, size, file_fingerprint, created_at)

Advance file head #

• advance file head -> FileHeadState
  • (abstract: state transition, move mutable head/pointer to new immutable version)
  • (concrete: DynamoDB UpdateItem on Files/FileHead table setting current_version_id -> new version_id, incrementing version_number, updating updated_at)

Rename or move file #

• rename or move file -> NamespaceEntryState
  • (abstract: overwrite/state transition, mutate namespace without mutating content)
  • (concrete: DynamoDB UpdateItem on NamespaceEntryState changing filename/path/parent_folder while leaving FileVersionState and ContentChunk references unchanged)

Download file #

• download file -> FileHeadState + FileVersionState + ChunkManifestState
  • (abstract: read source, resolve mutable namespace/head to immutable version then fetch content units)
  • (concrete: backend reads current_version_id from DynamoDB metadata, loads manifest, and returns CDN/S3 signed URL or a logical download plan for referenced content)

Sync file to another device #

• sync file -> FileHeadState + FileVersionState
  • (abstract: read projection, compare namespace/version state and fetch missing immutable units)
  • (concrete: client polls or receives change event, sees file head changed to a new version_id, and downloads only missing chunks or the full file depending on local cache state)

Share file #

• share file -> ShareRelationState
  • (abstract: create relation, grant another principal access to namespace entry/head)
  • (concrete: DynamoDB PutItem into Shares table mapping file_id or namespace entry to target_user_id with permissions)

View shared files #

• view shared files -> ShareRelationState + NamespaceEntryState
  • (abstract: read projection on sharing relation + namespace/head state)
  • (concrete: query DynamoDB share index by user_id, then resolve file metadata for filename/path/current version info)

Resume interrupted upload #

• resume interrupted upload -> UploadSessionState + ContentChunk
  • (abstract: read pending upload state and continue filling immutable units)
  • (concrete: lookup upload session by file_fingerprint/upload_id in DynamoDB, return which chunks/parts are already present, upload only remaining chunks)

Garbage collect orphaned chunks #

• garbage collect orphaned chunks -> ContentChunk
  • (abstract: delete unreferenced immutable content units after retention window)
  • (concrete: background GC scans chunk reference counts / manifest reachability and deletes chunks from blob storage only when no FileVersionState references them)

Important notes #

1. the mutable namespace/head must be modeled separately from immutable content chunks
2. file updates should create a new immutable version and then move the head pointer
3. content-defined chunking is what makes delta sync and dedup efficient for real file edits
4. chunk garbage collection must be reachability- or reference-count-based, not naive time-based deletion

Summary #

Relation / tagging #

• source truth in DynamoDB unique edge table
• bucketed read index for hot tags
• streams or event bus for popularity projections

Scheduler #

• future constraint in DynamoDB execution rows
• claimable run in SQS delayed/visible messages
• execution lifecycle in DynamoDB status updates

Ticketmaster #

• time-bounded exclusive allocation as hold with expiry
• pure DDB via conditional state transitions
• or DDB + Redis with Redis as lease layer and DDB as durable commit layer

Uber #

• durable request in DynamoDB
• assignment as guarded claim/proposal
• execution as separate workflow state

Auction #

• append bids as immutable competitive events
• maintain explicit BestBidState
• close the auction window with a guarded transition
• commit an explicit AuctionWinnerState

Critical transaction #

• use PaymentIntentState as workflow root
• use TransactionState for money-movement attempts
• persist idempotency keys to prevent double charges
• reconcile uncertain external outcomes explicitly

Shared mutable subject #

• model the document as durable shared state plus append-only operation history
• use a single authoritative ordered apply path per document
• keep cursor/presence ephemeral and separate from document truth
• snapshot periodically to cap replay and cold-load cost

Versioned namespace + immutable content #

• separate mutable namespace/head from immutable content chunks
• create new immutable manifests and file versions instead of overwriting content
• move the file head pointer to publish a new version
• use content-defined chunking and chunk hashes for delta sync and dedup

These are good reference realizations for:

1. relation hot-spot handling
2. future scheduling
3. frontier processing
4. temporary exclusive allocation
5. matching and assignment
6. competitive bidding windows
7. critical transaction workflows
8. shared mutable subject systems
9. versioned namespace + immutable content-addressed systems