The Source → Ledger Pipeline
How external financial activity — today, Plaid bank transactions — becomes journal entries in the general ledger. This section explains the runtime shape (webhook receiver → SQS → sync worker → Postgres), the four-record domain pipeline it feeds, and the operational quirks Plaid imposes on it.
Why this pipeline exists
A bank feed is untrusted, mutable, and semantically mute: Plaid tells us “a $5M wire left account X”, not “the fund funded its XYZ Corp commitment”. The ledger, by contrast, is a self-contained double-entry engine that must never know what Plaid is. The pipeline’s whole job is to carry a raw provider fact across that gap through explicit, auditable stages — with a human/semantic gate in the middle — rather than letting bank rows write to the GL directly.
End-to-end flow
Why two Lambdas
The receiver (apps/api/src/entries/plaid-webhook.ts) is deliberately outside the VPC and never touches RDS. It does exactly two things: verify the Plaid JWS (apps/api/src/middleware/plaid-jws.ts) and forward the raw JSON to SQS. The file header records why: faster cold starts, a smaller blast radius, and direct egress to api.plaid.com for the signing-key fetch. Its IAM is minimal — sqs:SendMessage plus an SSM SecureString read of the Plaid secret.
The worker (apps/api/src/entries/plaid-sync-worker.ts) is the opposite: an SQS event-source-mapped consumer inside the VPC with RDS Proxy IAM auth, Plaid credentials, and the full @opsdna/persistence surface. It reports per-message failures via batchItemFailures, so one poisoned message is retried/DLQ’d without re-delivering the batch.
The split also encodes the failure contract with Plaid, straight from the JWS middleware:
// apps/api/src/middleware/plaid-jws.ts (header)
// - Deterministic verification failures (missing/forged signature, stale iat,
// body mismatch, bad alg) → 401. Plaid does NOT retry 401s — correct.
// - Transient failures (Plaid key endpoint 5xx, secret fetch failure) → die,
// which the runtime renders as 500 so Plaid DOES retry.Do not add database access to the receiver. Its no-VPC posture is a design decision, not an accident — the moment it needs RDS it inherits VPC networking, slower cold starts, and a larger blast radius, and the “verify then enqueue” contract stops being cheap to reason about. Anything that needs the database belongs in the worker.
The four-record pipeline (ADR-0003)
The worker’s writes are stage i–ii of a four-record pipeline; stages iii–iv live in the ledger domain and only run after human acceptance:
| Step | Record | Domain | Meaning |
|---|---|---|---|
| i | source_bank_transaction_version | operating/source | what Plaid said (raw, append-only, permanent evidence) |
| ii | operating_source_event | operating/source | what it means (semantic, human-gated candidate) |
| iii | ledger_source_event | ledger | a posting-ready accounting instruction |
| iv | ledger_journal_entry | ledger | debits/credits, DRAFT → POSTED |
The ii→iii boundary is a deliberate seam — see Domain seams for why the two “source event” records and the two evidence-link tables must never be merged.
In this section
Related: the ledger side of the boundary is covered under /fund-accounting, and the HTTP surface conventions under /api.