Source Documents: Upload, Scan, Encrypt
Bank transactions are one input to the pipeline; the other is paperwork — capital call notices, subscription docs, statements — that arrives as uploaded source documents and re-links into the provenance graph as evidence (ADR-0004 D4). Because these are confidential LP/fund documents, the ingestion chain is built around one load-bearing invariant:
A document is not viewable or downloadable until
source_documents.scan_status = 'clean'.
The state machine (migration 0063, enforced in packages/portal-documents/src/ingestion.ts):
uploaded → hash_verified → scan_pending → clean | rejectedThe scan is out-of-band, not a worker stage (ADR-0029)
The original plan’s “ingestion workers → virus scan” bullet is a trap: the drain worker does not run the scan. Scanning is event-driven around Amazon GuardDuty Malware Protection for S3, chosen because the bytes never leave the AWS account — no confidential LP document is egressed to a third-party scanning vendor, and no ~1GB ClamAV signature database ships inside a Lambda.
The engine is coupled to OpsDNA only at the verdict → scan_status mapping in the object-event router, so it stays swappable. The drain worker owns only post-clean stages, every one gated on scan_status = 'clean'.
Fail closed on ambiguity. A scan failure or unsupported file type leaves the document scan_pending forever rather than exposing it; a time-based reaper alerts on stuck documents (it never auto-cleans). Never add a code path that renders or presigns a document whose scan_status is anything but clean — quarantine is defence in depth, but the viewing gate is the primary control.
Per-tenant encryption and crypto-shred (ADR-0030, V2 draft)
ADR-0030 (proposed, first slice OPS-484) extends the chain to per-tenant cryptographic isolation. The design is forced by one constraint: malware scanning needs plaintext. If a document were app-layer-encrypted before storage, GuardDuty would scan ciphertext and a malicious payload would pass. Hence scan-then-promote:
Land under a shared key
Uploads land in the landing bucket under a single shared CMK — the only key GuardDuty is ever granted decrypt on. Tenant keys never see an AWS-managed service principal.
Promote on clean
On NO_THREATS_FOUND, a mover (EventBridge → SQS → Lambda) does a server-side CopyObject re-encrypting the object under the tenant’s CMK into the clean bucket’s per-tenant prefix, then deletes the landing copy. The bytes never transit the Lambda.
Crypto-shred on offboarding
Destroying one tenant’s documents = disable (instant, reversible) and/or schedule deletion of their CMK. No other tenant is affected. A legal hold must block ScheduleKeyDeletion — shredded data in retained backups is unrecoverable once the key is gone.
A key correction recorded in the ADR: the tenant KEK binds at the promote step, not upload-init. Encrypting the landing object under a tenant key (as the original portalDocumentStorageEncryptionPolicy in packages/portal-security/src/index.ts did) makes it unscannable, since GuardDuty only holds the shared landing key.
Closing the loop with the bank pipeline
An uploaded document that corroborates a bank transaction (the “$5M wire — which deal? upload the paperwork” flow) re-links as evidence via proof_links, and onto the eventual GL line via ledger_item_evidence_links — see Domain seams. The document tables (source_documents, extracted_assertions, evidence_spans) are shared surfaces; ADR-0004 explicitly forbids forking them per-workflow.
Related pages: the worker/Lambda plumbing that runs the object-event router and drain stages is covered in Background workers; the tenant-scoping rules that keep one tenant’s documents invisible to another are in Tenancy & RLS.
References: docs/adrs/0029-source-document-malware-scan-engine.md, docs/adrs/0030-source-document-per-tenant-encryption-and-crypto-shred.md, docs/plans/ops-484-tenant-kek-provisioning.md.