Background workers
Nine of the files in apps/api/src/entries/ are not HTTP Lambdas at all: they are SQS consumers or EventBridge-scheduled workers. They live in the same app because they share the persistence layer, the failure protocol, and the RDS pools with the HTTP entries — but they are invoked by event-source mappings or cron rules wired in AppApiStack (see /infra), never by API Gateway, because webhook drains, reminder sweeps, and cleanup reapers must run on their own retry/DLQ semantics instead of a request/response timeout.
The nine workers
| Entry | Trigger | What it does |
|---|---|---|
plaid-sync-worker.ts | SQS (opsdna-plaid-webhook) | Drains verified Plaid webhooks; syncs transactions; promotes versions to candidate operating-source-events |
collection-acceptance-worker.ts | queue | Commits accepted portco share-up grants into the GP tenant, via SET ROLE into the least-privilege opsdna_collection_acceptance_worker role |
collection-reminder-worker.ts | EventBridge cron | Materializes each cycle’s reminder_policy into due reminders in sharing.notification_outbox, using the same computeReminderSchedule fn as the portal preview (“preview == scheduler equality”) |
kpi-stale-watchlist-worker.ts | EventBridge cron | Mirrors overdue expected KPI periods into sharing.inbox_item as stale_metric actions |
notification-dispatch-worker.ts | EventBridge cron | Claims due outbox rows tenant-by-tenant under normal tenant RLS and hands them to a transport; prod is dry-run by default, staging sends via SES with a recipient override |
recognition-sweep-worker.ts | EventBridge cron (daily) | Drafts deferred-recognition journal entries into the maker-checker inbox — never posts silently |
source-document-ingestion-drain-worker.ts | EventBridge cron | Claims due post-clean ingestion jobs for the stages it has handlers for (e.g. mime_sniff) |
source-document-ingestion-reaper-worker.ts | EventBridge cron | Re-queues/dead-letters stale running jobs; self-heals scan_pending documents from the object’s durable GuardDuty scan tag |
source-document-upload-cleanup-worker.ts | EventBridge cron | Expires abandoned upload items and deletes uncommitted candidate objects |
The three source-document workers and the upload cleanup are cross-tenant sweeps — they connect via the dedicated maintenance role pool, not opsdna_app (see Database access and /source-pipeline).
SQS pattern: partial batch response
From entries/plaid-sync-worker.ts — each record is processed independently, and only the failed message IDs are returned, so one bad message is retried/DLQ’d without re-delivering the whole batch:
export const handler: SQSHandler = async (event: SQSEvent): Promise<SQSBatchResponse> => {
const deps = await buildDeps(); // memoized per warm container (async secret fetch)
const batchItemFailures: { itemIdentifier: string }[] = [];
for (const record of event.Records) {
try {
const message = JSON.parse(record.body) as PlaidWebhookMessage;
await processPlaidWebhook(message, deps);
} catch {
batchItemFailures.push({ itemIdentifier: record.messageId });
}
}
return { batchItemFailures };
};Dependencies are built once per warm container and memoized as a promise (depsPromise), because the Plaid secret comes from an SSM SecureString at runtime (PLAID_SECRET_PARAM).
Pitfalls
- Config throws at cold start.
readConfig()throws ifPLAID_CLIENT_ID/PLAID_SECRET_PARAMare missing orPLAID_ENVisn’tsandbox/production— the worker then fails every batch until env is fixed. - Don’t swallow-and-succeed. Catching an error without adding the record to
batchItemFailuresacknowledges the message — it is gone from the queue forever. Terminal observation belongs to the worker runtime (ADR-0041), not ad-hoc logs. - Wrong role, empty sweep. A cross-tenant worker on the RLS-scoped
opsdna_apppool sees (and writes) nothing — no error, just a silently useless sweep. Use the maintenance pool for cross-tenant work.