Skip to Content
InfrastructureOverview

Infrastructure

OpsDNA runs on two clouds with one deliberate seam: AWS hosts everything stateful (RDS Postgres 18, the per-route Lambda API, KMS, backups), while Cloudflare hosts everything edge-shaped (the LP/GP portal Worker, the chat agent, the landing page on Pages). The split exists because tenant data and its RLS enforcement belong close to Postgres, while the portal wins latency by running at the edge. GitHub Actions is the only thing allowed to deploy either side, authenticated to AWS via OIDC β€” there is no long-lived AWS secret anywhere in CI.

Everything AWS-side is a single CDK app in infra/ (its own Bun package with its own lockfile, separate from the workspace root). This page maps the topology and the stack set; the rest of the section covers environments and deploys, database operations, secrets and config, and observability.

Topology

The portal Worker is deliberately stateless β€” it has no DATABASE_URL and never touches Postgres. Every data access goes through api.opsdna.ai (the AWS HTTP API), which is also why the portal can deploy independently of database state.

The CDK stack set

infra/bin/opsdna.ts wires eight stacks. Stack IDs are Opsdna<Name>Stack for prod and OpsdnaStaging<Name>Stack for staging β€” same code, selected by CDK context (see environments).

StackWhat it ownsPer-env?
SecurityStackPer-boundary KMS CMKs: secrets, RDS storage, backup vault, Plaid access-token, app encryption, source documentsyes
CiStackGitHub Actions OIDC provider + the github-ci-opsdna deploy roleprod only (account-global; staging deploys through the same role)
SecurityBaselineStackGuardDuty detector + bootstrap-ECR scan-on-pushprod only (account-global)
NetworkStackVPC, subnets, NAT, security groupsyes
DatabaseStackRDS PostgreSQL 18 instance, RDS Proxy, CloudWatch alarmsyes
AccessStackThe SSM relay EC2 instance for ad-hoc DB access and CI migrations (default stopped; t3.micro on purpose β€” t4g.nano hit InsufficientInstanceCapacity)yes
AppApiStackThe HTTP API + one Lambda per route, S3 source-document bucket, GuardDuty S3 malware protectionyes
BackupStackAWS Backup vault + multi-tier retention plan for RDS archivalyes

Adding an API route means adding a RouteLambdaSpec in infra/lib/constructs/route-lambda.ts (bundling, IAM, RDS grants, alarms come from the factory) plus the matching entry in apps/api/src/entries/<name>.ts.

      • env.ts
      • app-api-stack.ts
      • database-stack.ts

Deploying by hand

CI is the normal path, but a manual deploy from a workstation looks like this β€” always diff first:

cd infra env -u NODE_OPTIONS ./node_modules/.bin/cdk diff --all --profile opsdna env -u NODE_OPTIONS ./node_modules/.bin/cdk deploy --all --profile opsdna --require-approval never

Synth bundles every Lambda into cdk.out (disk-heavy, but the big write happens before any AWS change, so a disk failure fails safely), and CloudFormation auto-rolls-back on a failed deploy.

Before any prod deploy or DB operation, read docs/runbooks/prod-redeploy-and-db-rebuild.md in the monorepo. It is the canonical runbook β€” ordered deploy sequence, DB-connection cheat-sheet, and a troubleshooting section covering nearly every failure mode ever hit (IAM em-dash CREATE_FAILEDs, GuardDuty bucket-ownership permissions, proxy-PAM traps, migration-checksum pitfalls). If a deploy breaks, the answer is almost certainly already in there.

Hard rules learned the hard way

  • IAM description strings must be pure ASCII. An em-dash, arrow, or smart quote in a role description fails the stack with CREATE_FAILED.
  • Do not deploy the IAM_AUTH proxy changes while the DB roles are not in rds_iam (open decision, OPS-448). A half-deploy breaks login for the entire API. Prod currently uses SECRETS+password proxy auth β€” see Database operations.
  • Some prod state is operational, not code. Role passwords, SSM secrets, the portal’s service token, and rds_iam membership are set by scripts, not reproduced by a redeploy β€” see Secrets and config.
  • Run bun run verify before pushing. Infra changes can break cross-package typecheck or the Lambda bundle budgets that per-package checks miss β€” see Getting started.
Last updated on