RDS Dev Runbook

The AWS RDS PostgreSQL development database instance backing the Phenom staging environment and development services.
Audit stamp: Verified — 2026-06-19 — Phenom AI Agent
Verified · 2026-06-19 · Phenom AI Agent
Source: asset-registry.yaml; host: phenom-dev-postgres (RDS type); access via phenom-oneoff-sql ECS Fargate task
C2PA signed · SanMarcSoft AI content credential

What it is

phenom-dev-postgres is the AWS RDS PostgreSQL database instance powering the Phenom development and staging environment. It backs api-staging.thephenom.app, analytics.thephenom.app, and any other development-tier services. It stores non-production data. Direct connections from developer machines are not permitted; all SQL access goes through the phenom-oneoff-sql ECS Fargate task.

Deployment chain

Layer Value
Identifier phenom-dev-postgres
Engine AWS RDS PostgreSQL
Region us-east-1
AWS profile phenom
VPC Phenom VPC (private subnets – no public endpoint)
Access method ECS Fargate task phenom-oneoff-sql (no direct external access)
Consumers api-staging.thephenom.app, analytics.thephenom.app

Common operations

Check RDS instance status

aws rds describe-db-instances \
  --db-instance-identifier phenom-dev-postgres \
  --profile phenom \
  --region us-east-1 \
  --query 'DBInstances[0].{Status:DBInstanceStatus,Class:DBInstanceClass,Engine:EngineVersion,Endpoint:Endpoint.Address}'

Connect to the database via phenom-oneoff-sql task

# Run a one-off ECS Fargate task that opens a psql session inside the VPC:
aws ecs run-task \
  --cluster phenom-dev-cluster \
  --task-definition phenom-oneoff-sql \
  --launch-type FARGATE \
  --network-configuration "awsvpcConfiguration={subnets=[<PRIVATE_SUBNET_ID>],securityGroups=[<DB_SG_ID>],assignPublicIp=DISABLED}" \
  --overrides '{"containerOverrides":[{"name":"psql","command":["psql","$DATABASE_URL","-c","\\l"]}]}' \
  --profile phenom \
  --region us-east-1

# For interactive sessions, use ECS Exec:
TASK_ARN=$(aws ecs run-task ... --query 'tasks[0].taskArn' --output text)
aws ecs execute-command \
  --cluster phenom-dev-cluster \
  --task "$TASK_ARN" \
  --container psql \
  --interactive \
  --command "psql \$DATABASE_URL" \
  --profile phenom \
  --region us-east-1

Start / stop the dev RDS instance (cost saving)

# Stop (dev only -- never stop prod):
aws rds stop-db-instance \
  --db-instance-identifier phenom-dev-postgres \
  --profile phenom \
  --region us-east-1

# Start:
aws rds start-db-instance \
  --db-instance-identifier phenom-dev-postgres \
  --profile phenom \
  --region us-east-1

# Wait until available:
aws rds wait db-instance-available \
  --db-instance-identifier phenom-dev-postgres \
  --profile phenom \
  --region us-east-1

Take a manual snapshot

aws rds create-db-snapshot \
  --db-instance-identifier phenom-dev-postgres \
  --db-snapshot-identifier "phenom-dev-manual-$(date +%Y%m%d-%H%M%S)" \
  --profile phenom \
  --region us-east-1

Resize the instance class

aws rds modify-db-instance \
  --db-instance-identifier phenom-dev-postgres \
  --db-instance-class db.t3.medium \
  --apply-immediately \
  --profile phenom \
  --region us-east-1

Verify it is working

aws rds describe-db-instances \
  --db-instance-identifier phenom-dev-postgres \
  --profile phenom \
  --region us-east-1 \
  --query 'DBInstances[0].DBInstanceStatus' \
  --output text
# Expected: "available"

Common failure modes

Symptom Likely cause Remediation
api-staging returns DB errors RDS instance stopped or unreachable Check instance status; start if stopped; check security group rules
Instance in “modifying” state Pending parameter group or class change Wait for modification to complete; check RDS events
Storage full Autoscaling not enabled or log bloat Enable storage autoscaling; run VACUUM ANALYZE; clear old logs
Slow queries Missing indexes or table bloat Connect via phenom-oneoff-sql; run EXPLAIN ANALYZE; add indexes
Connection limit hit Too many open connections from services Restart services to release idle connections; consider PgBouncer

Last modified June 19, 2026: fix(ci): use committed c2pa-manifests.json; verify signer_cn resolved (#365) (#367) (58fe9f5)