Infrastructure & Service Map

Complete map of all services, APIs, databases, and credentials that power the N.E.S.T. (Nexus for Evidence, Screening, and Tracking) platform. Use this when debugging cross-service issues or onboarding new team members.

Sensitive Infrastructure: This page documents credential locations and service endpoints. Access is restricted to INT team members via Cloudflare Access.

Service Architecture

                    ┌─────────────────────────────────────────────┐
                    │              User's Browser                  │
                    │  ┌─────────────────────────────────────────┐ │
                    │  │  N.E.S.T. SPA (React + CesiumJS)       │ │
                    │  │  - 3D Globe with Google Photorealistic  │ │
                    │  │  - Marker rendering (UFO/airplane SVG)  │ │
                    │  │  - Day/night imagery blending           │ │
                    │  └──────────┬──────────┬──────────┬────────┘ │
                    └─────────────┼──────────┼──────────┼──────────┘
                                  │          │          │
                    ┌─────────────▼──┐  ┌────▼────┐  ┌─▼──────────┐
                    │ Cloudflare     │  │Firebase │  │ nest-api   │
                    │ Access (OAuth) │  │Firestore│  │ (CF Pages  │
                    │                │  │         │  │  Functions)│
                    └────────────────┘  └────┬────┘  └─────┬──────┘
                                             │             │
                              ┌──────────────┴──┐    ┌────▼─────┐
                              │  AWS S3          │    │ CF D1    │
                              │  (Drop media)    │    │ (Lists,  │
                              │                  │    │  Users)  │
                              └──────────────────┘    └──────────┘

Deployments

SiteURLPlatformDeploys From
Productionnest.thephenom.appNetlifymain branch auto-deploy
Devdev-nest.thephenom.appCF Pages (dev-nest-phenom)wrangler pages deploy from ai
Dev (direct)*.dev-nest-phenom.pages.devCF PagesSame — per-deploy preview URLs

Deployment Commands

# Build on ai.matthewstevens.org (Apple Silicon, has Node + npm)
ssh ai "cd .../admin_sandbox/phenom && npx vite build"

# Deploy to dev-nest CF Pages
wrangler pages deploy dist --project-name=dev-nest-phenom --branch=main

# Production deploys automatically via Netlify on push to main

Authentication

Cloudflare Access (Login Gate)

SettingValue
Team domainthephenom-app.cloudflareaccess.com
IdP typeGitHub OAuth App
OAuth App Client IDOv23li8PtxSU0LskOBU9
OAuth App secretpass sanmarcsoft/github-oauth/phenom-oauth-client-secret
Callback URLhttps://thephenom-app.cloudflareaccess.com/cdn-cgi/access/callback
CF AccountPavestar (pass cloudflare/phenom-account-id)

Access Apps (all use the same IdP):

AppDomainPolicy
nestnest.thephenom.appGitHub login required
Dev NEST Dashboarddev-nest.thephenom.appGitHub login + IP 84.112.12.104
The Phenom App Docsint-docs.thephenom.appGitHub login required
phenom-backend Pages*.phenom-backend.pages.devGitHub login required

GitHub App (Team Data)

SettingValue
App nameN.E.S.T. Team
App ID3085870
Client IDIv23li5FmbWUV3ME5ZOt
Client secretpass sanmarcsoft/github-oauth/phenom-nest-client-secret
PurposePull INT team member profiles from Phenom-earth GitHub org
NOT used forCF Access login (that’s the OAuth App above)

CRITICAL: The OAuth App and GitHub App are DIFFERENT things. The OAuth App (Ov23li...) controls CF Access login. The GitHub App (Iv23li...) pulls team data. Never overwrite the CF Access IdP with GitHub App credentials — this breaks production login for ALL sites.


Data Sources

Firebase Firestore

CollectionSourceData
phenomPhenom mobile appVideo recordings with GPS, timestamps, owner info
phenom/{id}/shootsPhenom mobile appIndividual recording segments with video URLs
dropsPhenom Drop web appImage submissions with C2PA, GPS, sensor data

Firebase Project: phenom-7ee1a Auth: Email/password service account Credentials: .env file (VITE_FIREBASE_AUTH_EMAIL, VITE_FIREBASE_AUTH_PASSWORD) Must be baked into build: Yes — Vite replaces import.meta.env.VITE_* at build time

AWS S3 (Drop Media)

SettingValue
Bucketphenom-dev-media-staging
Regionus-east-1
URL formathttps://phenom-dev-media-staging.s3.us-east-1.amazonaws.com/{s3Key}
AccessPublic GET for uploads/* prefix, restricted by Referer

CORS Configuration (docs/s3-cors.json) — two rules as of 2026-03-14:

Rule 1 — N.E.S.T. origins (GET, HEAD):

  • https://nest.thephenom.app
  • https://dev-nest.thephenom.app
  • https://phenom.matthewstevens.org

Rule 2 — Drop origins (GET, PUT, HEAD; exposes ETag header):

  • https://www.thephenom.app
  • https://thephenom.app

Drop origins require PUT for presigned URL uploads and HEAD for integrity checks. The ETag exposed header is needed so the client can verify upload success.

Bucket Policy Referers (docs/s3-bucket-policy.json):

  • https://nest.thephenom.app/*
  • https://dev-nest.thephenom.app/*
  • https://phenom.matthewstevens.org/*

Adding a new domain: If you deploy N.E.S.T. or Drop to a new domain, you MUST update the S3 CORS config (the appropriate rule for the site type) AND the bucket policy to include the new origin. N.E.S.T. domains need GET + HEAD; Drop domains need GET + PUT + HEAD. Without this, media loading or uploads will fail with CORS errors.

Cloudflare D1 (Lists & Users)

SettingValue
Database namenest-db
Database IDc7929d36-0f8e-4343-8f18-99bee03e326e
Tablesusers, lists, list_items, list_shares, item_shares, transcriptions
Accessed byPages Functions at /api/*
BindingDB in CF Pages project settings

Schema: admin_sandbox/nest-api/schema.sql


API Endpoints

nest-api (CF Pages Functions)

Lives at: admin_sandbox/phenom/functions/api/[[path]].js

All endpoints require CF_Authorization cookie (CF Access JWT).

EndpointMethodPurpose
/api/healthGETHealth check (no auth)
/api/listsGETList all user’s owned lists
/api/listsPOSTCreate new list
/api/lists/:idPATCHRename list
/api/lists/:idDELETEDelete list
/api/lists/:id/itemsGETGet items in a list
/api/lists/:id/itemsPOSTAdd item to list
/api/lists/:id/items/:itemIdDELETERemove item
/api/lists/:id/sharePOSTShare list with team member
/api/sharesPOSTShare individual item
/api/shares/inboxGETGet items shared with me
/api/shares/:id/seenPATCHMark share as seen
/api/usersGETList all users
/api/users/meGETCurrent user profile + notification counts
/api/notifications/countGETUnseen share counts
/api/teamsGETGitHub org teams (requires GitHub App)
/api/teams/:slug/membersGETTeam members

Drop Upload API (AWS Lambda)

SettingValue
Endpointhttps://lxd3gpzlph.execute-api.us-east-1.amazonaws.com/development/upload/generate-url
PurposeGenerate presigned S3 PUT URL for drop upload
AuthUpload password (server-side only)

Drop Backend (Python)

SettingValue
Repophenom-drop/backend/server.py
Endpoints/api/drop/send-password, /api/drop/verify-password, /api/drop/upload
PurposeEmail verification, hash registry, S3 upload proxy

Drop Upload Flow

User (phenom-drop web app)
  │
  ├─1─▶ Verify C2PA credentials (client-side)
  │
  ├─2─▶ POST /api/drop/send-password
  │      → Backend sends 6-char OTP via Brevo email
  │
  ├─3─▶ POST /api/drop/verify-password
  │      → Backend validates OTP + registers hash
  │
  ├─4─▶ POST /api/drop/upload
  │      → Backend proxies to AWS Lambda
  │      → Lambda returns presigned S3 PUT URL
  │      → Backend writes metadata to Firestore `drops` collection
  │
  ├─5─▶ PUT to S3 presigned URL (direct upload from browser)
  │      → File stored at s3://phenom-dev-media-staging/uploads/...
  │      → S3 CORS Rule 2 allows PUT from Drop origins
  │
  └─6─▶ Backend updates Firestore with mediaUrl + s3Key
         → N.E.S.T. PhenomFetcher reads this to display image

Note (2026-03-14): computeFileHash() in www/web/drop.html now uses streaming SHA-256 (chunked reads via ReadableStream) for files >100MB. This avoids loading the entire file into memory at once, preventing browser tab crashes on large uploads.


Credential Locations (GPG Pass)

PathPurpose
cloudflare/phenom-account-idCF account ID (Pavestar)
cloudflare/phenom-zone-idthephenom.app zone ID
cloudflare/phenom-api-tokenCF API token (limited perms)
verifieddit/CLOUDFLARE_API_KEYGlobal API Key (works across accounts)
verifieddit/CLOUDFLARE_AUTH_EMAILAuth email for Global API Key
sanmarcsoft/github-oauth/phenom-oauth-client-idOAuth App for CF Access
sanmarcsoft/github-oauth/phenom-oauth-client-secretOAuth App secret
sanmarcsoft/github-oauth/phenom-nest-client-idGitHub App for team data
sanmarcsoft/github-oauth/phenom-nest-client-secretGitHub App secret

CesiumJS Globe Configuration

Imagery Layers (bottom to top)

LayerProviderVisibility
Day imageryEsri World Imagery (satellite)dayAlpha=1.0, nightAlpha=0.0
Night imageryNASA VIIRS Black Marble 2012dayAlpha=0.0, nightAlpha=1.0
Political bordersCartoDB dark_only_labelsToggled via toolbar button
Google 3D TilesGoogle Maps Tile APIHidden above 500km altitude

Google Maps Tile API

SettingValue
API KeyAIzaSyDLi52bh9UXdiqZows69Z41S6qWnsPwzII
Endpointhttps://tile.googleapis.com/v1/3dtiles/root.json?key=...
Loaded viaCesium3DTileset.fromUrl()
Night dimmingimageBasedLightingFactor adjusted per frame by sun angle

Day/Night Effects

EffectMechanismScope
Globe day/nightglobe.enableLighting = true + SunLightBase globe surface
AtmospheredynamicAtmosphereLighting + dynamicAtmosphereLightingFromSunSky limb
3D Tile dimmingimageBasedLightingFactor per frameGoogle 3D Tiles only
Night visionCSS hue-rotate + brightness + desaturate filterEntire canvas below 500km
Tile visibilitytileset.show = height < 500kmReveals day/night imagery at global view

Clock

  • Default: current real time (JulianDate.now())
  • On marker click: jumps to recording timestamp
  • On deselect: resets to real time
  • shouldAnimate = false — time only changes on marker selection

Build & Deploy Checklist

Before deploying any changes:

  1. Firebase .env must exist on build server (ai): VITE_FIREBASE_AUTH_EMAIL, VITE_FIREBASE_AUTH_PASSWORD
  2. Google Maps key is hardcoded in CesiumGlobe.tsx (not an env var)
  3. S3 CORS must include the deployment domain
  4. D1 binding must be configured in CF Pages project settings
  5. CF Access must have an app + policy for the deployment domain
  6. OAuth App callback must be https://thephenom-app.cloudflareaccess.com/cdn-cgi/access/callback

Last Updated: 2026-03-14