Operations, CI/CD & Deployment Runbook (HUB-OPS-001)
| Field | Value |
|---|---|
| Document ID | HUB-OPS-001 |
| Pillar | Software Engineering |
| Standard Alignment | ISO/IEC/IEEE 26514 / SRE Runbook Standard |
| Status | Active / Synchronized with Codebase |
| Last Updated | 2026-11-21 |
| Source of Truth | .github/workflows/portal.yml, Dockerfile, docker-compose.yml, nginx.conf, vercel.json, .env.example, ops/deploy-ssh.sh, ops/release-on-vm.sh, ops/p3mdi-sso/Dockerfile, ops/p3mdi-sso/compose.yaml, ops/authentik/README.md, google-apps-script/Code.gs, scripts/generate-master-sql.mjs, docs/SECURITY.md, docs/p3mdi-sso.md |
1. Deployment Environments & Topology
Section titled “1. Deployment Environments & Topology”Soedirman Hub runs in three primary topologies:
- Local Development (
npm run devordocker compose up --build -d): Vite 6 hot-reload server or local Docker container (127.0.0.1:${PORTAL_PORT:-8080}:80) backed by a Supabase project (or?demo=trueoffline synthetic sandbox). - Production VM Edge (
https://app.p3mdi.id): Containerized static SPA served bynginxinc/nginx-unprivileged(UID101, port80) under/opt/legacy-batch-p3md, paired with the Traefik-routed Deno SSO & Core Broker (ops/p3mdi-sso/on:8000) and external P3MDI Authentik IdP (https://auth.p3mdi.id). - Static Preview / Alternative Edge (
vercel.json): Compatible SPA rewrite + identical security header configuration for Vercel static builds.
flowchart TD
Push["GitHub PR / Push to main"] -->|"Job 1: security"| Sec["npm audit --audit-level=high + Gitleaks Scan"]
Push -->|"Job 2: build"| Build["npm test (240 tests) + Deno Test + Postgres SQL Test + npm run build"]
Build -->|"Site Bundle Artifact"| Art["site.tar.gz + site.tar.gz.sha256"]
Sec --> Gate{"main push & PORTAL_AUTO_DEPLOY == 'true'?"}
Build --> Gate
Gate -->|Yes| DBCheck["POST portal_critical_batch_ready & portal_pengasuh_release_ready"]
DBCheck -->|"Both true"| SSH["Forced-Command SSH -> ops/deploy-ssh.sh"]
SSH -->|"flock lock"| VM["ops/release-on-vm.sh (Build Container + Tag last-good + Health Verify)"]
VM -->|"/healthz or asset cmp fails"| Rollback["Automatic Rollback to legacy-batch-p3md:last-good"]
2. Environment Variables & Configuration Catalog
Section titled “2. Environment Variables & Configuration Catalog”2.1 Frontend Build & Runtime Variables (.env.example, Dockerfile, .github/workflows/portal.yml)
Section titled “2.1 Frontend Build & Runtime Variables (.env.example, Dockerfile, .github/workflows/portal.yml)”(Vite embeds VITE_* variables statically into dist/ at npm run build time.)
| Variable | Required | Used By | Purpose |
|---|---|---|---|
VITE_SUPABASE_URL |
Yes (fails closed if missing) | src/lib/supabase.js |
HTTPS Supabase project URL (http://localhost / 127.0.0.1 allowed locally). |
VITE_SUPABASE_ANON_KEY |
Yes (fails closed if missing) | src/lib/supabase.js |
Supabase publishable/anon key sent in apikey header (never use service_role / sb_secret_ here). |
VITE_GDRIVE_WEBHOOK_URL |
For photo uploads | src/lib/supabase.js |
HTTPS script.google.com or script.googleusercontent.com /exec URL (VITE_GAS_UPLOAD_URL supported as fallback). |
VITE_GDRIVE_FOLDER_URL |
Optional | src/store/portal.js |
External Google Drive root folder quick-link displayed in Dokumentasi UI. |
VITE_NOMINATIM_REVERSE_URL |
Optional | src/components/absensi/LivePhotoModal.vue |
Same-origin reverse geocoding proxy (e.g., /api/geocoding/reverse) for coarse photo labels. |
VITE_GOOGLE_MAPS_API_KEY |
Optional | .env.example, PetaPersebaranView.vue |
Build-time Maps JS key alternative to user-supplied browser key. |
PORTAL_PORT |
Local Docker | docker-compose.yml |
Host loopback port mapped to container :80 (default 8080). |
2.2 Deno SSO & Core Broker Variables (ops/p3mdi-sso/Dockerfile & compose.yaml)
Section titled “2.2 Deno SSO & Core Broker Variables (ops/p3mdi-sso/Dockerfile & compose.yaml)”The container runs under UID deno with strict --allow-env=... enumeration:
| Variable | Required | Used By | Purpose |
|---|---|---|---|
P3MDI_OIDC_CLIENT_ID |
Yes | index.ts, profile.ts |
Public OIDC client identifier returned on GET /api/sso. |
P3MDI_OIDC_CLIENT_SECRET |
Yes | index.ts, profile.ts, security.ts |
Confidential OIDC secret for /application/o/token/. |
P3MDI_HUB_ORIGIN |
Yes | index.ts, profile.ts, security.ts |
Exact origin enforced on Origin header (https://app.p3mdi.id) & OIDC redirect_uri. |
P3MDI_HUB_PUBLIC_KEY |
Yes | profile.ts, accounts.ts |
Supabase publishable/anon key used to validate caller sessions via portal_me. |
P3MDI_ACCOUNT_API_TOKEN |
Yes | profile.ts, security.ts, accounts.ts |
Bearer token for https://auth.p3mdi.id/api/v3/core/*. |
P3MDI_SERDIK_GROUP_ID |
Yes | profile.ts, security.ts, accounts.ts |
Target Authentik group UUID/PK (Soedirman Hub Serdik). |
SUPABASE_URL |
Yes | index.ts, profile.ts, accounts.ts, core.ts |
Supabase project base URL. |
SUPABASE_SECRET_KEY |
Yes (sb_secret_*) |
index.ts, profile.ts, accounts.ts, core.ts |
Privileged Supabase API key passed only from broker -> PostgREST RPCs. |
P3MDI_CORE_URL |
Optional | core.ts |
Core Directory base URL (defaults to https://core.p3mdi.id). |
P3MDI_CORE_TOKEN |
For /api/sso/core |
core.ts |
Bearer token for /v1/bridge/members/*. |
2.3 External Integrations (Code.gs, GitHub Actions, Authentik)
Section titled “2.3 External Integrations (Code.gs, GitHub Actions, Authentik)”- Google Apps Script Properties (
google-apps-script/Code.gs):SUPABASE_URL,SUPABASE_ANON_KEY. Optional script constantSPECIFIC_FOLDER_IDpins an existing Drive folder instead of creatingP3MD_Grup1_Dokumentasi. - GitHub Actions (
portal.yml): Repository varsPORTAL_AUTO_DEPLOY,VITE_SUPABASE_URL,VITE_SUPABASE_ANON_KEY,VITE_GDRIVE_WEBHOOK_URL,VITE_GDRIVE_FOLDER_URL,VITE_NOMINATIM_REVERSE_URL,PORTAL_SSH_KNOWN_HOSTS; Repository secretPORTAL_DEPLOY_KEY. - Authentik Custom App (
ops/authentik/README.md): Mountops/authentik/p3mdi_hubread-only (/p3mdi_hub:ro) into Authentik server & worker, registerp3mdi_hub.apps.HubConfiginINSTALLED_APPS, grant globalauthentik_core.add_userto roleSoedirman Hub Serdik Activation, and run the one-time Django shellsync_userloop.
3. CI/CD Pipeline & Atomic VM Release (.github/workflows/portal.yml)
Section titled “3. CI/CD Pipeline & Atomic VM Release (.github/workflows/portal.yml)”3.1 Pull Request & Main CI Jobs (ubuntu-24.04)
Section titled “3.1 Pull Request & Main CI Jobs (ubuntu-24.04)”securityJob:- Checks out full commit history (
fetch-depth: 0,persist-credentials: false). - Runs
npm audit --audit-level=high(Node 22). - Scans entire reachable Git history with pinned
ghcr.io/gitleaks/gitleaks(git --redact --log-opts=HEAD .).
- Checks out full commit history (
buildJob:npm ci --ignore-scripts->npm test(240Node tests).- Builds
ops/p3mdi-ssoDocker image (runsdeno check --no-lock index.tsduring image build) and runssetup_test.ts,security_test.ts, andaccounts_test.tsinside Deno. - Boots ephemeral
postgres:16-alpine(--network none), copiestests/+supabase/, and executespsql -f /tests/sql/sso-management.test.sql. - Builds production SPA (
npm run build), archivesdist/intosite.tar.gz+site.tar.gz.sha256, and uploadsportal-<sha>artifact (7-day retention).
3.2 Gated Production Deployment (deploy Job)
Section titled “3.2 Gated Production Deployment (deploy Job)”Triggered only on pushes to main when vars.PORTAL_AUTO_DEPLOY == 'true':
- Verifies
site.tar.gz.sha256and checks GitHub API confirminggithub.shais stillrefs/heads/mainHEAD (prevents stale concurrent overwrites). - Queries Supabase RPCs
portal_critical_batch_readyandportal_pengasuh_release_readywithcurl. If either returns anything other thantrue, skips release gracefully with an operator notice until pending migrations are applied. - Streams
site.tar.gzover pinned SSH (BatchMode=yes,StrictHostKeyChecking=yes) to forced-command wrapperops/deploy-ssh.sh(portal-deploy@76.13.21.243 "deploy $GITHUB_SHA $GITHUB_RUN_NUMBER").
3.3 VM Execution & Automatic Rollback (ops/deploy-ssh.sh & ops/release-on-vm.sh)
Section titled “3.3 VM Execution & Automatic Rollback (ops/deploy-ssh.sh & ops/release-on-vm.sh)”ops/deploy-ssh.shenforces strict^deploy ([0-9a-f]{40}) ([1-9][0-9]{0,11})$regex validation and delegates to/usr/local/sbin/portal-releaseorlegacy-portal-releaseviasudo -n.ops/release-on-vm.shexecutes inside/opt/legacy-batch-p3md:- Acquires exclusive
flock(/run/lock/legacy-portal-release.lock) and discards out-of-order run numbers (last-release-run). - Streams tar safely via Python
tarfile(filter="data", hard<= 100 MBsize bound), verifies non-emptyindex.htmland primary/assets/*.jschunk. - Tags running container image as
legacy-batch-p3md:last-goodand buildslegacy-batch-p3md:release-<sha>atop pinnednginxinc/nginx-unprivilegedSHA-256 digest. - Updates
compose.override.yamlatomically and runsdocker compose up -d --no-build --wait --wait-timeout 90. - Runs live parity verification against
https://app.p3mdi.id/healthz(ok),/login(byte-equalindex.html), and/assets/*.js. Any mismatch triggers immediate restore tolegacy-batch-p3md:last-goodand non-zero exit.
- Acquires exclusive
4. Database Migration, Backup & Seeding Runbook
Section titled “4. Database Migration, Backup & Seeding Runbook”- Pre-Deploy Migration Order: Always apply new SQL files under
supabase/migrations/chronologically in Supabase SQL Editor (or Postgres superuser session owningpublic.serdik) before promoting frontend code.- Specifically, when
20261130060000(portal_critical_batch_ready) and20261130070000(portal_pengasuh_release_ready) are applied, re-run CI workflow dispatch (workflow_dispatch) if a deploy was previously skipped.
- Specifically, when
- View Ownership Verification: Whenever altering
public.serdikor views touching it, verifypublic.portal_serdik_directoryretainssecurity_invoker = false,security_barrier = true, and identicalrelowneraspublic.serdik(HTTP200onGET /rest/v1/portal_serdik_directory?select=id,nosis&limit=1). - Backup & Rollback: Before schema changes affecting state tables, capture a Supabase/Postgres logical dump (
pg_dump). Migrations are written idempotently (IF NOT EXISTS,CREATE OR REPLACE) to permit safe re-application. - Master Data PIN Generation: Run
node scripts/generate-master-sql.mjs --rotate=pengasuh(orall/none) locally to producesupabase/private/01_master_data.sql(crypt(..., gen_salt('bf', 8))) and temporary distribution sheetsupabase/private/pin-baru.csvwithout storing plaintext PINs in repository code. - Local Container Smoke Test: Validate edge headers, MIME types (
/manifest.webmanifest->application/manifest+json; charset=utf-8), and/healthzlocally with:Terminal window docker compose downdocker compose up --build -d
5. Health Monitoring & Incident Playbooks
Section titled “5. Health Monitoring & Incident Playbooks”| Endpoint / Check | Expected Output | Diagnostic Command |
|---|---|---|
Edge Liveness (nginx.conf) |
HTTP 200, body ok |
curl -fsS https://app.p3mdi.id/healthz |
Container Loopback (docker-compose.yml) |
Healthy Docker status | wget -qO- http://127.0.0.1/healthz |
| Directory View Health | HTTP 200 (NOT 42501) |
curl -s -o /dev/null -w "%{http_code}\n" "$SUPABASE_URL/rest/v1/portal_serdik_directory?select=id,nosis&limit=1" -H "apikey: $ANON_KEY" |
| SSO Broker Readiness | HTTP 200, JSON {"clientId":"..."} (503 if unconfigured) |
curl -fsS https://app.p3mdi.id/api/sso |
| GAS Drive Webhook Health | HTTP 200, JSON {"status":"active"} |
curl -fsSL "$VITE_GDRIVE_WEBHOOK_URL" |
Common Incidents & Recovery Actions
Section titled “Common Incidents & Recovery Actions”- Sync Banner HTTP
42501(permission denied for table serdik): A manual edit or Supabase Security Advisor action flippedportal_serdik_directorytosecurity_invoker = trueor mismatched its owner. Re-applysupabase/migrations/20261121_restore_serdik_directory_view.sql(and20261128_core_member_cache.sqlif missingcore_*columns) as the owner ofpublic.serdik. - Users Locked Out After 5 Failed PINs (
locked/retry_after_seconds: 900): Wait 15 minutes for sliding-window expiry or asksuper_adminto invoke Unlock Login inAdminConsoleView(portal_admin_unlock_login). - Stuck Service Worker / Chunk Load Error After Release:
nginx.conf&vercel.jsonmark/index.htmland/sw.jsno-cache, max-age=0, must-revalidate, whilesrc/router/index.jsperforms one automaticwindow.location.assign(target)usingsessionStorage['soedirman_chunk_reload']. Instruct affected mobile PWA users to clear site data or log out and tap Settings -> Clear Cache if stale UI persists. - Account Stays Disabled in Core After Sync (
core_is_active = false):portal_loginintentionally blocks archived Core accounts (20261129). Reactivate the member in P3MDI Core (core.p3mdi.id) or trigger/api/sso/corecache sync.