Operational analytics and planning
Open Analitik & perencanaan in the administrator dashboard. Select 7, 30, or 90 days, inspect the interactive charts/tables, and download an aggregate CSV. Dates and hours use Asia/Jakarta (WIB); the current day is partial. The endpoint is GET /manage/api/analytics?days=30, restricted to the existing administrator session. Each response is computed from a repeatable-read, read-only database snapshot and includes its measurement time and definitions.
Decisions and metrics
Section titled “Decisions and metrics”The primary operating measures are current active backlog, first ticket completions in the selected period, and recorded time to completion. Supporting signals are incoming tickets, overdue/unassigned work, backlog age and priority, first recorded staff action, category demand, and current assignee workload. They help admins prioritize cases, identify likely constraints, and plan team capacity. They are not staff performance rankings or causal findings.
| Measure | Definition and limits |
|---|---|
| Active backlog | All tickets currently open or in_progress, including older tickets outside the selected period. |
| Overdue | Active tickets past their current due date; not historical SLA compliance. Targets can change. No new SLA target is invented. |
| First completions | Unique ticket’s first recorded transition from active into resolved/closed during the period. Later resolved→closed does not double-count. Reopened work can remain active despite an earlier first completion. |
| Reopened | Unique tickets with a recorded resolved/closed→open/in_progress transition during the period. Historical events lacking before/after state are not inferred. |
| First action | First recorded attribute/status/assignment change or substantive note, measured from ticket creation. This is not the time of a human chat reply. |
| Resolution time | Calendar hours from creation to first recorded completion. Median and P90 show valid sample size. Completed tickets without a usable event timestamp are counted as a data-quality gap, never assigned a fabricated completion time. |
| Assignee workload | Current active and overdue ticket counts for active staff, plus former/disabled staff who still own active tickets. Not effort, productivity, availability, or hours worked. |
| Category demand | Tickets created in the period grouped by their current category. Category edits can change historical grouping. |
| Message demand | Available case-message archive (at most 30 days), unique member identities, screenshot count, and day/hour heatmap. Includes API traffic if archived there. A member identity is not proof of a unique human. Deleted/expired archive cannot be reconstructed. |
| Knowledge use | New worker telemetry distinguishes KB-grounded responses, general guidance, insufficient evidence, unrelated requests, greetings, ticket flows, fallbacks, media errors, disabled access, and unclassified legacy output. KB use does not prove accuracy or problem resolution. Some basis classifications reflect the model’s declared basis, with backend citation checks able to downgrade invalid attribution. |
| Retrieval gaps | Searches with zero returned candidates divided by all measured searches. Separately show insufficient-evidence outcomes; these groups can overlap and are not added into a single failure rate. Candidate matches do not prove the source answers the question. |
| Queue/processing latency | Worker wall-clock processing duration and estimated wait from job creation to start, shown as median/P90 where applicable. Does not include WhatsApp delivery time. Retries contribute to waiting time; the successful attempt duration or terminal failed attempt is measured. |
| Delivery | WAHA accepted, not sent, pending result, or processing failed. API jobs have no WhatsApp delivery outcome. WAHA acceptance is not delivery/read confirmation. A crash between acceptance and recording may leave pending; no automatic resend is introduced. |
The readiness section adds queued jobs, retained failed jobs, published public evidence, source reviews/failures, and public documents approaching expiry. Current configured retention for ordinary jobs is 30 days. The UI provides readable definitions and raw chart tables; blanks mean unavailable measurements, not zero.
Predictive baseline
Section titled “Predictive baseline”Forecasts are deliberately gated until there are 28 complete days after the first ticket day and at least 20 tickets in the history used. The first observed day and current incomplete day are excluded. Missing calendar days after the first observed ticket are zero-filled; dates before observation are not treated as observed zero demand. At most 84 complete days are used. Calendar gaps assume the helpdesk was collecting data; system downtime or unavailable historical data can invalidate that assumption.
Two local baselines are supported:
- Mean demand over the most recent 28 days.
- Mean of up to eight previous occurrences of the same weekday.
Once 14 rolling historical test days with at least 28 prior training days are available, the system chooses the baseline with lower mean absolute error (MAE, tickets/day). Every test prediction uses only earlier observations. With 28–41 days it displays a provisional recent-mean baseline, without an asserted test error or planning range. The current implementation selects and reports performance on the same 14-day evaluation window; this is a model-selection diagnostic, not an independent future-accuracy guarantee.
For validated baselines, a per-day planning band uses the empirical 90th-percentile absolute error from those test days, clipped at zero. It is not a statistical confidence interval, a calibrated probability, or a guarantee. Zero variation in a small synthetic/real sample can produce a zero-width band; future variation can still occur. Upcoming events, policy changes, outages, category/priority shifts, and holidays are not modeled. Forecasts concern ticket counts, not individuals or user behavior.
Prescriptive actions and scenario planner
Section titled “Prescriptive actions and scenario planner”Action cards show their evidence, reason, and a link to the relevant management screen. Rules highlight overdue work, missing assignment, active tickets older than seven days, missing completion-event history, evidence gaps, source reviews/failures/expiry, delivery/processing failures, and insufficient forecasting history. The seven-day age threshold is a review heuristic, not a newly imposed SLA. Capacity advice is withheld when completed-ticket timestamps are missing.
With an available forecast, the local simulator accepts team completions/day. Its starting value is measured first completions across up to 28 complete observed days, not guaranteed staffing capacity. Admins can substitute a realistic capacity. The planner computes:
estimated backlog after 7 days = max(0, current active backlog + expected arrivals over next 7 complete days − daily completions × 7)
It also shows completions/day needed to hold backlog steady or clear it within that horizon. This is a conditional what-if calculation, not optimization or a promise: it assumes stable capacity and demand, omits work reopened later, priority differences, and remaining arrivals today. Decimal forecasts are expected values, not fractional actual tickets. No automatic assignments, status changes, target changes, messages, or external actions are performed.
Collection, privacy, and retention
Section titled “Collection, privacy, and retention”analytics_configuration.started_at marks when outcome measurement became available. Old AI outcomes are not backfilled by guessing from reply text. New jobs write deduplicated analytics_messages rows keyed by a hash of job ID: event time, completion time, channel, screenshot flag, outcome, search/candidate count, durations, and delivery outcome. They store no message/answer text, member ID, name, phone, email, image, provider prompt, or key. They are operational metadata, not a guarantee of irreversible anonymization.
Instrumentation commits with the job result. Measurement failures use a savepoint and warning so they do not prevent helpdesk replies. Retries deduplicate by the hashed job ID. Recording happens only in the ordinary worker; configuration dry runs do not pollute these measurements. New metadata is removed after 180 days during the existing hourly worker cleanup. Original conversation retention remains 30 days. Ticket and event retention are unchanged. The source PDF worker and existing services gain no analytics credentials or exposed ports.
CSV export contains aggregate metrics, dates, scope/availability information, and daily ticket counts. It excludes message text, contacts, and individual staff rows. This feature runs locally and makes no Google API calls.
Validation and rollback
Section titled “Validation and rollback”tests/test_operational_analytics.py covers authentication/ranges/privacy, backlog and completion semantics, telemetry dedup/delivery, evidence-based classification, low-data gating, provisional behavior, constant and weekly baselines, rolling MAE, and capacity calculations. Run the full suite against a fresh disposable test database before fixture seeders. Browser fixtures are synthetic and must assert current_database() == 'test'; scripts/check_analytics_browser.py tests forecast display, period switching, simulator validation, action drilldowns, CSV, keyboard heatmap inspection, and responsive widths.
Pre-update source: /opt/dirman-assist/code-backups/before-analytics/app. A protected database dump was taken before migration. To roll back, restore that source, rebuild, and recreate API/worker/dashboard only. The additive telemetry tables/index can remain. No WAHA, PostgreSQL service, source-import worker, or shared proxy restart is required for code rollback.