Case study / Operational continuity
GOHL
Operational
Overwatch
Recover the knowledge.
Make the evidence usable.
After a critical integration platform lost its principal developer, I led an AI-assisted reconstruction of its operating model. Independent monitoring, read-only order tracing and expert-graded exceptions gave support staff clearer investigation paths.
Conceptual workflow
Observe independently.
Review before changing.
01 / Continuity before replacement
Understand the system
before deciding to rewrite it.
The platform still processed transactions, but schedules, file movements, dependencies and recovery behavior lived partly in tribal knowledge. I chose to reconstruct that knowledge before considering replacement. The immediate requirement was continuity, not a new transaction engine.
Initial discovery mapped 15 scheduled jobs and surveyed 68 Python files, alongside templates and static assets. Source maps, schedule inventories, dependency notes, log traces and tests became reusable support knowledge rather than another conversation someone had to remember.
My role and Dirk’s boundaries
I owned scope, requirements, architecture choices, prioritization, investigation boundaries, approvals and acceptance. My custom engineering agent, Dirk, supported bounded inspection, mapping, documentation and implementation, with other agents assisting design and review. Uncertainty had to remain visible. Human judgment retained responsibility for production decisions; higher-risk dependency changes were deferred during a critical operating period.
02 / Evidence beyond uptime
Reachable is not the same
as working.
A web application can stay reachable while a batch stalls. A batch can finish without completing the business outcome. Application uptime, useful execution and business completion are separate claims.
Healthwatch, the independent monitoring layer, collected evidence every 15 minutes and published JSON state. Browser refresh read that published state independently; refreshing the page did not make collection continuous. Dashboard failure was not intended to block transaction processing.
The counts below come from retained project records, not a fresh coverage or production-health audit for this website.
The late-August record reports 35 running modules and 56 cataloged checks. These are different scopes, not interchangeable coverage totals. A catalog entry was not proof of an active or independently verified check.
Health and verification answer different questions
An unhealthy process can be accurately monitored. A green indicator can remain unverified if its check has not been adequately tested. Unknown, not due, stale and unavailable states stayed explicit. Testing included malformed input, path and symlink escape, false process matches and history handling. A Linux fixture depended on unspecified file enumeration order; the trial stopped for correction and review.
The stack in context
The existing platform used Ubuntu Linux, nginx, Gunicorn, Flask and Supervisor, with Python, SQLAlchemy and PostgreSQL around ERP and EDI file exchange. Scheduled work, shared storage, logs and backup artifacts supplied operational evidence. The independent Python collector, published JSON snapshots and SQLite alert history supported observation; they did not replace transaction processing or require dashboard-serving AI inference.
Interface evidence
The dashboard,
with demonstration data.
03 / Read-only investigation
Find the last proven step.
Then the next safe check.
September tools examined open queues, selected orders and related outbound and return messages without editing business transactions. An order journey organized the observations for responders, rather than pretending one record proved the entire outcome.
Expected waits matter. Warehouse activity without ERP confirmation could require checking a return message, allocation state or the next freight-processing window, not declaring failure.
Current identity matters. A historical routing mismatch needed corroboration against the current transaction before becoming a critical alert.
A duplicate is not an acknowledgment. Matching hashes identify identical local files, not partner acceptance. They do not authorize retransmission; resending remained a separate human decision.
Conceptual order journey
Direction, evidence
and expected waits.
04 / Accountable exceptions
Keep the evidence.
Review the correction.
The exception engine combined current evidence with expected timing. An independent SQLite store persisted severity, affected record, evidence snapshot, rule identity, timestamps and current or cleared state. That local write ownership did not grant permission to modify orders or ERP transactions.
Experts graded findings as real issues or false alarms, retaining reviewer identity, time, notes and evidence. Rule revisions required review. A comment did not retrain a model or automatically change production logic. The monitor did not autonomously resend files, restart services, edit schedules or repair transactions.
What the improvement does not quantify
Revised gating distinguished historical flags and legitimate waits from corroborated current faults. The record describes a smaller set of actionable persisted exceptions, but supplies no labeled baseline and follow-up for numeric alert precision, fault-finding time reduction or financial return.
05 / The engineering lesson
Make knowledge durable.
Keep AI accountable.
The result was shared support knowledge, recurring observation, specialist investigation and reviewable exceptions around the existing platform. This demonstrates accountable AI-assisted engineering, not a deployed RAG service, vector database or end-user chatbot. Engineering models were investigation tools, not dashboard-serving dependencies; no dedicated GPU inference requirement was established for the monitor.
These are documented July to September 2026 milestones, not a fresh live audit. They do not prove every undocumented behavior was recovered, uninterrupted availability or complete transaction correlation.
Next steps, not delivered features
A future phase could measure precision from graded history, improve correlation where reliable identifiers exist and evaluate private question-and-answer access. Legacy service identities and mount dependencies need a separate change plan. None of these roadmap items is established as delivered by the September record.
The wider picture