Case Files

Production Case Studies

Anonymized stories showing the evidence, decisions and recovery work behind real production systems.

Case Study

Resolving NGINX 499 Errors in a Kubernetes Production Environment

An anonymized Kubernetes investigation involving NGINX 499s, upstream failures, PHP-FPM port mismatch and database pressure.

Business risk: Intermittent failures affected user transactions while healthy pod counts suggested the platform was available. Increasing timeouts risked converting visible failures into a larger queue and exhausting workers or database connections.
Resolution: The risky release path remained rollbackable while Service and FPM ports were aligned. Probes were adjusted to represent runtime readiness, and PHP-FPM capacity and database behavior were tuned from measured queue and query evidence rather than by extending…

Managed Kubernetes production workloadRead →

Case Study

Regional failover planning for multiple App Services, Front Door, SQL failover groups and storage replication.

Business risk: A regional loss could produce a misleading partial recovery: Front Door might route successfully while an application used the wrong database role, lacked a storage object, or called a regional…
Resolution: The runbook established ordered gates: confirm incident scope, freeze risky deployments, verify secondary capacity and configuration, change the SQL role where approved, validate storage behavior, warm application instances, run synthetic transactions, then adjust Front Door routing.

Azure App Services, Front Door, SQL Failover Group and replicated storageRead →

Case Study

An anonymized recovery narrative for an EC2 workload flagged for suspicious outbound behavior.

Business risk: The abuse report created two simultaneous risks: continued outbound activity could harm third parties and trigger account restrictions, while an aggressive shutdown could destroy volatile evidence and extend customer downtime.…
Resolution: The recoverable path was a trusted rebuild with the exposed application path patched, reviewed data restored, instance-readable credentials rotated and application code made less writable. Required egress was reopened by dependency instead of restoring unrestricted outbound access.

AWS EC2 and Linux web workloadsRead →