back
#microsoft 365 #azure #outage #automation #network engineering 3 min

Microsoft 365 Outage: Automation Error Cripples Services

A maintenance error in an automated system caused a massive Microsoft 365 and Azure outage on July 23, 2026. Learn how it happened and which services were affected.

Deutsche Version verfügbar — auf Deutsch lesen.

Inhaltsverzeichnis
  1. Faulty maintenance request triggered a routing catastrophe
  2. Cascading service outage – which components went dark?
  3. Response and restoration – timeline and actions
  4. Concrete steps for cloud operators
  5. Practical actions for enterprises

A single mis‑issued command in an automated network‑maintenance routine crippled Microsoft 365 and Azure on July 23 2026.

Faulty maintenance request triggered a routing catastrophe

At 10:44 AM ET on July 23 2026, Microsoft’s internal maintenance‑request tool, which routinely validates and updates IP routes between the West‑US Azure data center and the Microsoft WAN, hit a code‑path error. Instead of removing a handful of stale entries, the script erroneously deleted more devices than intended, cutting off traffic to the Microsoft‑365 gateway clusters. Because the tool lacked an automatic undo for bulk deletions, the network became isolated within minutes.

Cascading service outage – which components went dark?

The routing failure immediately impacted every service that relied on the affected path. Core Microsoft 365 products – SharePoint Online, Microsoft Teams, OneDrive, the Admin Center, and Excel – were taken offline, as were newer offerings such as Fabric, Power BI, Power Apps, Copilot Studio, Windows 365, and Microsoft Defender. Azure‑side components like Azure App Service, Azure Kubernetes Service (AKS) and Azure Cosmos DB also suffered disruptions. Downdetector recorded 2 403 reports at 11:11 AM ET, compared with a typical baseline of 29 reports, with 78 % of complaints targeting SharePoint.

Response and restoration – timeline and actions

Microsoft’s first reaction was an automated traffic‑shift to alternative network paths. This alleviated some load but could not restore the missing routes. After the faulty change set was manually rolled back, the infrastructure began to recover. The rollback occurred at 2:26 PM ET, and by 3:41 PM ET the entire Microsoft‑365 stack returned to normal operation. The incident was logged under ID MO1437424 and triggered immediate reviews of “Safety Checks” across all automated maintenance pipelines.

Concrete steps for cloud operators

The outage demonstrates that automation alone can cause massive disruption without protective layers. Three actionable steps can reduce this risk:

  1. Layered monitoring – Real‑time metrics must capture not only the success of a change but also its impact on routing health. A sudden spike in error alerts should automatically trigger a rollback signal.
  2. Operator gate‑keeping – Before committing any change that affects more than a defined threshold, a human should review and approve the request.
  3. Safety‑check testing – Mandatory simulations and canary rollouts in an isolated environment must be run. A canary test affecting a small percentage of devices would expose connectivity issues before production deployment.

Practical actions for enterprises

For organizations that rely heavily on Microsoft 365 and Azure, the incident yields concrete action items:

  • Geographically diversified backups – Replicate critical data across at least two Azure regions to ensure availability despite a regional outage.
  • SLA verification – Ensure contracts with Microsoft define clear escalation paths and guaranteed recovery times (RTO/RPO) that exceed the standard service‑level agreement.
  • Automated fail‑over drills – Schedule regular, automated tests that simulate switching to alternate network paths and measure actual recovery times.
  • Integrated monitoring dashboards – Feed alerts that detect anomalous routing‑table changes directly into the central IT‑operations console for rapid triage.

By adopting these practices, enterprises can mitigate the risk that a single automated command brings down entire workforces. The July 2026 outage proves that speed without robust safety nets and vigilant oversight can lead to massive disruption.

Sources