Microsoft 365 Services Disrupted for Nearly 24 Hours
A major disruption hit Microsoft 365 services at the end of last week, resulting in one of the largest outages in the company's history.

A major disruption hit Microsoft 365 services at the end of last week, resulting in one of the largest outages in the company's history.
The root cause of the outage was identified as a hidden defect in the cloud gateway that had been dormant since July. This latent bug, combined with changes in traffic and reduced server capacity, led to widespread disruptions affecting much of Microsoft 365 and Exchange infrastructure.
According to a preliminary Post-Incident Report released by Microsoft, the incident occurred between November 12 at 16:20 UTC and November 13 at 16:10 UTC, lasting approximately 24 hours. Monitoring continued through November 14 to ensure affected systems were stable.
The outage had far-reaching consequences, impacting businesses and public offices that rely on Microsoft's cloud and email exchange infrastructure. As a result of the disruption, users experienced difficulties accessing their accounts and services.
Microsoft has categorized the incident as "a technical issue not related to cyber or AI/agents" in its preliminary report, which is available online for those with the necessary admin credentials.
A change implemented in July on Microsoft's cloud gateway had created a bug that led to CPU and memory exhaustion under specific conditions. This defect was not detected during pre-release testing because the hardware used for testing was less powerful than many systems already deployed in production.
These newer machines with higher CPU core counts apparently exacerbated the issue, resulting in a lockout of users who relied on the primary gateway system. Microsoft maintains two independent gateways to ensure resilience, but the switchover from one system to the other during routine network maintenance exposed a previously latent bug.
Traffic had been shifted from the alternate gateway to the primary about 15 hours before the outage occurred. This standard fail-safe measure was intended to keep traffic flowing in case of trouble with the front door, but it inadvertently left the primary gateway handling more traffic than usual.
The bottleneck caused by the changed traffic patterns had another critical effect: it exposed a capacity issue within Microsoft's gateway server fleet. The servers were operating at 14 percent below their optimal capacity due to an existing fault, which would normally provide a buffer against increased demand.
However, when combined with the altered traffic flow and underlying defect, this reduced capacity left the gateways struggling with lock contention as requests continued to pile up. Normally, this missing headroom could have absorbed some of the demand, but in this case it was insufficient.
As US business operations reached their peak, the gateway servers began to freeze, unable to handle the high volume of requests. The backup systems that shifted traffic away from unresponsive servers were met with the same lockouts on supposedly healthy backup gateways, exacerbating the issue.
The failure of mechanisms designed to isolate affected systems by region added to the problem, leaving more than 80 percent of Microsoft's gateway servers frozen in the sweep. It appears that these isolation systems did not function as intended during the outage.
Microsoft's efforts to recover were hindered by the need for a targeted system update from their engineers, which took longer than usual to implement. Restarting the gateway servers was ineffective, as large request queues remained once they came back online, triggering the same resource exhaustion that caused the outage in the first place.
The lingering effects of a July bug continued to hinder Microsoft's efforts to restore its services. Despite patching servers and returning them to service gradually, the problem persisted because the underlying issue remained undetected and unresolved. This meant that reverting to a previous state where things were functioning normally was not an option.
Microsoft has taken steps to prevent similar outages in the future by adding capacity and implementing traffic-distribution guardrails. These measures aim to mitigate the impact of large request queues on its servers, which had previously led to resource exhaustion and service disruptions. Additionally, Microsoft extended delivery timeouts for affected emails to prevent delayed messages from being bounced back as undeliverable.
Microsoft anticipates publishing a comprehensive incident report on November 20, which may provide further insight into the reasons behind the regional isolation systems' failure and the corrective actions planned to prevent such outages in the future. As of Monday morning, all of Microsoft's cloud services were operational, marking a return to normalcy after the disruption.
Facts based on reporting originally published by Ars Technica.
You may republish this story, in full or in part, if you credit Noti Group and link to it (licence CC BY 4.0). Photos are not included.











