In a new report on last summer’s compromise of more than 20 Microsoft customers, including the Department of State, by a Chinese state-affiliated threat group, the Cyber Safety Review Board (CSRB) cited a series of failures in Microsoft’s internal security practices and cloud platform controls that led to the attackers gaining access to a cryptographic signing key that in turn handed them the ability to access victims’ cloud-based email inboxes.

The CSRB report, released Tuesday, is the end product of a months-long review of the intrusions, which were first reported publicly in July, but began in May when the attackers first established access to victim inboxes. The operation may have started as early as 2021, however, when the threat group, known as Storm-0558, compromised a device belonging to a Microsoft engineer. The entire operation hinged on the fact that Storm-0558 actors were able to steal a MIcrosoft Services Account (MSA) signing key that the company had created in 2016. That key enabled the attackers to forge authentication tokens for targeted cloud-based email inboxes and gain complete access to them. Among the victims were several senior United States government officials, including members of Congress and R. Nicholas Burns, the U.S. ambassador to China.

State Department officials were the first to notify Microsoft of the intrusions, and the company initially believed that the attackers had used stolen credentials or some other common method to gain access to the mailboxes. But that notion soon dissipated when Microsoft security investigators realized that the old, but still valid, signing key was being used to forge authentication tokens for both consumer Outlook Web Access accounts as well as enterprise email accounts.

“Such tokens should only come from Microsoft’s identity system, yet these had not. Moreover, tokens used by the threat actor had been digitally signed with a Microsoft Services Account (MSA) 17 cryptographic key that Microsoft had issued in 2016. This particular MSA key should only have been able to sign tokens that worked in consumer OWA, not Enterprise Exchange Online. Finally, this 2016 MSA key was originally intended to be retired in March 2021, but its removal was delayed due to unforeseen challenges associated with hardening the consumer key systems,” the CSRB report says.

“This was the moment that Microsoft realized it had major, overlapping problems: first, someone was using a Microsoft signing key to issue their own tokens; second, the 2016 MSA key in question was no longer supposed to be signing new tokens; and third, someone was using these consumer key-signed tokens to gain access to enterprise email accounts.”

Most of the problems that led to these intrusions are neither unique to Microsoft and its products nor simple to address. Cloud service providers such as Microsoft, Google, Amazon, and others, face serious challenges in delivering complex products to a wide range of customers in a secure and usable way in the face of attacks by state-backed actors as well as cybercrime groups.

“It’s a hard and unimaginable scope. Not many organizations understand the scope and scale of trying to protect the oldest operating system and office productivity apps on the market. Anything that is very successful in the marketplace, you have a duty to protect it while you're trying to innovate at the same time,” said Katie Moussouris, CEO of Luta Security and a member of the CSRB. Moussouris was redacted from the board’s review of this incident because she is a former Microsoft security employee.

The report paints a dispiriting picture of the security practices and culture inside Microsoft, a company that has gone through more than one transformation of its security organization in the past two decades. Beginning with the publication of Bill Gates’s famous Trustworthy Computing memo in 2002, Microsoft began shifting a massive amount of resources toward security, adding new security features to its products and investing in a new security development lifecycle that prioritized secure coding practices. That initiative resulted in a major shift in the company’s culture, the eventual creation of the Microsoft Security Response Center, and significant improvements to the security of Windows and the company’s other major products. But over time, priorities have changed, the Trustworthy Computing Group dissolved, and much of enterprise computing moved to the cloud. Microsoft, like other major platform providers, has had to adapt to these shifts, but the CSRB found that adaptation has not been fast enough.

“Throughout this review, the Board identified a series of Microsoft operational and strategic decisions that collectively point to a corporate culture that deprioritized both enterprise security investments and rigorous risk management,” the report says.

The Storm-0558 group responsible for this operation is no stranger to high-profile intrusions and has been operating for the better part of 20 years. The group is tied to China’s state government and is one of the more capable and longstanding threat groups on the scene.

In its report, the CSRB, which comprises federal government officials and private sector experts, recommended that Microsoft take a hard look at its security culture, controls for its cloud platform, and the way that it prioritizes security in relation to features in its products. The board also recommends that Microsoft develop and publish a plan and timeline to “make fundamental, security-focused reforms” inside the organization.

“In the meantime, Microsoft leadership should consider directing internal Microsoft teams to deprioritize feature developments across the company’s cloud infrastructure and product suite until substantial security improvements have been made in order to preclude competition for resources. In all instances, security risks should be fully and appropriately assessed and addressed before new features are deployed,” the report says.

One of the constant challenges for platform providers who face attacks from APTs is the agility and adaptability of those adversaries. As organizations shift their defenses and resources to address evolving threats, the adversaries respond in kind, creating a constant push-pull between attackers and defenders.

“Microsoft is like the self-healing cement of the Roman empire. You see them take hit after hit over the years and they keep coming back. Adversaries adapt and Microsoft has to adapt, and it keeps going,” Moussouris said.