How Washington evaluates software vulnerabilities

The US government keeps some security flaws for itself. We take a look inside the secretive process to decide which ones to keep - and which ones to reveal to tech companies.

In August, the National Security Agency (NSA) found itself scrambling to figure out how a group dubbed the Shadow Brokers obtained the agency’s alleged hacking tools, some of which they posted online and others they offered to the highest bidder. The startling breach not only revealed that the NSA seemed to rely on previously unknown security vulnerabilities – called zero-days – in Cisco and Fortinet commercial software to carry out digital espionage campaigns, it also exposed NSA tactics to foreign adversaries.

But the breach may have been most significant — at least in the short term — to networking giant Cisco and digital security firm Fortinet and their customers. The Shadow Brokers revealed unpatched flaws in their systems that criminal hackers and foreign spies could exploit. It remains unclear whether the NSA used these tools for surveillance operations, but it appears the agency kept the flaws from the software vendors, depriving them of a chance to patch their systems.

This dispute between the US intelligence community and the tech sector has gone on for more than a decade. In April 2014, White House Cybersecurity Coordinator Michael Daniel published a blog post detailing the general guidelines by which the US government determines whether to disclose a flaw. The process is known as the Vulnerabilities Equities Process (VEP).

“Disclosing a vulnerability can mean that we forego an opportunity to collect crucial intelligence that could thwart a terrorist attack,” he wrote. But even Mr. Daniel recognized the potential problem of hoarding too many of these flaws, saying that “building up a huge stockpile of undisclosed vulnerabilities while leaving the internet vulnerable and the American people unprotected would not be in our national security interest.”

Daniel listed nine criteria that agencies – which may include representatives from the NSA, CIA, FBI and Homeland Security – involved with the VEP take into account when deciding whether to disclose a vulnerability. The blog post says the agency that finds a vulnerability considers “how much the vulnerable system (is) used in the core internet infrastructure … in the US economy, and/or in national security systems.” The agencies also consider if the vulnerability imposes a significant risk if left unpatched.

So, how many zero-days does NSA keep?

“Nobody has any idea,” said Bruce Schneier, a noted cybersecurity researcher and cryptographer. “Well, some people do — they won’t tell you because it’s classified. So anybody who tells you that they have an idea, doesn’t know...I wish we did, but we don’t.”

But in 2015, NSA Director Adm. Michael Rogers said the agency discloses 91 percent of the serious flaws it finds. Yet that leaves one big question: Does it disclose 91 percent of 10 flaws, or 91 percent of 10,000 flaws? Or does it keep even more vulnerabilities? Jason Healey, a senior research scholar at Columbia University’s School for International and Public Affairs who looked into that question, says his research indicates that the government hangs onto only a few dozen zero-days, at most.

“It didn’t really seem reasonable that NSA is keeping like 5,000,” Healey said. “That means that they would be keeping so many, and we would only be discovering a tiny, tiny, tiny, tiny fraction of them.”

There’s also no indication of how long the NSA waits to disclose a vulnerability.

Ari Schwartz, a former White House cybersecurity adviser, said that most documents related to the VEP are classified for national security reasons. Mr. Schwartz, currently the managing director of cybersecurity services at the law firm Venable, said the exact groups involved in the VEP can’t be disclosed because the government doesn’t want adversaries to “game the system.” But, he said, NSA heads up the process and reviews the zero-days that other government agencies may uncover. But the review isn’t restricted to the intelligence community.

“We emphasize the importance of having nonintelligence agencies as part of the process, such as the Commerce Department, the State Department and the US Trade Representative,” said Peter Swire, a professor of law and ethics at Georgia Tech University Professor, who helped craft the VEP process. “And the Commerce [Department] and Trade Representative are important because there are clearly commercial implications [of the VEP].”

Tech companies have been the main opponents of the government stowing away vulnerabilities. Think about it: If firms aren’t aware of a security hole, they can’t patch it. That means the American public is also affected by the government’s decisions.

“We all use the same technology,” said Chris Soghoian, formerly a principal technologist at the American Civil Liberties Union and currently a TechCongress Congressional Innovation Fellow. “We all use the same laptops, we all use the same web browsers, we all use the same word processing programs.”

Mr. Soghoian’s argument mirrors Apple’s case in its dispute with the government following the 2015 San Bernardino terrorist attacks. Lacking the technical ability to get around security features on the shooter’s iPhone, the FBI took the tech company to court for refusing to comply with a request for special assistance to unlock the device. Apple CEO Tim Cook called the request “chilling” and refused to create what he called “a master key, capable of opening hundreds of millions of locks.”

In the end, Apple didn’t have to comply — the FBI hired a third party contractor to access the device. The FBI has not disclosed the name of the contractor nor the tool it used to hack into the phone. It’s also unclear whether Apple has been able to patch the flaw.

Is the government sacrificing the security interests of its citizens to preserve its own offensive capabilities? Civil liberties advocates think so. “The parts of the government that are most capable of channeling the needs and interests of the American public are not even invited into the room,” said Soghoian, suggesting the Federal Trade Commission plays a part in the VEP process. “You’re really sitting a bunch of wolves around the table asking them how you want to design the hen house.”

Even Schwartz, a former Obama administration official, said the US government could try to assuage concerns by issuing a more in-depth explanation beyond Daniel’s blog post - even “just an unclassified version of the process.”

“Government policy,” Schwartz said. “Especially national security policy, through a blog post isn’t the greatest practice.”

Video by Andrew Merica.