How much is a security flaw worth? An inside look into Yahoo’s bug bounty program

As companies try to balance the need to be transparent with outside researchers while protecting their own sensitive business information, the often opaque bug valuation process can be controversial.

Hackers

Every week, the Paranoids – charged with protecting the digital security of Yahoo's more than 1 billion users – discuss one of the more mysterious parts of the cybersecurity business: How much is a security flaw worth?

On a videoconference with digital security teams spanning New York to California, the Paranoids assess weekly reports from freelance security researchers who say they found flaws in Yahoo’s platforms. There, they decide whether a hacker will get a cash prize as high as $15,000 – or just a box of Yahoo-branded swag.

A relatively new part of the cybersecurity ecosystem, so-called bug bounty programs such as this one give security researchers all over the world an avenue to alert companies to digital flaws and make some cash without fear of prosecution. Their popularity has grown exponentially in recent years, especially as bug bounty coordination firms such as Bugcrowd and HackerOne (which coordinates Yahoo’s bounty program) make it easier for companies to post their programs’ bug-hunting guidelines and cash prize ranges online for throngs of eager hackers.

Yet even as bug bounty programs mature, deciding exactly how much to pay for a bug is “oftentimes more art than science,” says Doug DePerry, senior Paranoid at Yahoo who oversees its bounty program, which has paid out $1.6 million to hackers since its late 2013 inception.

That art is not always easy to explain, even to the researchers who find the flaws. A bug’s value is determined by a wide variety of factors – including how severely it affected the company's security – in a discussion behind closed doors. Companies are grappling with how to communicate the reasoning behind their payment decisions with an army of hackers whose help they are actively soliciting, while still protecting their own digital security and safeguarding sensitive business information.

It’s a delicate balance, and while some bug hunters may be happy to find out they're getting a nice chunk of change – some, Mr. DePerry says, have made over six figures from Yahoo bounties in the last year alone – others may end up feeling jilted. 

“Whether you’ve done 10 minutes or 10 hours of work to submit a bug, you only have what you perceive to be a security issue,” DePerry says. “Unfortunately, from their perspective, the payment process can be a little opaque. That’s something we’re working to rectify but it’s sensitive, because that can have to do with sensitive company information. At the end of the day, this is a business – you show your hand too much in security and it’s going to bite you.”

A case that made headlines in the tech press this week illustrates this challenge. Security researcher Behrouz Sadeghipour last week discovered that a vulnerability that plagued ImageMagick, a popular image-processing software suite, could also be used to target Polyvore, a fashion e-commerce website acquired by Yahoo last year. Mr. Sadeghipour filed a report explaining how he uploaded a fake image file, gleaned from that previously known ImageMagick vulnerability, as his profile picture on Polyvore and was able to access its server.

Sadeghipour made $2,000 and Yahoo says it patched the flaw in under two hours. But Sadeghipour says it wasn’t enough. “I thought I’d be paid more because of the severity of this vulnerability,” he said.

In deciding how much a bug is worth, the Paranoids ask each other some key questions. Where does the vulnerability sit on the network? Was any kind of data compromised? Was that data sensitive? “For the most part, the type of vulnerabilities that can affect a larger population are worth more money, because typically they’re few and far between, and because patching that one security hole can greatly increase your security,” DePerry says. “A flaw that affects hundreds or thousands or millions of users is a big deal,” he says. “That’s worth good money to me."

In this case, for instance, Yahoo says it paid Sadeghipour that particular amount because the ImageMagick vulnerability he used was already public; DePerry says his team already knew about it. ImageMagick is also a third party library – meaning Yahoo did not write the code where the original bug was found. What’s more, the issue Sadeghipour focused on was in Polyvore, rather than one of Yahoo’s core domains, which does not store sensitive data or have access to Yahoo user data.

For his part, Sadeghipour says he had no way of verifying whether there was actually sensitive information accessible from that server because the constraints of the bug bounty program forbade him from trying to leverage his access to infiltrate the company further. “I got underpaid, but there’s not much to do about it,” he says. “It is what it is. The fact that I can report it and still not get sued for it is still better than nothing.”

As bug bounties grow more mainstream, there are even more opportunities for hackers to make money than ever before. Tech companies such as Google and Twitter are not the only ones with bounty programs – automakers such as Tesla and General Motors, and even financial services including Western Union and Square, have hopped on the vulnerability disclosure program bandwagon.

The competition has created a largely self-regulated market, since, if researchers don’t like the prices companies are willing to pay, they can try another program to see if it’s more lucrative, says Katie Moussouris, who consults companies and governments on vulnerability disclosure programs as founder of Luta Security. “That’s the beauty of an open market,” she says. “When they discover a bug, hackers have a choice about what to do with it.”

In some cases, the companies responsible for the products might not be the only ones interested in buying bugs. As the defense market and bug bounty programs mature and become more professional – so, too, is the dynamic underground market to buy and sell vulnerabilities for cyberattacks.

Some these companies on the so-called “offense market” have started advertising what they will pay for previously unknown hacking tools they can resell to customers – and not back to the company for fixing. Premium exploit acquisition firm Zerodium, for instance, boasts that it pays “big bounties, not bug bounties”: It shelled out $1 million for a new tool to remotely hack an iPhone last year.

Other price points were revealed in e-mails leaked from Hacking Team, an Italian company that sells surveillance tools and malware to governments and companies around the world, which Moussouris notes appeared to assign greater value to those that were sold exclusively to them – so that they could be exploited for longer.

Even the FBI had to quantify, at least internally for the purposes of a secret contract, exactly how much one flaw was worth to its investigation. FBI Director James Comey recently hinted that the government may have paid around $1 million to an undisclosed contractor to hack into the iPhone 5C used by the San Bernardino, Calif., shooter after Apple refused to help bypass built-in security features on the device. Though later reports cast doubts on his public estimate, this type of open, such public discussion and speculation about prices for bugs on both offense and defense markets is a relatively new phenomenon – “and it will be interesting to see how it plays out” in the hacker community, Moussouris says.

The newfound transparency has made one thing clear: “The prices for people buying vulnerabilities in order to use them for attacks is always going to be much higher than the defense market, which includes bug bounties,” Moussouris says.

But hackers aren’t exclusively motivated by money. The growing convenience of bug bounty programs can be alluring. “Instead of finding an offense market or a buyer, it’s now very straightforward now to go to an organization and report a bug – and not be arrested,” Moussouris says. What’s more: If the hackers are themselves consumers of those products, their own security will improve if their bug is patched.

So will everyone else’s.

And that’s why companies like Yahoo say they are taking the transparency challenge seriously. “The more fairly you treat your researchers, the more likely they are to come back and continue looking at your code, and with the overall breadth and depth of Yahoo we need all the eyeballs we can get,” Yahoo's DePerry says.

In the coming weeks, Yahoo is planning to release a blog post explaining more in detail the inner workings of how it assigns bounties to bugs, and internally, decide on more clear guidelines for that process.

At the end of the day, though, “it will never be 100 percent science,” DePerry says.