White-hat hackers debug state security systems

States are increasingly paying hackers to break into their systems. These white-hat hackers uncover security flaws to prevent real breaches in the future. 

First Look

Hackers aren’t always sneaky, black-hat cybercriminals out to steal information and wreak havoc. Sometimes, they’re the good guys – ethical hackers who uncover security flaws to help prevent the bad guys from winning.

That community of white-hat hackers is exploding, from tech-savvy high school students who discover bugs on websites to large companies that help businesses and government uncover vulnerabilities within their computer networks.

Some states have for several years turned to white-hat companies to see if they’re able to penetrate their systems. Now a handful are also considering edgier “bug bounty” programs that use networks of hackers and reward those who find hidden security flaws.

“The cyber threat is only growing. States are looking at ways to do things creatively,” said Jeffrey McLeod, director of the National Governors Association’s homeland security division. “The goal is to find vulnerabilities before something happens.”

Some of those vulnerabilities are discovered by those on the outside. Nearly half of state information technology officials reported in 2016 that they sometimes used third parties to try to penetrate their systems; one third said they did so at least once a year, according to a study by the National Association of State Chief Information Officers and the consulting firm Deloitte & Touche LLP.

There are reasons more aren’t using the service: Some states might not have the money, or might be nervous about allowing white-hat companies to try to breach their networks.

But states that have been doing it say it’s a valuable exercise.

“It’s peeling back the onion. We’re challenging the company to do what any competent hacker would do to try to break into our systems,” said Elayne Starkey, Delaware’s chief security officer, whose office hires white-hat companies to do penetration testing regularly at a cost of $10,000 to $25,000.

They have simulated threats. They have set up phishing scenarios and sent fake emails to employees. One time, they even had a tester put on a uniform and pretend to be a delivery man to see how far he could get inside the data center, Ms. Starkey said.

“The results of these tests allow us to tighten up our defenses and close gaps before the real bad guys find them.” How far the fake delivery guy got, she wouldn’t say.

Missouri also hires white-hat companies. One conducted exercises this year in which hackers pretended to be black hats trying to get into the network any way they could, without the knowledge of state employees. The idea was to test staffers’ readiness and how they would respond to well-armed bad guys. The state paid about $90,000 for the tests, which lasted several weeks.

“This gives you a good idea how well your organization can respond to a sophisticated adversary,” said Missouri’s chief information security officer, Michael Roling.

Bug bounties

Hackers and cybercriminals have become increasingly sophisticated and are constantly scanning state computer networks looking for vulnerabilities. In recent years, they have stepped up attacks on those networks, which contain personal information such as the Social Security, bank account, and credit card numbers of millions of people and businesses.

In Missouri, Mr. Roling said the state’s firewall each day blocks 95 million unwanted attempts to get into the computer network. That compares with about 100 million to 120 million legitimate connections a day. So far, the state hasn’t had a major data breach, but Roling knows that could change at any moment.

That’s why he is interested in trying a more nontraditional method of connecting to white-hat hackers: bug bounties. His office is in discussions with multiple bug bounty services to figure out how the procurement process would work; then it will examine the legal implications.

With bug bounties, ethical hackers are given rewards, usually money, for finding and reporting undiscovered “bugs,” which are errors, flaws, or faults within computer networks and data systems. Reporting a bug can earn bounty hunters from several hundred to tens of thousands of dollars.

“It’s crowdsourcing hacking,” said Dan Lohrmann, chief security officer for Security Mentor, a security training firm based in Monterey, Calif., that works with states. “You’ve got a global audience out there. There are people doing this full time, sitting in Norway next to a snow drift, making a living off of it.”

Some cybercriminals send phishing emails to try to gain access to state networks. Some use hacking tools to crack passwords to try to get administrative privileges, or launch denial-of-service attacks.

Big tech companies such as Google, Facebook, and Microsoft have been using bug bounties for several years. The US Department of Defense has used them, too, launching Hack the Pentagon and later Hack the Army and Hack the Air Force. The federal programs awarded bounty hunters more than $300,000 in total for discovering vulnerabilities.

While some companies contact bug bounty hunters directly, others, including the federal government, go through broker-type businesses such as HackerOne and Bugcrowd, both based in San Francisco. They act as middlemen who turn to a network of hackers they say have been vetted. The companies manage the program, triage the hackers’ submissions, and try to ensure that clients get only verified, well-documented reports. They pay hackers a bounty on behalf of their clients.

Bug bounties may be popular in the private sector, but they’re a somewhat controversial concept for states, said Mr. McLeod of the national governors group.

“You’re inviting folks to come and hack your system. That raises red flags for folks,” he said. “Obviously, optics matter. If they find some big gaps in the system, it doesn’t look good for the state.”

Nonetheless, Delaware hopes to start a bug bounty program later this year, said security chief Starkey. If it does, it apparently would become the first state to do so.

To start, the state is creating a disclosure policy and plans to add a link to every Delaware.gov webpage allowing people to click on a button and report a vulnerability. It will set up ground rules for ethical hackers who spot software bugs on public websites and apps but don’t know how to report them.

The policy will make it clear the state is committed to following up promptly, Starkey said, which is important because hackers can get frustrated if they point out a problem and no one gets back to them. It also will include warnings about what hackers are not allowed to do, such as misuse data or shut down a website. Hackers who report legitimate vulnerabilities may be awarded a certificate of recognition.

Once those changes are completed this summer, Starkey said her office will seek approval to hire a bug bounty company. Initially, it would pay management expenses, not bounties, and only offer hackers public recognition. “Hiring one of these companies is not the Wild West,” she said. “Hackers have to be registered and vetted. We know who they are. There’s a lot more structure to it than meets the eye.”

Red flags

Doug Robinson, executive director of the state chief information officers group, said states that want to start such programs need to perform lots of due diligence.

“You need to have a pretty tight contract that deals with potential liability or injury to the state if they turn out not to be white hats,” he said. “Sometimes these hackers were black hats before. I’d be concerned about that.”

Some cyber experts caution that states may not be able to deal with all the problems that bug bounty hunters may uncover.

“You have to have people who can fix the bugs that are found,” said Katie Moussouris, founder and CEO of Luta Security, a cybersecurity consulting firm based in Kirkland, Wash.

Ms. Moussouris, a former white-hat hacker who started Microsoft’s first bug bounty program and was involved in creating Hack the Pentagon, said states already may be too busy struggling to deal with vulnerabilities they already know about to take on those they don’t.

But state cyber officials interested in bug bounties say they’d rather be proactive and do everything they can to prepare for the inevitable.

“The bad actors are coming after you either way,” Roling said. “So if we can get the white hats on our side, that’s a good thing.”

This article was reported by Stateline, an initiative of the Pew Charitable Trusts.