NSA data-mining 101: two 'top secret' programs and what they do

In the name of fighting terrorism, the US government has been mining data collected from phone companies such as Verizon for the past seven years and from Google, Facebook, and other social media firms for at least four years, according to government documents leaked this week to news organizations.

The two surveillance programs – one that collects detailed records of telephone calls, the other that collects data on Internet-based activities such as e-mail, instant messaging, and video conferencing – were publicly revealed in "top secret" documents leaked to the British newspaper the Guardian and the Washington Post. Both are run by the National Security Agency (NSA), the papers reported. 

The existence of the telephone data-mining program was previously known, and civil libertarians have for years complained that it represents a dangerous and overbroad incursion into the privacy of all Americans. What became clear this week were certain details about its operation – such as that the government sweeps up data daily and that a special court has been renewing the program every 90 days since about 2007. But the reports about the Internet-based data-mining program, called PRISM, represent a new revelation, at least to the general public. 

Data-mining can involve the use of automated algorithms to sift through a database for clues as to the existence of a terrorist plot. One member of Congress claimed this week that the telephone data-mining program helped to thwart a significant terrorism incident in the United States "within the last few years," but could not offer more specifics because the whole program is classified. Others in Congress, as well as President Obama and the director of national intelligence, sought to allay concerns of critics that the surveillance programs represent Big Government run amok.

But it would be wrong to suggest that every member of Congress is on board with the sweep of such data mining programs or with the amount of oversight such national-security endeavors get from other branches of government. Some have hinted for years that they find such programs disturbing and an infringement of people's privacy. Here's an overview of these two data-mining programs, and how much oversight they are known to have. 

Phone-record data mining

On Thursday, the Guardian displayed on its website a top-secret court order authorizing the telephone data-collection program. The order, signed by a federal judge on the mysterious Foreign Intelligence Surveillance Court, requires a subsidiary of Verizon to send to the NSA “on an ongoing daily basis” through July its “telephony metadata,” or communications logs, “between the United States and abroad” or “wholly within the United States, including local telephone calls.” 

Such metadata include the phone number calling and the number called, telephone calling card numbers, and time and duration of calls. What's not included is permission for the NSA to record or listen to a phone conversation. That would require a separate court order, federal officials said after the program's details were made public.

After the Guardian published the court's order, it became clear that the document merely renewed a data-collection that has been under way since 2007 – and one that does not target Americans, federal officials said.

“The judicial order that was disclosed in the press is used to support a sensitive intelligence collection operation, on which members of Congress have been fully and repeatedly briefed,” said James Clapper, director of national intelligence, in a statement about the phone surveillance program. “The classified program has been authorized by all three branches of the Government.”

That does not do much to assuage civil libertarians, who complain that the government can use the program to piece together moment-by-moment movements of individuals throughout their day and to identify to whom they speak most often.

Such intelligence operations are permitted by law under Section 215 of the Patriot Act, the so-called “business records” provision. It compels businesses to provide information about their subscribers to the government.

Some companies responded to the new reports, but obliquely, given that by law they cannot comment on the surveillance programs or even confirm their existence.

Randy Milch, general counsel for Verizon, said in an e-mail to employees that he had no comment on the accuracy of the Guardian article, the Washington Post reported. The “alleged order,” he said, contains language that “compels Verizon to respond” to government requests and “forbids Verizon from revealing [the order's] existence.”

PRISM

The existence of PRISM, the Internet-based data-mining program, appeared to take many in Congress by surprise, except for lawmakers serving on intelligence committees, who have been briefed about it. PRISM involves the collection of digital photos, stored data, file transfers, e-mail, chat services, videos, and video conferencing from nine Internet companies, according to a “top secret” document posted on the Washington Post website on Thursday.

What PRISM reveals is NSA’s desire to hunt for terrorist threats where people communicate most these days – over the Internet, cyber experts say. Its existence, unveiled just hours after the initial news story about the wholesale collection of phone records, rattled privacy advocates.

“These revelations are a reminder that Congress has given the executive branch far too much power to invade individual privacy, that existing civil liberties safeguards are grossly inadequate, and that powers exercised entirely in secret, without public accountability of any kind, will certainly be abused,” Jameel Jaffer, ACLU deputy legal director, said in a statement Thursday.

PRISM was apparently launched not long after President George W. Bush’s secret program of warrantless domestic surveillance came to light in 2007, according to a timeline on the top-secret document. Congress enacted the Protect America Act in 2007 and the FISA (Foreign Intelligence Surveillance Act) Amendments Act of 2008, which gave companies immunization from privacy lawsuits when they cooperate with US intelligence agencies seeking to use company data. Soon thereafter, PRISM signed up Microsoft, according to the document on the Post website.

“We provide customer data only when we receive a legally binding order or subpoena to do so, and never on a voluntary basis,” Microsoft said in a statement to the Post. “In addition we only ever comply with orders for requests about specific accounts or identifiers. If the government has a broader voluntary national security program to gather customer data, we don’t participate in it.”

Yahoo also issued a statement. So did Joe Sullivan, chief security officer for Facebook, who told the Post that “we carefully scrutinize any such request for compliance with all applicable laws, and provide information only to the extent required by law.”

Details aren't known concerning how the PRISM operation analyzes data. But PRISM is apparently intended to help catch terrorists and is described in the title page of the top secret NSA document as “the SIGAD Used Most in NSA Reporting.” A “SIGAD” is short for Signals Intelligence Activity Designator, a source of intelligence for the NSA. 

The document points toward a program that collected social media information beginning in 2007 with contributions from Microsoft. A year later, Yahoo was roped in, and then in 2009 Google, Facebook, and PalTalk. YouTube was included starting in 2010, then Skype and AOL in 2011, and finally Apple in October 2012.

Google, for its part, said in a statement to the Guardian that its actions are lawful.

"We disclose user data to government in accordance with the law, and we review all such requests carefully,” its statement said. “From time to time, people allege that we have created a government 'back door' into our systems, but Google does not have a back door for the government to access private user data."

The ACLU's Mr. Jaffer had this foreboding comment about PRISM: “The stories published over the last two days make clear that the NSA – part of the military – now has direct access to every corner of Americans’ digital lives. Unchecked government surveillance presents a grave threat to democratic freedoms.”

Surveillance program oversight

Asked about the secret programs Friday during a press conference, Mr. Obama said they are authorized under existing law and reviewed regularly by congressional committees.

“The programs that have been discussed over the last couple days in the press are secret in the sense that they're classified, but they're not secret in the sense that when it comes to telephone calls, every member of Congress has been briefed on this program,” Obama told reporters. "With respect to all these programs, the relevant intelligence committees are fully briefed....

“These are programs that have been authorized by broad, bipartisan majorities repeatedly since 2006,” he added. “And so, I think at the outset it's important to understand that your duly elected representatives have been consistently informed on exactly what we're doing.”

Some in Congress, including a few Republicans, backed him up on the thrust of the programs. On Thursday, Sens. Dianne Feinstein of California and Saxby Chambliss of Georgia, the top Democrat and the top Republican on the Senate Intelligence Committee, said the court order concerning the telephony metadata program was just a routine reauthorization of a wider program lawmakers had long known about and had approved.

The business-records provision of FISA authorizes the executive branch to collect "metadata" concerning telephone calls, including such things as a telephone number or the length of a call, Senator Feinstein said in a statement.

“This law does not allow the government to listen in on the content of a phone call,” she said. “The executive branch’s use of this authority has been briefed extensively to the Senate and House Intelligence and Judiciary Committees, and detailed information has been made available to all members of Congress prior to each congressional reauthorization of this law."

House Intelligence Committee Chairman Mike Rogers (R) of Michigan said the NSA phone-records collection has helped thwart a terrorism incident in the United States.

But other members of Congress have expressed reservations over such wholesale data-gathering by the government.

“The program Senators Feinstein and Chambliss publicly referred to today is one that I have been concerned about for years,” Sen. Ron Wyden (D) of Oregon said in a statement. “I believe that when law-abiding Americans call their friends, who they call, when they call, and where they call from is private information. Collecting this data about every single phone call that every American makes every day would be a massive invasion of Americans’ privacy.”

The news reports seem certain to revive debate in Congress over whether current intelligence-gathering provisions are overbroad and whether the public should have more information about such programs.

“The problem is: we here in the Senate and the citizens we represent don't know how well any of these safeguards actually work," freshman Sen. Christopher Coons of Delaware said in December, in a statement from the Senate floor. “We know that at least one FISA [Foreign Intelligence Surveillance Act] court has ruled that the surveillance program violated the law,” he continued. “Why? Those who know can't say, and average Americans can't know."