Is Silicon Valley losing the fight over user data?

News that Yahoo and Apple collect information that can assist with government surveillance highlights a privacy tug-of-war between tech firms and law enforcement. 

Since Edward Snowden revealed that tech companies and the National Security Agency collaborated on US surveillance programs three years ago, Silicon Valley has ramped up efforts to encrypt user information and fight off government requests for data.

But in recent weeks, revelations about how Washington and state law enforcement agencies are digging into internet companies' user data reveal many of the limits that tech firms face when trying to protect users' information.

Last month, The Intercept published documents from the Florida Department of Law Enforcement showing how law enforcement is able to get access to metadata from Apple's Messages app. Metadata can provide information about communications that can reveal who someone messages, when the messages are sent, and the location of the sender. And recent allegations surfaced that Yahoo let US government officials access to all of its users' emails to scan for suspicious messages.

And with criminals and terrorists turning to digital platforms to communicate, US government requests for information on social media accounts have surged. In a report released last month, Twitter reported that government requests for information spiked by 13 percent, with more than 40 percent coming from the US.

Silicon Valley has bucked government requests to hand over user data or provide access to its devices, such as when Apple denied an FBI request to help unlock the San Bernardino, Calif., shooter's iPhone, and many tech firms have enhanced privacy protections. 

But that still may not protect metadata, which is regularly used by law enforcement during investigations. The documents from the Florida Department of Law Enforcement revealed that Apple provided the agency with metadata information after receiving a court order.

"When law enforcement presents us with a valid subpoena or court order, we provide the requested information if it is in our possession," Apple said in a statement to Passcode. "Because iMessage is encrypted end-to-end, we do not have access to the contents of those communications. In some cases, we are able to provide data from server logs that are generated from customers accessing certain apps on their devices."

Collecting metadata might be the digital equivalent to a mailman reading envelopes, taking notes about what was sent to whom and when, and then holding on to that data. This can reveal a lot about a person, such as where they travel and who they communicate with. Former National Security Agency and CIA chief Michael Hayden said in a 2014 debate that the US government kills people based on metadata.

"Metadata alone is really all you need to know about what a person is doing," Wickr Foundation chief executive Nico Sell says. "If you went to people and said that a government agency is keeping an activity log on every single thing they do throughout the day, most of them would not be happy with that arrangement."

Still, many Americans don’t seem to worry as much about metadata leaks as they care about the actual contents of their communications, documents, and other digitally stored information.

In a 2013 survey of US internet users by Pew Research, 68 percent of respondents said it’s "very important" that access to the contents of their emails is restricted. That could at least partly explain the lack of uproar over Apple gathering data from Messages.

Efforts to constrain metadata collection have had some success. Last year, bowing to public pressure from Mr. Snowden’s disclosures, President Obama signed the USA Freedom Act into law, which could curb NSA metadata collection by transferring that authority to telecommunications companies.

But Ms. Sell says this type of data collection is most damaging to journalists, activists, and others who could be in danger if the people with whom they communicate is revealed. That’s why Wickr, the secure communications app Sell co-founded, says it doesn’t store any user metadata. 

"People are making switches because of news like this – it just needs to be explained to them," Sell says. "Everyone thought, 'Oh, it's encrypted, I'm safe,' but they need a better understanding of what's actually happening."