How reliable is personal DNA testing?

Kevin Davies sent a sample of his saliva to a genetic testing laboratory in Iceland to learn about his health risks. When he received his results, Mr. Davies learned that, based on his genetic makeup, he had an above-average risk of contracting prostate cancer.

Out of curiosity, he checked back three months later and found that the company, called deCODE, had changed its assessment: His risk was now below average.

DeCODE had recalculated its algorithm, based on new data. Davies, who is himself a geneticist by training, wasn't too surprised by this about-face: "The information that these companies can give you can change and evolve over time," he says.

That isn't the only way today's genetic tests offer slippery conclusions. According to a US government study, results often vary widely among genetic-testing companies, largely because each has its own way of choosing and analyzing data.

When the project to map human DNA was finally completed in 2003, many predicted a revolution. Drugs could be chosen to match individual patients with maximum therapeutic effect and minimum side effects, the advent of so-called personalized medicine.

But a summer spate of troubling stumbles for genetic-testing companies and programs shows just how long and twisting the road can be between advances in basic scientific research and their application.

It also has spotlighted the question of how medicine will be practiced in an era when anyone can research ailments and treatments on the Internet, sometimes becoming more familiar with new therapies and tests than their physicians.

In July, a report from the US Government Accountability Office presented at a congressional hearing on genetic testing roiled the waters. The GAO found that direct-to-consumer (DTC) genetic test results were "misleading and of little or no practical use to consumers." As a test, the GAO sent identical saliva samples to four leading companies. It found that disease predictions varied widely, "indicating that identical DNA can yield contradictory results" from each company.

The GAO also found 10 "egregious examples of deceptive marketing" by companies, such as saying they could predict what sports children would be good at. Two companies said it would be fine for a woman to surprise her fiancé with the results of his genetic test, which she would secretly have done on him, an obvious privacy violation.

The problem is "there's no gold standard. There's no way of saying what would be the correct answer" in interpreting the data that companies present to their customers, says Troy Duster, a sociologist at the University of California and New York University who has long studied the ethical issues surrounding genetic research.

"The idea that consumers should be relying on this kind of spurious information whose interpretation is so questionable, and whose utility is so questionable – it's bogus," says Marcy Darnovsky, associate executive director of The Center for Genetics and Society (CGS), a nonprofit group in Berkeley, Calif.

In August, an article in the prestigious New England Journal of Medicine urged that consumers be protected from "unrealistic claims and misinterpretations of complex, dynamic genomic information." In many cases, the authors said, "there is little or no evidence of the clinical validity of genetic tests."

Genetic-testing companies beg to differ.

"We are confident in our service's accuracy and reliability," says a blog entry on the website of 23andMe, a genetic-testing company based in Mountain View, Calif., one of several that were singled out for criticism by the GAO. "It is widely accepted that the technology we are using is sound." The company called the GAO report "deeply flawed" and "unscientific" and noted that the "GAO did not find any problem with the underlying data that we provide...."

In an e-mail, 23andMe spokeswoman Jane Rubinstein adds that the company has met with the Food and Drug Administration and "anticipates actively working with the FDA," including sharing a proposal on ways to regulate the genetic-testing industry. "Because we are in early discussions we do not want to be more specific," Ms. Rubinstein says.

Geneticist Davies, author of the new book "The $1,000 Genome," found that receiving his gene-test results was an "empowering and exhilarating" experience, even though he knew the results should be viewed as very "preliminary." "It's the very early days," he says. "For the most part, the DTC companies do a good job of stressing that, [but] I think they can do a better job."

He's also critical of the GAO report. "It seemed to me they had an answer in mind and then went about proving their thesis," Davies says. "I just didn't like the way it was done at all."

In the view of Hank Greely, director of the Center for Law and the Biosciences at Stanford University, genetic testing is overdue for stronger federal regulation, including a ban on DTC tests done without the supervision of a doctor or other health-care provider.

"If [a test] is health-related, it should go through a health professional ... who can help and advise you as to what it means," he says.

All the leading DTC genetic-testing companies say they offer either in-house counseling or referrals to genetic counselors to help explain the results. But interpreting the data can be tricky business.

Based on identical test results, for example, a customer may be told that she has a five-times-higher-than-average likelihood of contracting a certain disease. Or the same data could be rephrased to say that her chance of contracting that illness are 5 in 1,000 instead of the average of 1 in 1,000, a much less troubling message.

What's more, there's wide agreement that genetic factors play only a partial, and poorly understood, role in the development of disease. Other variables, such as a person's lifestyle and environment, are also part of the equation and make extracting the role of genes more complex.

"The companies are pretending that the science has advanced enough that they can say something. But it hasn't," Dr. Greely says.

The problem of how to properly conduct genetic testing and inform participants arose again in August when the University of California, Berkeley, asked more than 5,000 incoming freshmen and transfer students to submit saliva samples for genetic testing as part of an effort to teach them something about genetic tests. The students were to be tested for three supposedly benign traits: their responses to dairy products, folic acid, and alcohol. The program was halted after serious concerns were raised about it. Those students who did submit samples now will not receive their results.

"You're going to tell 17- and 18-year-olds that they are efficient metabolizers of alcohol? What are they going to do with that information?" CGS's Ms. Darnovsky asks. "That seemed to me to be so irresponsible."

Current genetic tests analyze about 600,000 SNPs (single nucleotide polymorphisms) or 0.1 percent of the entire genome. What these DTC genetic-testing companies are doing is "a dying technology," Greely says. Whole-genome sequencing is a far more elaborate, and potentially much more helpful, look at a person's genetic makeup, he says. The cost was $50,000 last year and is down to $10,000 this year. In two or three years it will be $1,000. "That will be the next direct-to-consumer issue," he says.

"Everybody's whole-genome sequence will say something scary, either for them directly or for children they may have in the future or may have had in the past," he says. "The idea that people are just going to get this [information] over the Internet [without counseling] scares me."