Offshore drilling moratorium: US cites blowout preventers' weak spot

The Obama administration on Monday cited control systems on subsea blowout preventers as one reason for its offshore drilling moratorium. But more than a year before the BP oil spill, authorities learned that balky control systems were the most common cause of blowout preventer failure.

Before he leaped into a lifeboat in the middle of the night on April 20, Christopher Pleasant tried to trigger a deep-sea safety device to squelch the oil-well blowout and fire raging aboard the Deepwater Horizon drill rig.

In those last frantic minutes in the rig's central control room, Mr. Pleasant – a subsea superviser for Transocean, the rig's owner – pressed two buttons simultaneously to activate a 450-ton blowout preventer sitting 5,300 feet below the sea surface. If its massive "shear ram" valve closed, it would slice through the drill pipe and stop the torrent of burning gas.

For one fleeting moment, the control panel lights offered a ray of hope – showing the shear ram and other BOP valves apparently closing. Valve indicator lights flicked from green (open) to red (closed), he told federal investigators in May in New Orleans.

Yet something wasn't right. Another set of gauges just above the BOP control panel showed no hydraulic pressure at all – "no flow," he testified. Also, the fire was still burning. So, despite BOP control system lights showing the shear-ram closed, it was not.

"I knew it was time to leave," Pleasant testified.

No one knows yet just why the rig's BOP did not work – or why the device's hydraulic-electric control system gave Pleasant the wrong readings. That mystery won't be solved unless the BOP is pulled off the sea floor months from now. But when the Obama administration on Monday issued a revised offshore drilling moratorium in the Gulf, it cited fresh concerns about the reliability of BOP control systems as one reason for its action.

Indeed, more than a year before Pleasant's frantic efforts to stop an inferno, a large study of BOP reliability in the Gulf of Mexico had warned industry experts and federal safety officials that balky control systems were by far the most common cause of BOP failure – and apparently getting worse.

Altogether, 63 percent of blowout preventer test failures cited in that 2009 study, a joint effort by the industry and the regulatory US Minerals Management Service (MMS), involved control systems. By contrast, a similar study a decade earlier had found control systems were responsible for 51 percent of BOP failures.

Control systems are vulnerable to leaks

Blowout preventer control systems are hydraulic and electrical units housed in two waterproof pods – the electronic brains of the school-bus-size BOP stacks. The pods – one blue, one yellow – are identical systems, each a backup for the other. Hydraulic and electrical conduits run from the rig deck above the ocean's surface to the pods on the sea floor.

But these control systems, many of them newer-generation units activated by microprocessors, are vulnerable to leaks and failures that can render a BOP useless, according to the closely held study, first reported by the Monitor.

Those microprocessor-based, or "multiplex," control systems enable BOPs to respond in an instant to an electrical signal from the rig – a rate much faster in deep water than that of older, hydraulic-only systems. But there is a downside, too.

"These systems are newer within the industry and thus efforts to improve reliability have not been as extensive," the 2009 study found. They "are much more complex, with more subsea components."

Because the newer systems "have electrical and electronic components in the water; leaks are more likely to have significant consequences in these systems," the report warned. After noting that hydraulic fluid may leak to the sea in some circumstances, the report states: "Leaks in electrical systems can be expected to render them inoperative."

It was not the first such warning. A smaller 1999 study of BOP reliability in the Gulf had warned that "a single leakage can jeopardize complete BOP control."

But the big new 2009 study, funded by the industry and the MMS, was focused on finding ways to cut the cost of BOP safety testing – with potential savings of about $193 million annually, the study's prospectus estimated.

In fact, the study authors – West Engineering Services, a Texas consulting firm specializing in BOP technology – did recommend that the MMS relax its standards concerning how often to test several subsea BOP components. The study, for instance, recommended a reduced schedule for pressure testing of some BOP valves, from at least once every 14 days to once every 35 days. It recommended testing the shear ram valve just once in 77 days instead of once in 30 days. (See related story.)

Emphasis on testing control systems

But because control systems were found to be the part of the BOP most susceptible to failure, the report recommended no change to rules for testing control systems: at least once a week.

"It is important to understand and focus on the fact that control system failures are the most likely category of failures on subsea BOP equipment," the authors wrote. "Even though control systems seem to have the larger margin of failures, due to redundancy there were not any cases where a control system failure would have compromised the well control abilities."

While the control system failure rates may be accurate, the study's conclusions are questionable, some industry veterans say.

Leaks of hydraulic fluid in deep water are the bane of BOP control systems and need more, not less, oversight, says Robert Bea, an engineer at the University of California at Berkeley who was a chief engineer for Shell Oil for 18 years and an expert reviewer for President Obama's recent 30-day safety study on offshore drilling.

The BOP hydraulic lines of the Deepwater Horizon were leaking, according to the drill logs, he says.

"It's like the brakes on your car. Those brakes get mushy when there's a leak," Dr. Bea says. "At some point they just won't work. With a BOP, you don't need much of a leak for that control system's hydraulics to fail."

The complexity of newer BOP control systems makes them vulnerable, says another BOP engineer who asked not to be named because he is still active in the industry. "The biggest problem with a BOP is the control system," he says. "If a BOP has maybe 50 parts, then its control system has 500 parts. Many of them are vulnerable to water leaks."

As part of the Deepwater Horizon's last line of defense, a “dead man’s switch” in the BOP unit was supposed to trigger the shear ram, if both electrical and hydraulic communications with the rig were lost. But this system failed, too.

"We already know one of the battery pods [providing power to the BOP control system] was dead," Bea says. "The other one was functional. But you have this system designed for redundancy, and because of neglected maintenance it is no longer redundant. It's like going parachuting, but without a backup chute. We were relying on this BOP like a parachute – and when it came time, the backup didn't work."

'The problem child'

The study shows that industry representatives reviewing the findings wrestled with their meaning and sought clarification about the vulnerability of control systems on BOPs. In an appendix to the 2009 study, unidentified industry officials commented on control system failures and asked the consultant about it.

"What's being said [in the report] is that the control system is the problem child in the system, but redundancy is the savior?" the industry officials commented.

"Yes, control systems are the problem and redundancy is the savior," the authors responded.

Officials at West Engineering did not return Monitor e-mails or phone calls. MMS officials, along with the Department of Interior, responded to e-mailed questions but refused requests for an interview.

Asked if the MMS had changed, or considered changing, its testing frequency for control systems, MMS said: "No, MMS has always required operators to function test annular and ram BOPs every 7 days between pressure tests."

The MMS added that the 2009 study "is not an MMS study" and therefore the high frequency of control system failures was "proprietary" information that never became part of the agency's safety research program data.

But key people at MMS did know of the control system problem. The agency provided funding for the study, several industry experts told the Monitor. Moreover, a prospectus for the study and the study itself both refer to a close partnership between BP, at least eight other oil companies, and the MMS.

Three senior MMS officials are listed in the study as involved and raising questions about it: Lars Herbst, regional director of the Gulf of Mexico OCS Region; William Hauser, chief of the Regulations and Standards Branch, and Kirk Malstrom, an agency petroleum engineer. The Interior Department, which is coordinating media requests for the MMS during its reorganization, did not grant requests to interview those MMS officials. In review comments on the report, MMS officials asked questions about rig performance, failure detection, and a few other issues. None questioned the report's central finding: the high rate of control system failures.

"This sort of touch-and-feel evolutionary approach to loosening testing standards is not good engineering," says Bea, who has read the study. "I would never want to fly on an aircraft whose safety margins are regulated this way."

Paul Helfer, a former senior engineer with BOP-maker Cameron International, who now does consulting work in the industry, says control system standards need reexamination and a new set of testing requirements.

"It's a pretty good chore to ensure a control system works and stays working," he says. "That has to include some serious testing."

Related:

• Studies suggest MMS knew blowout preventers had 'critical' flaws
• Before BP oil spill, Big Oil-led study urged feds to cut safety testing
• BP oil spill: MMS shortcomings include 'dearth of regulations'