NASA Is Said to Loosen Risk Standards for Shuttle
By JOHN SCHWARTZ

Published: April 22, 2005

NASA officials have loosened the standards for what constitutes an
acceptable risk of damage from the kind of debris that led to the
disintegration of the shuttle Columbia as it was returning from space
two years ago, internal documents show.

The move has set off a debate within the agency about whether the
changes are a reasonable reassessment of the hazards of flight or
whether they jettison long-established rules to justify getting back
to space quickly.

Experts who have seen the documents say they do not suggest that the
shuttle Discovery - scheduled to lift off from Cape Canaveral, Fla.,
on May 22 - is unsafe, but a small but forceful minority say they
worry that NASA is repeating a practice that contributed to the
Columbia disaster: playing down risks to continue sending humans into
space.

The documents were given to The New York Times by several NASA
employees, who did so on the condition of anonymity, saying they
feared retribution.

Documents that had been revealed earlier showed that NASA was
struggling to meet safety goals set by the independent board that
investigated the Columbia accident. The new documents suggest that the
agency is looking for ways to justify returning to flight even if it
cannot fully meet those recommendations.

The documents, by engineers and managers for the space agency, show at
least three changes in the statistical methods used in assessing the
risks of debris like ice and insulating foam striking the shuttle
during the launching. Lesser standards must be used to support
accepting the risks of flight, one presentation states, "because we
cannot meet" the traditional standards.

Paul A. Czysz, emeritus professor of aerospace engineering at St.
Louis University, who read the documents at The Times's request, said
they did not demonstrate that the shuttle was too dangerous to return
to space or that NASA was stinting on efforts to make it safer.

To achieve a profound safety improvement, he said, NASA would need to
replace the shuttle fleet, which was designed in the 1970's, with an
entirely new vehicle. But Professor Czysz, who spent some 30 years
with McDonnell Douglas, a NASA contractor, compared the statistical
shifts to moving the goal posts at a football game. "I was amazed at
how they were adjusting every test to make it come out right," he
said.

NASA officials say that the shuttle is safer than it has ever been
because of changes made after the Columbia accident in February 2003,
and they have long acknowledged that not all debris risk can be
eliminated. "There is still going to be a possibility that a golden BB
could get us," N. Wayne Hale Jr., the deputy director of the space
shuttle program, told reporters in briefings this month.

Two years of testing since the loss of the Columbia and its crew of
seven have shown that the shuttle's skin, designed primarily to resist
the blistering heat of re-entry, is far more vulnerable to debris from
the external fuel tank than had been thought.

The tank is filled before launching with 535,000 gallons of liquid
hydrogen and liquid oxygen fuel, and insulating foam keeps the tank
from icing over. From the beginning of the program, NASA rules said no
foam or ice should hit the shuttle. The investigation board found that
managers played down the risks over time as the craft survived
thousands of blows from small pieces of debris.

After the accident, NASA officials initially expressed doubt that the
1.67-pound hunk of foam that struck the left wing could have brought
it down. But tests have since proved that a 0.023-pound piece could
cause catastrophic damage under the worst circumstances. NASA now says
it has reduced the size of debris that will fall off of the tank to
0.01 pound or less, but admits that the only way to know is to monitor
actual launching conditions.

One of the two internal documents, dated Feb. 17, 2005, was written by
John Muratore, the manager of systems engineering and integration for
the shuttle program, and a colleague. It describes ways to compensate
for what it calls "overly conservative" assessments of the ability of
the shuttle to withstand debris impacts, including these:

¶Moving from the traditional worst-case situation certification, or
"worst-on-worst" approach, to "our best estimate of actual
conditions."

¶Reducing safety ratios, which measure capability to withstand
expected impact.

¶Relaxing standards - measured with mathematical models similar to a
6-sigma quality-control process widely used in industry - to allow a
sharply increased rate of failure.

A second presentation was prepared last month. It provides extensive
tables showing the expected failure rates of the carbon composite
panels on the wings' leading edges, expressed as sigma values. Sigma
represents the standard deviations from the mean; in practice, the
higher the sigma number, the lower the probability of failure. A
traditional 3-sigma failure rate is about 1 in 800, Mr. Muratore said,
a 2-sigma is 1 in 40, and a 1-sigma is 1 in 2.

Several tables describing the impact tolerance of the nose cone and
panels show no tolerance for ice impact from several sources under a
worst-on-worst environment. In many cases, even the best-estimate
environment does not meet the impact tolerance goals without dropping
from 3-sigma to 2-sigma or below, and in some cases show "no
capability" of surviving impact. Several of the charts state that
"orbiter certification impact threshold does not meet N.S.T.S.
requirements," meaning that the designation did not meet the National
Space Transportation System's safety requirements.

Mr. Muratore argues that a 2-sigma standard is "not unprecedented" -
that it was used to approve the risks associated with aborting
missions after an engine had failed. In an interview, he said the
process was not intended to move the goal posts.

"In my entire 25-year career I've never jiggled a number," Mr.
Muratore said. The traditional standards of airplane quality
certification are unrealistic for the shuttle, he said.

The engineering challenge is enormously complex, he said. "If you plan
for the absolute worst-on-worst case, it can fake you out into
thinking you can't do anything," he said.

Even after two years of research, he said, "we're just beginning to
get our arms around some of these kinds of numbers."

Moving from a 3-sigma to 2-sigma "is pretty reasonable," he said, if
analysis can show the likelihood of debris of the right size striking
that vulnerable spot with the precise amount of force to cause severe
damage is only 1 in 1000 or 1 in 10,000. He said there had been "a lot
of healthy debate."

NASA is completing its analysis of 177 possible debris sources and is
about to present the data to the task force evaluating whether the
shuttle Discovery is safe enough to lift off as scheduled during the
two-weeks beginning May 22. That group - named for Thomas P. Stafford
and Richard O. Covey, the two former astronauts who are leading it -
postponed its final public meeting late last month and told NASA that
it needed more information, including further details about the
shuttle's ability to withstand impact from debris.

When asked whether the task force had seen the documents obtained by
The Times, David Drachlis, a spokesman, said that it had not but that
"essentially, all of the information you described has been presented
to the task group."

NASA went back to perform additional debris reviews before making a
final presentation to the Stafford-Covey group, which will then
present a report to Dr. Michael D. Griffin, the agency's new
administrator.

Dr. Griffin has said that NASA may return to flight without having
fulfilled all the accident board's recommendations. "Accident boards
make recommendations that seem good to them at the time but which may
not in all cases be capable of implementation," he said last week at
his Senate confirmation hearing.

The notion that NASA is declaring its evaluation tools "overly
conservative" has parallels to the Columbia disaster, when NASA turned
to Boeing engineers for analysis. They used a computer program called
Crater, which predicted potentially severe damage, but the engineers
played down the results on the ground that Crater was a "conservative"
tool, and mission managers squelched further efforts to look for
damage. NASA officials have said the agency now knows more about the
problems of foam and ice, and has taken steps to reduce risk.

Even though it would be virtually impossible to determine the risk of
a serious foam strike in time to abort a future shuttle mission, NASA
is developing methods of repairing the shuttle's skin in orbit and has
developed a risky "safe-haven" plan that would allow astronatus in a
damaged shuttle to remain at the International Space Station for up to
45 days until a rescue mission could be attempted.

A NASA employee who provided the documents said optimism based on past
success is unwarranted, and compared it to saying, "We've run the red
light again and again, and we've gotten away with it."

"They haven't gotten away with it," he continued. "They've destroyed
two orbiters - 14 people are dead."

James Wetherbee, a former shuttle commander and safety official who
recently retired from NASA, said he had attended briefings that
discussed lowering the risk standards a year ago. He expressed concern
about NASA's tendency to oversimplify complex calculations and then
for management to put the best face on the result. Mr. Muratore's
analysis, he said, is honest - if not, it would not show so many areas
of the shuttle failing traditional tests - but "the numbers came out
worse than we thought they would."

He said the shuttle should fly the limited number of missions to
complete the space station, but only after NASA fully explained the
risk to the American people - "and I think people would agree." He
said work must continue to make the shuttle safer. "You can't simply
accept lower standards and decide to go fly," he said. "You must do
something else to earn the privilege," with further redesign to fix
the debris problem and to toughen the leading edges.

But that it is not likely, he said.

"You know what's going to happen? They'll have no problem on this
flight or the next flight," he said, and the issue "won't be on the
front burner any more. We'll forget about it."