MSNBC (JimO) Scoops more Inside-NASA Shuttle Documents

From Jon Berndt:
"Stuf4" wrote in message

You may have heard that on top of Return To Flight, there is a current
push within some JSC communities (such as CB) to launch -114 with a
brand new untried software load, known as "OI-30" (which stands for
Operational Increment #30, or something like that).

Not the most conservative approach. Bugs are found in each rev. I
guess their hope is that the bugs that get through aren't the
"Texas-size flying cockroach" kind of bugs.

Each OI flies "untried" once, according to your logic. However, "untried" is
a grossly misleading and ignorant use of the word in this case. The software
is run in actual GPC hardware in rigorous testing - after lots of prior
rigorous testing - well prior to flight. It can take years for software
changes to be certified, and make their way into operational loads, and by
then the testing has been exhaustive. Additionally, the new software
release, if I am not mistaken, will include changes that directly address
lessons learned in STS-107.

'Untried' was used to mean that it hasn't been tried in flight. If
you don't like 'untried', we can use 'unflown'.

There is no disagreement here that software has been thoroughly
tested. But if your position is that the software has been
*perfectly* tested, then our opinions have diverged.

What would you have them do?

I see the more conservative route to be staying with software that has
been flown time and again. I would like to see RTF concentrate on
fixing things that were found to be broken. Software, as far as I
know, was not one of those items.

But this post-tragedy software jump is not without precedent. I have
been informed that STS-26 RTF had a new software load that had never
flown before as well. Those involved with verification were extremely
anxious about mishap potential in their unflown software.

This new software, ironically, capitalizes on the advantages of MEDS.

Yes, with safety-conscious software changes. Would you rather they not fly
with software that gives the crew more situational awareness?
[http://www.space.com/missionlaunches...s_020516.html]

A point I've repeated in the MEDS/WLE upgrade discussion is that
something has to break before the benefit of SA makes any practical
difference, so the smarter strategy is to focus your efforts on
*preventing* situations that you would need such ascent SA.

Say you RTF with unflown code and some software glitch forces you into
an abort. Yes, it's great that MEDS will give the crew added SA
(useful for that next failure depth of no comm). But I'd say that it
would have been much smarter to have avoided that software abort
situation by using the previously flown OI.


But there are situations where the new software might help, and as
stated before the final analysis is a risk trade. From what I know, I
see that balance tipped in favor of the conservative path of loading
software that is known to be successful in flight.


~ CT

From Jeff Findley:
(Stuf4) writes:

I would agree that the chances of a shuttle crashing due to a software
glitch is very small. The point was why take a chance when you have
proven flown software available? It becomes a risk trade with any
potential safety gains in the new version.

Because the benefits of the bug fixes and enhancements to the software
might just outweigh the risks, especially considering the high level
of quality in this particular software development group.

I acknowledge validity to that argument. But I hope you see that it
also skirts an "if it isn't broken, fix it anyway" philosophy.


~ CT

"Stuf4" wrote in message

I see the more conservative route to be staying with software that has
been flown time and again. I would like to see RTF concentrate on
fixing things that were found to be broken. Software, as far as I
know, was not one of those items.

Look a little deeper:

STS-112, flying at -2 deg. alpha: debris hits SRB skirt
STS-107, flying at 4 deg. alpha: debris hits wing leading edge

I believe the above figures are correct, but I think you'll get the drift.
Software may not have been a problem, but it can definitely be part of the
solution. If my sources are correct and I have not mis-heard it, a flight
software change will be in place for RTF that flies the stack at a smaller
(or negative) alpha at and around max q. IMHO, the conservative route is to
avoid debris impacts, and this provides a quantifiable way to reduce the
impact probability.

Jon

(Stuf4) writes:

From Jeff Findley:
(Stuf4) writes:

I would agree that the chances of a shuttle crashing due to a software
glitch is very small. The point was why take a chance when you have
proven flown software available? It becomes a risk trade with any
potential safety gains in the new version.

Because the benefits of the bug fixes and enhancements to the software
might just outweigh the risks, especially considering the high level
of quality in this particular software development group.

I acknowledge validity to that argument. But I hope you see that it
also skirts an "if it isn't broken, fix it anyway" philosophy.

The devil is in the details. My point is that you have to look at
what fixes and functional enhancements are in the (extensively ground
tested) new version and compare that to the existing flight tested
version. The flight tested version may very well have known problems
and functional deficiencies that directly or indirectly impact flight
safety.

In the end, you can never improve on the released verion of software
unless you ship patches and/or new versions. When doing so, you
typically run all sorts of "regression testing" and don't allow the
new version to ship with known regressions. This "regression testing"
suite would typically contain all of the tests possible. In the case
of STS software, I'm sure the regression testing is extensive.

Jeff
--
Remove "no" and "spam" from email address to reply.
If it says "This is not spam!", it's surely a lie.

From Jon Berndt:
"Stuf4" wrote in message

I see the more conservative route to be staying with software that has
been flown time and again. I would like to see RTF concentrate on
fixing things that were found to be broken. Software, as far as I
know, was not one of those items.

Look a little deeper:

STS-112, flying at -2 deg. alpha: debris hits SRB skirt
STS-107, flying at 4 deg. alpha: debris hits wing leading edge

I believe the above figures are correct, but I think you'll get the drift.
Software may not have been a problem, but it can definitely be part of the
solution. If my sources are correct and I have not mis-heard it, a flight
software change will be in place for RTF that flies the stack at a smaller
(or negative) alpha at and around max q. IMHO, the conservative route is to
avoid debris impacts, and this provides a quantifiable way to reduce the
impact probability.

I agree with the strategy of avoiding impacts. I'd be extremely
interested to learn more about correlation between ascent AOA and
impacts.


~ CT

From Jeff Findley:

I would agree that the chances of a shuttle crashing due to a software
glitch is very small. The point was why take a chance when you have
proven flown software available? It becomes a risk trade with any
potential safety gains in the new version.

Because the benefits of the bug fixes and enhancements to the software
might just outweigh the risks, especially considering the high level
of quality in this particular software development group.

I acknowledge validity to that argument. But I hope you see that it
also skirts an "if it isn't broken, fix it anyway" philosophy.

The devil is in the details. My point is that you have to look at
what fixes and functional enhancements are in the (extensively ground
tested) new version and compare that to the existing flight tested
version. The flight tested version may very well have known problems
and functional deficiencies that directly or indirectly impact flight
safety.

In the end, you can never improve on the released verion of software
unless you ship patches and/or new versions. When doing so, you
typically run all sorts of "regression testing" and don't allow the
new version to ship with known regressions. This "regression testing"
suite would typically contain all of the tests possible. In the case
of STS software, I'm sure the regression testing is extensive.

Those sound like excellent points. I expect that all of this was
taken into consideration by NASA.


~ CT

(Stuf4) writes:

Those sound like excellent points. I expect that all of this was
taken into consideration by NASA.

I know it is. They do extensive ground testing of the software by
running it on the same hardware found in the shuttle. The shuttle
flight software development process is one of the best to be found in
terms of high quality, bug free code.

The software development process I'm used to at work isn't anywhere
near a Level 5, but we have a fairly extensive suite of automated
tests that are run on every build of the software (you've got to do
something productive with your computers at night). Before releasing
to the customer, the complete suite of regression tests are run (these
take a weekend or more to run, even when running in parallel on as
many machines as we can lay our hands on). These are based on
covering functionality as well as past problems and issues encountered
(to prevent them from cropping up again). Any problems found are
flagged as "must-fix before release". We simply don't allow
regression in quality, it's not acceptable to our customers.

This is in addition to alpha and beta testing with both internal users
and actual customers. I'm sure NASA wouldn't release a new version of
the shuttle flight software without many simulations involving actual
astronauts at the controls of the simulators.

Jeff
--
Remove "no" and "spam" from email address to reply.
If it says "This is not spam!", it's surely a lie.

From Jeff Findley:
snip
I'm sure NASA wouldn't release a new version of
the shuttle flight software without many simulations involving actual
astronauts at the controls of the simulators.

I agree. But if given the choice of launching with software that has
gone through many simulations versus software that has gone through
many simulations *and* several flights, I know what I'd choose.


~ CT

"Stuf4" wrote in message

From Jeff Findley:
snip
I'm sure NASA wouldn't release a new version of
the shuttle flight software without many simulations involving actual
astronauts at the controls of the simulators.

I agree. But if given the choice of launching with software that has
gone through many simulations versus software that has gone through
many simulations *and* several flights, I know what I'd choose.

Even if the new software could prevent a known and demonstrated failure
mode?

Jon

I agree. But if given the choice of launching with software that has
gone through many simulations versus software that has gone through
many simulations *and* several flights, I know what I'd choose.

However, if you read some of the available articles on the Shuttle's
avionics, you'll find that a lot, if not most of the bugs are found in
simulation runs. And it's pretty clear why that must be the case: the
hundred-odd flights so far have excercised only the nominal ascent/descent
code plus a very small part of the rest (e.g., on that one ATO). All the
TAL and RTLS code has only ever executed in simulation.

The last bug I can remember being mentioned that actually hit live was
for (IIRC) the Mir rendezvous, when an interative routine was numerically
unstable and didn't converge properly to the LSB.

Jan