471st Advisory Committee on Reactor Safeguards (ACRS) - April 5, 2000

UNITED STATES OF AMERICA
NUCLEAR REGULATORY COMMISSION
***
471ST ADVISORY COMMITTEE ON
REACTOR SAFEGUARDS (ACRS)

Conference 2B3
Two White Flint North
11545 Rockville Pike
Rockville, Maryland
Wednesday, April 5, 2000

The Committee met, pursuant to notice, at 8:30
a.m.

MEMBERS PRESENT:
DANA A. POWERS, Chairman, ACRS
GEORGE APOSTOLAKIS, Vice-Chairman, ACRS
THOMAS KRESS, ACRS Member
JOHN D. SIEBER, ACRS Member
GRAHAM B. WALLIS, ACRS Member
ROBERT L. SEALE, ACRS Member
WILLIAM J. SHACK, ACRS Member
ROBERT E. UHRIG, ACRS Member
JOHN J. BARTON, ACRS Members
MARIO V. BONACA, ACRS Member
PARTICIPANTS:
JOHN LARKINS, Executive Director, ACRS
MARK RUBIN, NRR
GLENN KELLY, NRR
MEDHAT EL-ZEFTAWY, ACRS Staff
PAUL A. BOEHNERT, ACRS Staff
MIKE CHEOK, NRR
GARETH PARRY, NRR. C O N T E N T S
NUMBER PAGE
1 AGENDA 1
2 DRAFT FINAL TECHNICAL STUDY OF
SPENT FUEL POOL ACCIDENT RISK AT
DECOMMISSIONING NUCLEAR POWER PLANTS 9
3 CONSEQUENCE EVALUATION OF SPENT
FUEL POOL ACCIDENTS 44
4 RESEARCH PLAN FOR DIGITAL INSTRUMENTATON
AND CONTROL (I & C) 92
5 RISK-BASED PERFORMANCE INDICATORS 164
6 NRC PROGRAM ON HUMAN PERFORMANCE IN
NUCLEAR POWER PLANT SAFETY 240
7 NRR HUMAN PERFORMANCE ACTIVITIES IN
REACTOR OVERSIGHT PROCESS 254. P R O C E E D I N G S
[8:31 a.m.]
DR. POWERS: The meeting will now come to order.
This is the first day of the 471st meeting of the Advisory
Committee on Reactor Safeguards.
During today's meeting, the committee will
consider the following; spent fuel pool accident risk for
decommissioning plants, proposed research plan for digital
instrumentation and control, proposed white paper on
risk-based performance indicators, human performance
program, proposed ACRS reports.
The meeting is being conducted in accordance with
the provisions of the Federal Advisory Committee Act. Dr.
John T. Larkins is the designated Federal official for the
initial portion of the meeting.
We have received no written statements or requests
for time to make oral statements from members of the public
regarding today's session.
A transcript of portions of the meeting is being
kept and it is requested that speakers use one of the
microphones, identify themselves, and speak with sufficient
clarity and volume so they can be readily heard.
I will begin with some items of current interest.
First, I would like to note there has been a change in the
agenda for Thursday's meeting. The briefing we have
scheduled on the reactor trip on loss of AC power at Indian
Point Unit 2 scheduled between 10:00 and 11:15 a.m. on
Thursday, April 6, 2000 has been postponed to the June ACRS
meeting, at the request of the NRC staff. Quite frankly,
there is enough going on there, it would be an imposition on
the staff to ask them to prepare a presentation and come
talk to us.
MR. BARTON: In June, will we hear both events?
DR. POWERS: I think we'll hear soup-to-nuts, at
least as much of the nuts as we can understand at that time.
MR. BARTON: Thank you.
DR. POWERS: I would also like to introduce to the
committee Barbara Jordan. She is working in Sam's office on
a rotational assignment from RES and -- how long, three
months, six months?
MS. JORDAN: Six months.
DR. POWERS: Six months you'll be with us.
Welcome aboard, Barbara.
MS. JORDAN: Thank you.
DR. POWERS: I will remind members that the rights
of spring are upon us and it's time to have ethics training
and we all know we need that. So tomorrow at noon we will
be having ethics training. We've generously offered an
additional 15 minutes for you to get lunch and come back up
here and get your training and you will all be ethical
thereafter.
I do have three items of interest in your package,
labeled items of interest. You do have a copy of
Commissioner Merrifield's speech at the Regulatory
Information Conference. I think Commissioner Merrifield
laid out an explicit program of areas that he thinks are
high priority for the coming year, and it's interesting
reading.
Another item of interest or maybe it's interest,
you can judge for yourself, is an extraordinary interview
with the Vice Chairman. I have this with two stars on it,
but I'm taking one away because he failed to mention the
triplet in his interview.
Finally, I will note that there is, in the package
of items of interest, registration forms for an ANS meeting
that you might be interested in attending.
I will also note this blue document is the joint
letter from the ACRS and the ACNW on defense-in-depth and
risk-informed regulatory process. I'd encourage the members
to examine this over the course of the day. This is a joint
committee letter. So we are constrained to approve it,
disapprove it, or approve it with additional comments, but
not to edit it.
With that, I will ask if members have any comments
they want to make in the opening statements.
[No response.]
DR. POWERS: Seeing none, I will turn to the first
item of business, which is the spent fuel pool accident risk
for decommissioning plants. Dr. Kress, I believe you're
going to lead us through this.
DR. KRESS: I'll get us started.
DR. POWERS: You'll get us started. I personally
have a large number of questions about how to look upon this
study. My first reaction to it was to look upon it as
something like the WASH-1400 for decommissioning plants.
Upon looking at it in more detail, I'm not sure I
want to look at it in that august of terms. But if you
could give me some guidance on this matter, I would
appreciate it.
DR. KRESS: Okay. The information is listed under
tab 2 in your book. We know that when a plant goes into
decommissioning, it's still pretty much constrained to
follow most of the regulations that are in place for
operating plants, even though, surely, as time goes by and
decay takes effect on fission products, that the risk is
going down with time.
So as a result, quite often the staff is faced
with a number of exemption requests to the regulations and
they have a number of diverse activities related to
decommissioning rulemaking that they would like to
consolidate and put under one package in an integrated
rulemaking and recognize the fact that the risk is
decreasing with time and see if they can come up with a
rulemaking package that would be risk-informed and
appropriate for decommissioning plants.
I guess that's all the introduction I need to
make, because we'll hear the details of the technical
analysis that back up, that they've made to back up this
rulemaking activity.
With that, I guess I'll turn it over to Mark
Rubin.
MR. RUBIN: Good morning, Dr. Kress and committee
members. I'm Mark Rubin. I am acting Branch Chief for the
day for the PRA Branch in NRR. We're here this morning to
discuss the draft report which was issued for public comment
on the spent fuel pool safety assessment for decommissioning
plants.
This was the follow-on work from the preliminary
draft report which was issued last June, after a very quick
sort of insight scoping assessment to help us focus on where
the more significant risks might appear from a spent fuel
pool in a decommissioned plant.
This report, as I said, is out for public comment
right now. The public comment period closes, I believe,
this Friday. We've gotten a few comments which we're
looking at, but we may receive some more.
Mr. Kelly coordinated the risk portion of the PRA
assessment. There's a large number of technical
contributors in both risk areas, as well as other
engineering disciplines, and we have most of the technical
contributors or at least many of them here today to answer
any questions you or any other committee members may have.
With that, I will let Mr. Kelly go through a high
level presentation on the draft report and also including
highlighting a few differences where this report evolved
from the very preliminary study done last year.
DR. POWERS: In the course of the presentation,
will I get some understanding of -- at least when I look at
the documentation associated with this, I get the impression
that this is an important technical basis of what you're
going to try to do and with respect to decommissioning
plants.
And so that it is not just an interim study, it is
going to be a very definitive technical basis for
regulation. Is that a correct understanding?
MR. RUBIN: The final study, yes, will be a
technical basis, proposed as a technical basis for future
rulemaking in the area of decommissioning.
MR. KELLY: This is Glenn Kelly. It will also be
used as the technical basis for exemption requests during
the period while the rulemaking is in process.
I'm Glenn Kelly, with the Probabilistic Safety
Assessment Branch in NRR. I was the coordinator of the risk
assessment portion of the analysis that we did on spent fuel
pool reactors.
As Mark Rubin mentioned, we had a large number of
technical contributors and I think that you may notice a
number of them. We believe they're a very high quality
group.
The members who were there, most of them were
members of what we call the spent fuel pool technical
working group.
Just a little background on how we got here. In
June of 1999, we put out for public comment what I would
consider a pretty high quality draft report identifying the
potential problem areas that you might find at a
decommissioning reactor. We put this out very early in
order to give stakeholders an opportunity to comment on our
early ideas and to get people involved in the process as
quickly as possible.
The June report concluded that zirconium fires can
occur during the first several years after shutdown of the
reactor, that the off-site consequences in the event of a
zirconium fire would be very high, and it concluded that the
frequency of a zirconium fire would be about
two-times-ten-to-the-minus-five per year. This was
dominated by human error.
The June report, as you know, received lots of
comments from stakeholders and the ACRS. At the same time,
the NRC sponsored a technical review by the Idaho National
Engineering and Environmental Laboratory to take a look at
the report, to determine whether or not we had missed any
significant initiators or to help us with also looking at
the human reliability aspects of the report, because
probably more than any other major PRA, this one really came
down to looking at a new area of human reliability.
That is, what happens, how do operators act over
long periods. We're looking at events that take days to
weeks versus normally a reactor, operating reactor events
that occur very, very quickly.
Subsequent to the issuance of the report, the
industry made a series of commitments which we've documented
in our report, in our current draft report, and they
proposed a seismic checklist. The risk of operation of
spent fuel pools was re-quantified and on February 15, we
put out for public comment our current draft.
So what are the major technical results coming from our
re-quantification? In the current draft report, the risk is
significantly reduced and this is primarily due to the
industry's commitments that they made. The human
error-driven sequences which dominated our June draft report
were reduced to about two-times-ten-to-the-minus-seven per
year.
DR. KRESS: Now, this is not just an assessed
reduction. It's a real risk reduction because of
commitments that --
MR. KELLY: That is correct.
DR. KRESS: -- the industry are making to do
things during decommissioning.
MR. KELLY: That's correct. I have a backup slide
that I can use to talk to you a little bit about where those
human error reductions actually came from.
The NEI commitments that we received made a
significant difference in the report. They reduced the
absolute value to dependencies among the human error
probabilities. The frequency of loss of inventory events,
which was also significant, was reduced due to NEI
commitments. And we also found that by taking a look at
some other numbers that we had in recovery of off-site
power, that we could make some improvements there.
DR. POWERS: I guess I was unaware that NEI had
any spent fuel pools.
MR. KELLY: Not the last time we checked, but as
representative of the industry, what they did is they
proposed to us a series of commitments for the industry
which they believe the industry would be willing to follow,
based on, I understand, their discussions with industry
leaders.
MR. RUBIN: If I could add. The specific
commitments that were relied upon in the enhanced modeling
and generally the sensitivity of the result to them are
specifically called out in the report to be captured during
the rulemaking or the exemption process later on.
So you're correct, NEI can't legally commit a
plant, but the commitments, quote, are highlighted so they
can be captured and carried through to completion.
DR. KRESS: So they will be part of the rule
itself.
MR. RUBIN: I can't speak for the rules myself,
but, yes, that's certainly the intent.
DR. KRESS: They could be. Okay.
DR. APOSTOLAKIS: Now, can you tell us a little
bit how the absolute value of the dependencies among human
error is reduced due to some commitment?
MR. KELLY: George, did you want a detailed
explanation of how we reduced it or --
DR. APOSTOLAKIS: We want the details here.
MR. KELLY: The high level answer is that the
commitments allowed us to do things such as assume that we
would be getting independent shift turnovers where each
shift would be looking at the plant in an independent
manner.
This made a big difference in our overall
dependency amongst the human error probabilities. If you'd
like significant details, we have Mike Cheok and Gareth
Parry here, who can --
DR. APOSTOLAKIS: Can you give us a short summary,
Gareth?
DR. POWERS: Including why you believe the numbers
are true.
MR. RUBIN: We'll let Gary and/or Mike Cheok, who
completed the assessment, talk.
DR. APOSTOLAKIS: Why don't you come up here,
Gareth? It's easier to shoot at you. Maybe you can give us
a little bit of background. What were the significant
dependencies and how --
MR. PARRY: Actually, I'm not really -- I wouldn't
necessarily characterize it specifically as pulling on the
dependencies. I think what we -- what we did was we tried
to establish what would be needed in terms of management
directives, operator practices to establish low human error
probabilities.
These included things like making sure that there
was good instrumentation, that there would also back up to
the instrumentation in terms of things like walk-downs that
would be done by the plant staff, so that there were -- in
this sense, this does get over into the dependency issue.
That there would be independent means of identifying that
there were problems at the plant, in sufficient time that
corrective action could be taken.
So to a certain extent, we did introduce a
redundant and diverse means of identifying problems, and I
guess that, in a way, is what Glenn was referring to as a
reduction in the dependency.
DR. KRESS: What was the instrumentation you
referred to?
MR. PARRY: Well, what we were saying is that you
would need to have -- and I think this is one of the NEI
commitments, that there would be instrumentation that would
refer to the pool temperature, there would be level
measurements that would be indicated in the control room to
give you the first line of defense, if you like, and then at
a second level, there would also be local instrumentation
that the people doing the walk-downs would be able to read,
which would be not necessarily powered from the same power
source and something that would just be visual.
DR. KRESS: Primarily pool temperature and level.
MR. PARRY: Temperature and level primarily yes,
but also radiation could be part of it. We were not that
specific in the sense that what we needed was an indication
that there was a problem.
MR. KELLY: The reason for that is that the
thermal margins associated with fuel heat-up and the large
volume of water in the pool are such that it primarily just
requires the fuel handlers who are going into the building
to make sure that they notice that there is a problem.
That's the major thing that we need to make sure happens.
MR. PARRY: I think, if I remember correctly, in
the original study, one of the contentions was that -- one
of the comments from many of the commenters was that we were
not giving enough credit for the identification that the
problems existed, rather than -- not so much that they
couldn't respond, but that we weren't giving enough credit
for recognition that there were problems, and I think that's
fair enough.
And with these commitments in place, I think it
makes it pretty difficult to miss something going on.
DR. POWERS: When I examined the document, I see
that in evaluating these human errors, that you reference
THERP, which is fairly well known; geriatric, but well known
process for estimating human errors.
MR. PARRY: Right.
DR. POWERS: You also cite something called, if I
can get it all right, the SARP human reliability estimation
technique, but you don't provide me a reference there.
Where do I find out about this technique?
MR. PARRY: This is the ASP one, right?
DR. POWERS: It says SARP.
DR. KRESS: I think it is the one that the ASP
people use.
MR. PARRY: It's the one that the ASP people use.
It was a --
DR. APOSTOLAKIS: Is it the SPAR?
DR. POWERS: Maybe it's SPAR. Yes. I'm sorry.
It's SPAR.
MR. PARRY: It's the one that INEL has designed
for use with the SPAR models.
DR. POWERS: And this is the one that --
MR. PARRY: Harold Blackman has done it.
DR. POWERS: There's nothing in the reference list
that leads me to SPAR HRE methodology. I do notice that
some of your critics felt uninformed about that technology,
if that's the right one, something invented by Idaho and not
exposed to the kinds of peer review and technical review
that some of the other HR, human reliability models have.
MR. PARRY: Yes. Actually, there is a reference
to it, it's reference nine in the report.
DR. POWERS: There is?
MR. PARRY: Yes.
DR. POWERS: Tell me what page to hit.
MR. KELLY: Appendix 2-A, page 66. It's Appendix
2-A. Under Appendix 2, it's the first sub-appendix under
Appendix 2.
DR. POWERS: There it is. Which reference is it?
MR. PARRY: Number nine, bias.
DR. POWERS: Okay. So it's a draft document.
MR. PARRY: Right.
DR. POWERS: And the acronym used in the text is
SPAR and here it's ASP. It's a dialectic model or something
like that.
MR. PARRY: Dialectic.
DR. APOSTOLAKIS: Is it different from THERP?
MR. PARRY: Yes. It's effectively a -- it's set
up in terms of looking at performance shaping factors and
how they impact basic error probability. It's different
from THERP in the sense that in THERP, you build a logic
model of the process and you identify all the different
error.
DR. APOSTOLAKIS: When we say THERP, we're
referring to the human reliability handbook.
MR. PARRY: Yes. It's Wayne's method, right.
DR. APOSTOLAKIS: Okay. Now, the human
reliability handbook does include performance shaping
factors.
MR. PARRY: It does. It does.
DR. APOSTOLAKIS: There is a long discussion.
MR. PARRY: But it also requires you to build
human reliability fault tree models for each of the --
DR. APOSTOLAKIS: If necessary.
MR. PARRY: If necessary, right. In this case, I
think this SPAR approach is more akin to something like
HART, if you're familiar with that, George.
DR. APOSTOLAKIS: Say it again.
MR. PARRY: It's more like HART.
DR. APOSTOLAKIS: Which really excited us last
time we looked at it, right? When we looked at the ATHENA.
MR. PARRY: Right. But, again, it splits things
up into two different portions, the cognitive part and the
execution part, and it deals with them separately.
DR. APOSTOLAKIS: I think you are giving me now
some thoughts to put in the letter on the human performance
program. I agree with Dr. Powers that we can't really
develop new models all the time.
MR. PARRY: Right.
DR. APOSTOLAKIS: And say, you know, in the SPAR,
we're going to use this and in other situations we're going
to use the reliability handbook, and then in other
situations we're going to use ATHENA.
I think the issue of peer review is really very
important.
MR. PARRY: Right. But to give you an indication
of why we chose to go down this path, there are some things
for which -- there are some elements of this analysis for
which we think THERP are appropriate. For example, the
response to alarms, that's a classic case where THERP is
used and is appropriate.
Other areas which are things like the performance
of administrative procedures, which is like during the
walk-downs and making sure that they're doing, things like
that, again, THERP fits nicely into that.
Where we use this ASP model is in the response
areas, which --
DR. APOSTOLAKIS: But that's ATHENA now. If it's
in the response area, we're getting into ATHENA.
MR. PARRY: Yes. But ATHENA, as you know, is not
a fully operational HRA method yet. And besides, I think to
-- really, if you than about ATHENA, ATHENA, you need a lot
more information than we have available typically. ATHENA
is meant to be a process whereby you're looking at very
detailed accident sequences to look for opportunities for
errors of commission, primarily.
DR. APOSTOLAKIS: But in this case, clearly, one could
define an error forcing context and you don't necessarily
have to go to ATHEANA's detailed analysis of that and say
the error forcing context here is perhaps they don't see the
level indication. Right?
MR. PARRY: Yes. At that level, I think we're
consistent with the ATHENA type of concept and, in fact, I
think we've designed it that way. But in terms of the
quantification model, which is really where we're heading to
now, as you know, ATHENA does not really have a
quantification model.
DR. APOSTOLAKIS: That's true.
DR. POWERS: I was most interested in this human
reliability model because it produces truly remarkable
results. They must be accurate -- unbelievable numbers of
digits, because there is no error bound put on these
estimates. There's no range given at all.
MR. PARRY: I understand that criticism. However,
I think the intent of this human reliability analysis was
really to illustrate the fact that if you have certain
commitments in place, the error probabilities are going to
be low, because you have the right defenses in place.
DR. APOSTOLAKIS: I think that's a valid argument,
because no matter what model you use, I think commitments
like the ones you mentioned do help you reduce the numbers.
The thing that worries me a little bit here is the
proliferation of HRA models and we're supposed to have a
program on human performance that coordinates these things
and if the agency is investing all this money into ATHENA,
one would think that that would be a model that people would
use, as appropriate.
MR. BARTON: But ATHENA is not applicable to all
situations, George.
DR. APOSTOLAKIS: But the ideas are. That doesn't
mean I can sit down and develop my own. If ATHENA hasn't
done it after all these millions of dollars, can I do it on
my own?
MR. PARRY: I think Mr. Barton has a very good
point, though, that ATHENA is primarily designed to, at
least the current way it's structured, is for between the
ears problems of an operating crew responding to an
accident. The situation here, I think, is a lot different.
We're talking really about organizational failures, because
the time scales that we're talking about are on the orders
typically of several shifts.
So we're not talking about the mindset of a
particular operator as he is responding to a situation.
We're talking about the mindset of an organization that is
coping with a deteriorating situation.
So I think the models are different for good
reasons in that they are dealing with different types of
situations.
DR. APOSTOLAKIS: Okay. I mean, if I look at the
human performance program, I don't recall an item that says
SPAR models, unless I missed it somewhere. It should be
coordinated better. We have a presentation later today on
that and maybe we can raise the issue then.
MR. PARRY: The SPAR model is not meant, I don't
think anyway, as a substitute for a really detailed HRA
model. It's meant to be more of an indicative one, I think,
for use with the ASP program and in a sense, I think that's
what we're doing here. We're making more indicative
analyses than definitive HRA analyses.
As long as we cover the points that -- as long as
we address the issues that we're trying to do, which is to
look at the defenses and see what they buy you, I think this
relatively more crude approach is appropriate.
DR. APOSTOLAKIS: But if the -- what is the new
total frequency of --
MR. KELLY: For HRA or for --
DR. APOSTOLAKIS: For the whole thing.
MR. KELLY: It's less than
three-times-ten-to-the-minus-six per year.
DR. APOSTOLAKIS: And what is the upper bound,
95th percentile?
MR. KELLY: We don't have those type of
percentiles, because we don't have distributions. In most
cases, we have extremely limited data.
DR. APOSTOLAKIS: These distributions in human
error were never based on data anyway. They were really the
judgments of experts.
MR. KELLY: Right.
DR. POWERS: The idea that I have very little
data, therefore, I won't do an uncertainty analysis strikes
me as unusual.
DR. APOSTOLAKIS: Yes.
MR. PARRY: I think one thing that Glenn didn't
say, though, is that that three-times-ten-to-the-minus-six
figure he's talking about, most of that comes from seismic.
I think that that is, in a sense, a bounding assessment.
DR. APOSTOLAKIS: Okay. Let's focus then on the
human error driven sequences, which are
two-ten-to-the-minus-seven. Could this be
ten-to-the-minus-five?
MR. PARRY: I wouldn't think so.
DR. APOSTOLAKIS: In which case I'm back to the
earlier estimate. I mean, shouldn't there be some
uncertainty evaluation here? Here they are. Okay. Where
are the human --
MR. KELLY: This shows how the numbers dropped
from our June 1999 report to our current report. For loss
of off-site power, fire, loss of pool cooling, loss of
inventory, these were primarily due to -- the changes were
due primarily to commitments made by the industry and by the
areas of clarification that the NRC came up with that they
believe need to be done in order to get these numbers down
there.
DR. POWERS: If you look at these numbers and you
say, gee, this is something less than the probability that
the vessel in an operating plant is going to break open, and
so your tendency is why do I worry about this, it's a very
low probability sort of event.
But then you're reminded that these numbers are
after we've done a bunch of things.
MR. KELLY: I'm sorry, they're?
DR. POWERS: They're after we have done -- they're
for situations after we've done a bunch of things.
MR. KELLY: That's correct.
DR. POWERS: Okay. What I'm wondering is, was
there a connection between the estimates before we did
things and what was decided to do or what you decided to do
is just kind of in an ad hoc way and you recalculated it and
found the numbers were very low.
MR. KELLY: The commitments that were made by the
industry were directly tied to the areas that we had
identified as being ones that we felt were important to be
clarified so that we could more -- we could take advantage
of the things we thought industry probably would be doing,
but there were no commitments or requirements for them to
do.
DR. POWERS: Okay.
MR. KELLY: And as a matter of fact, Gareth had
done some draft work that had been transmitted to the
industry and to the stakeholders, where people had an
opportunity to take a look at what were the type of things
that we thought were important in order to come up with the
human error failure probabilities.
DR. KRESS: What does seismic do to you? Does it
break a pipe and drain the pool?
MR. KELLY: No. This is catastrophic failure of
the pool itself.
DR. KRESS: The pool itself.
MR. KELLY: These numbers, seismic numbers are
about three times the SSE.
DR. KRESS: Is that the reason you say they're
bounding, because they're about three times the SSE?
MR. KELLY: The numbers are bounding because of
the -- these are based on looking at generic -- I shouldn't
say generic, but numbers across the industry. So for some
sites, these numbers are clearly very bounding. For others,
they may be closer, they're not as bounding, but we have Dr.
Kennedy here in the audience who can give you the details of
his estimates that he did and how he came up with the number
of three-times-ten-to-the-minus-six, if you'd like to have
him speak on that.
DR. KRESS: I think we would. I would, because
clearly it's the dominant sequence.
MR. KELLY: Dr. Kennedy is here ready to enlighten
you.
DR. KENNEDY: Should I go up there or sit back
here?
DR. KRESS: Up there would be better, if you
could.
DR. KENNEDY: Okay. Bob Kennedy speaking. What
industry committed to is to agree to do a set of screening
criteria on their spent fuel pools. This screening
criteria, if the plant passes it, would give high confidence
of a low probability of failure, essentially about a -- on a
mean one percent conditional probability of failure at
ground motions in the neighborhood of 1.2g spectral
acceleration or a half g peak ground acceleration.
So what I did is start with the assumption that
the plant would pass the screening criteria, but would
barely pass it. Now, some of the plants may pass it by
quite a bit and we're assuming they just barely pass the
screening criteria, but we are assuming they do pass. If
they don't pass, they would have to do something else.
With that set of assumptions, evolving a seismic fragility
curve that's based on that set of assumptions with seismic
hazard curves, we found for the 69 central and eastern U.S.
sites that Livermore hazard curves were developed at, that
only eight of the 69 sites had annual frequencies of failure
in excess of three-times-ten-to-the-minus-six. That's based
on a Livermore hazard study.
If, instead, EPRI hazard curves were used, only
four of the sites, if my memory is right, they studied 60 of
the 69 central and eastern U.S. sites, only four of those
sites would exceed .5-times-ten-to-the-minus six. So the
three-times-ten-to-the-minus-six number, there are eight
sites that would exceed it and pass the screening criteria,
if you use Livermore hazard study. If you use EPRI hazard
study, none of the sites will exceed this. In fact, only
one site exceeds one-times-ten-to-the-minus-six and only
four sites exceed .5-times-ten-to-the-minus-six.
So we're operating in an area where there is
significant uncertainty induced by just simply the
difference in the seismic hazard estimates in the east and
these two studies represent both very high quality studies.
They're in reasonable agreement with each other at the
ten-to-the-minus-four level, but by the time they get down
into the ten-to-the-minus-six level, they're producing
estimates of ground motion that are very dissimilar from
each other.
DR. POWERS: I thought we had a unification of
those two. I thought we had done a study that got rid of
this business of using two different --
DR. KENNEDY: The difference was gotten rid of in
about the ten-to-the-minus-four level, but the difference
still exists -- I mean, typically, it's
ten-to-the-minus-four, ten-to-the-minus-five level on the
hazard curves that are most interesting, from the
traditional seismic PRAs that looked at seismic-induced core
damage.
But now, as you try to go down to the
ten-to-the-minus-six level, these differences are still very
dramatic.
DR. APOSTOLAKIS: So let me understand this, Bob.
You say six sites would exceed three-ten-to-the-minus-six.
DR. KENNEDY: If we use the Livermore hazard
study, eight sites exceed, eight central and eastern sites,
and, of course, the western sites would exceed, under this
assumption that they barely meet a HCLPF capacity of .5 g.
DR. APOSTOLAKIS: Now, what's magical about
three-ten-to-the-minus-six? Why do we worry about whether
we exceed it or not?
MR. KELLY: We didn't.
Three-times-ten-to-the-minus six is not magical. It's where
the numbers turn out and that was the number assuming that
.5 g is the appropriate number for determining the capacity
of the pool.
DR. APOSTOLAKIS: But, I mean, why do I care that
eight sites exceed it? What's so special about it?
MR. KELLY: What we said is --
DR. APOSTOLAKIS: And by the way, Bob, do you
remember by how much? They would be what?
DR. KENNEDY: Yes, I do. If you'll give me a
second, I brought my report with me. One site, if you use
Livermore hazard curve and you, again, assume that the HCLPF
capacity is .5 g and use a fragility curve based on that,
one site, the highest site with the Livermore hazard curve
is basically 14-times-ten-to-the-minus-six.
DR. APOSTOLAKIS: So we are moving into a
ten-to-the-minus-five.
DR. KENNEDY: One site. A second highest site by
that particular -- now, the site that is
14-times-ten-to-the-minus-six by Livermore hazard curve is
.14-times-ten-to-the-minus-six by EPRI hazard curve. So do
keep in mind there is a big difference in these two hazard
curves at this level.
DR. KRESS: When you say site, could there be
multiple plants on those sites? Are we talking about
individual plants?
MR. KELLY: There could be multiple plants.
DR. KENNEDY: There could be more than one unit.
But that's the highest. The next highest is 8.3 and by EPRI
that one is two-times-ten-to-the-minus-six.
DR. APOSTOLAKIS: So the highest then we have is
1.4-ten-to-the-minus-five and that's intended to be, what, a
mean value?
DR. KENNEDY: That's a mean estimate.
DR. APOSTOLAKIS: The mean using the Livermore
curves.
DR. KENNEDY: Using the Livermore curves.
DR. APOSTOLAKIS: So there is model uncertainty
here in the sense of the whole bunch of curves could move to
lower values if one used EPRI.
DR. KENNEDY: Yes.
DR. APOSTOLAKIS: Okay. Why isn't that good
enough?
MR. KELLY: We haven't said that the numbers
weren't good enough. We just said that --
DR. APOSTOLAKIS: But I think you are giving the
wrong impression by comparing with the
three-ten-to-the-minus-six and say, well, gee, certain sites
are above and below. There is nothing about
three-ten-to-the-minus-six that's special and
1.4-ten-to-the-minus-five may very well be an acceptable
number.
MR. KELLY: It might be, yes.
DR. APOSTOLAKIS: Who is going to determine that?
MR. KELLY: Well, if they come in on a
plant-specific basis, we would say that they could -- I
mean, we have no -- there's no number out there for either
core damage frequency or for risk that is the number that a
plant has to meet. What we have here is we have looked at
numbers based on -- and I was going to get into that later
-- based on the Reg Guide 1.174 thought process of large
early releases at the level of
one-times-ten-to-the-minus-five per year being a level at
which we would allow a licensee to come in and make a --
below that, if your LERF was below that level, you'd be able
to make small increases in your LERF.
So we just used that number of
one-times-ten-to-the-minus five in our report as a surrogate
to allow us to judge what was appropriate and as a matter of
fact, the ACRS had suggested to us previously that that was
a good way to go.
DR. APOSTOLAKIS: But let's see now. These are --
but you can't really view this as being changes in the
licensing basis, is it? Are they changes?
MR. KELLY: No, this is not.
MR. RUBIN: This is Mark Rubin, again. Dr.
Apostolakis, if you look at the draft report, we evolved a
proposed, a suggested pool performance frequency guideline
of 1E-to-the-minus-five based on the logic processes in
1.174, which I think we discussed with you at the previous
and you endorsed.
So you're correct. Even the outlier plants would
essentially -- using the higher Livermore curve, would
essentially meet the total screening guideline we suggested,
as appropriate, 1E-to-the-minus-five. So, yes, there is
nothing at all magical about that.
DR. APOSTOLAKIS: But the existing numbers can be
greater than ten-to-the-minus-five and you can't force the
licensee to do anything about it.
MR. RUBIN: That's correct.
DR. APOSTOLAKIS: Right.
MR. RUBIN: Right.
DR. APOSTOLAKIS: So it's only where they're
requesting changes that these numbers become important.
MR. KELLY: This is in the context of licensees
asking for exemptions to essentially operating reactor
regulations.
DR. APOSTOLAKIS: To change them.
MR. RUBIN: Yes, sir.
DR. POWERS: Now, I would say the following. This
is how I would argue this. It's 1.4-ten-to-the-minus-five.
So you are really on the boundary there. Remember the
footnote in the pictures. Increased management attention,
right? That's really where we are now. And then Dr.
Kennedy is telling us this number is a result of one
approach and there is another approach that produces a
number that is lower.
So I'm not even sure that
1.4-ten-to-the-minus-five is the number to use. Chances are
I'm below the line, right?
MR. KELLY: That's correct. What we wanted --
again, it's still a draft, and so these comments are very
welcome. We want to identify sort of a high confidence
bound for a large group of plants where the evaluation here
would be generally representative, that could be used in
looking at exemptions and looking at rulemaking.
As we started going into the analysis, we weren't
trying to draw very fine lines to try to say, well, everyone
is pretty much okay. There were some -- there was enough
variation that we went with this high confidence group,
which we generally covered by the evaluation here, and then
the plants that fell perhaps a little outside, perhaps not,
we'd look at a little closer.
That's the increased management attention aspect.
DR. APOSTOLAKIS: So what it comes down to is that
senior management at the NRC will ultimately have to pass
judgment here in these cases, because you are in that region
that is dark.
DR. KRESS: While we're on the subject of these
regions and the darkness and the increased management
attention, I recognize that the ACRS might have said that
one-times-ten-to-the-minus-five might be a good criterion to
use as an acceptance criterion, but I would like to raise
some questions about that.
In the first place, it was -- in 1.174, that was a
surrogate for prompt fatalities and it was an acceptance
criteria for reactors, of which there might be 100 or so out
there operating for 40 or 60 years.
Then we have decommissioning plants, not reactors,
although there are some similarities, but very few, there's
not that many of them. They're only at risk for a short
time, like five years or so, at any risk. And not only
that, the 1.174 surrogate for prompt fatalities was based on
a source term that was driven by steam oxidation of the
core.
Here we have a situation where the source term is
driven by air oxidation of spent fuel, which gives you an
entirely -- possibly gives you an entirely different mix of
fission products, which puts into question the back
derivation of one-times-ten-to-the-minus-five from a prompt
fatality.
So I don't think I see a proper thinking out of
this acceptance criteria. It doesn't seem to me like it's a
defendable acceptance criteria because it's so different
from 1.174, and I don't see the connection to 1.174 here.
MR. RUBIN: Dr. Kress, there's quite likely a lot
of validity in the general comments that you made. It's not
a one-to-one mapping certainly. There are a number of
differences here.
We tried, though, to at least make those
differences transparent in the draft report to lay out what
our thought processes were that let us sort of meander
around from 1.174 numbers into this proposed guideline.
Very briefly, certainly, the guideline could be
even higher because of the differences in the source term,
some might argue even lower because there possibly are some
lower levels of defense-in-depth for pool versus reactor.
I think what we tried to do was establish some
balance between the thought processes that might lead you up
and might lead you down, and basically what we did was based
on some very good work from the Office of Research in
looking at the source terms and the consequences.
It was pretty clear that this was a highly
undesirable event. Whether it would be precisely the same
as a large early release in a reactor, most probably not.
There would be fewer prompt fatalities. There would still
be a very large release, though, a significant number of
latent fatalities.
DR. KRESS: Let me ask you about this.
MR. RUBIN: Certainly.
DR. KRESS: That study was based on a reactor
source term, a reactor like source term.
MR. KELLY: No, it wasn't. It was based -- the
one that we did, the first one that we showed in the June
report, actually the June report didn't have it, but our
June report was based on knowledge that we had at the time
of using source terms that came out of a Brookhaven report
that were supposed to be for --
MR. RUBIN: Glenn, excuse me. Why don't we let
Jason Schaperow, from Office of Research, who did the
evaluation, provide details to the committee.
DR. APOSTOLAKIS: One last question before Dr.
Kennedy leaves. Did you also do an uncertainty analysis
given, say, Livermore curves?
DR. KENNEDY: No, I did not. But typically
there's large uncertainty. Typically, with Livermore hazard
curves, the median value, for instance, is typically about a
factor of ten less than the mean value. That is some
indication of the uncertainty.
With the EPRI hazard curves, there is somewhat
less uncertainty claimed by EPRI in their hazard curves. So
you could get large uncertainties on these numbers.
DR. APOSTOLAKIS: So that
1.4-ten-to-the-minus-five could be close to
ten-to-the-minus-four, if one accepted the Livermore curves.
DR. KENNEDY: I don't think it could get as high
as -- I tried to -- the numbers I gave are based upon the
mean hazard curve, which tends to be up around the 85th
percent non-exceedance probability on Livermore hazard
curves.
The median number, of which 50 percent above, 50
percent below, would be typically a factor of ten lower. So
the 1.4-times-ten-to-the-minus-five mean would typically be
in the neighborhood of 1.4-times-ten-to-the-minus-six
median.
MR. PARRY: Can I also remind you that the
acceptance guidelines, if we use them, from Reg Guide 1.174
are intended to be compared with mean values.
DR. POWERS: That's right, and I don't know how to
do that. I mean, I don't know how to compare these numbers
to means and subsequently to 1.174. I mean, how do I do
that? I don't know what the statistical significance of
these numbers is.
DR. APOSTOLAKIS: So the first point is that there
is no rigorous uncertainty analysis here. But the second
point is that even though the guide says use mean values, as
you approach the forbidden area and you have increased
management attention, then it seems to me input like the one
Dr. Kennedy just gave us, because important to management to
have an idea what they're talking about, because the mean
values is a convenient way to act, but I would act
differently if I were senior management if an expert told me
the mean -- the 95th percentile could be a factor of five
above or whatever, or it could be a factor of two.
These are important inputs to the deliberation,
even though the guide says use the mean value.
MR. PARRY: I agree. I think perhaps one of the
more important inputs, though, is the fact that there is
such a big difference between the Livermore curves and the
EPRI curves.
DR. APOSTOLAKIS: Of course, there is, and that's
also important.
MR. PARRY: And that's also part of the --
DR. APOSTOLAKIS: Of course. That's why I keep
saying if one accepts the Livermore curves. There is no
question about it.
DR. POWERS: I guess I don't understand. You say
the bigger thing is the difference between the --
MR. PARRY: Well, the modeling uncertainties.
DR. POWERS: Between Livermore and EPRI is at this
range, but at least the numbers quoted to me sounded like
they were like the difference between the median and the
mean for a given curve.
MR. PARRY: In one case, maybe.
DR. KENNEDY: In the case that had the highest
value from Livermore, and the difference between Livermore
and EPRI varies from site to site, and that happens to be
one of the sites where it's one of the bigger differences.
In that particular case, the numbers based upon a mean
Livermore hazard curve were 14-times-ten-to-the-minus-six,
based on a mean EPRI hazard curve were
.14-times-ten-to-the-minus-six, a factor of 100 difference.
I didn't do this study doing convolution of the
uncertainties, but I've done that in many other cases, and
typically the difference from Livermore hazard curves
between the mean and the median values are about a factor of
ten.
So if that held true for this site, and I can't
confirm that, but if it did hold true for this site, the
median numbers from Livermore would be a factor of ten less
than the mean numbers, but the mean EPRI number is a factor
of a 100 less.
So there is -- basically, the two sets of hazard
curves, at ten-to-the-minus-six, do not overlap.
DR. APOSTOLAKIS: Now, you also mentioned in
passing earlier something about the west. What happens in
the west?
DR. KENNEDY: Western sites would have to do
something beyond this screen. This screening criteria that
NEI put out, that I happen to agree with as being a good
screening criteria, is aimed at some fairly simplified
reviews that the plant can do and if they pass those
reviews, demonstrate that they have a high confidence low
probability of failure capacity greater than .5 g, such a
capacity is not adequately high for some of the western
sites.
So they would not be able to use this same
screening criteria to demonstrate adequate seismic
ruggedness. They would have to do further review.
MR. BAGCHI: Can I say something? This is Goutam
Bagchi. I'm a senior level advisor in Division of
Engineering. The biggest difference between east and west
is the hazard curve itself. In the west, the hazard is well
known and driven by some well known seismogenic sources.
The hazard curve, therefore, instead of being
asymptotic, becomes almost vertical and that kind of
uncertainty at the high level, at the low probability level
is just not there.
So we have chosen a ground motion level which is
two times this, as in the report, for the western sites and
we think that's quite adequate. And what has not been said
all this time is that in our preliminary report, many of our
staff members, particularly geophysicists, and I myself
agree with that, that three times the SSE for the eastern
United States, that's the threshold of credible ground
motion.
It is extremely high ground motion. If you can
show a HCLPF level at that kind of ground motion, I don't
think there is any uncertainty, I mean very little
uncertainty beyond that that there will be a catastrophic
failure of these pool structures.
You have to understand how robust and rugged the
pool structures are.
DR. APOSTOLAKIS: Now, three times the SSE is how much?
MR. BAGCHI: Three times the SSE for the eastern
United States at the highest level would be .75 g. There is
no known seismotectonic behavior that could lead to that
kind of a ground motion.
DR. APOSTOLAKIS: How does that fit into what Dr.
Kennedy does?
MR. BAGCHI: It does not. This is a deterministic
consideration and we're saying to you that the hazard curves
that were compared, EPRI versus Livermore, back in 1993
study, were only looked at up to the SSE level. So when you
go beyond the something-times-ten-to-the-minus-four to the
level of something like ten-to-the-minus-five and minus-six,
then the uncertainty zooms out and there has been no direct
correlation or methodological understanding of why those
differences exist.
DR. KENNEDY: I think if you look at the Livermore
hazard curves at the ten-to-the-minus-six level, you get to
ground motions that a number of people in the seismological
community do not feel are really very credible ground
motions in the east.
The EPRI hazard curves tend to curve over and
truncate and it's the difference that Goutam just mentioned
is really the source of difference between the Livermore and
the EPRI hazard curves. The EPRI hazard curves would tend
to follow the approach that many seismologists would agree
with that there is some limits to what this ground motion
might be in the east at reasonably annual frequencies.
Well, I'm not sure ten-to-the-minus-six is
reasonable annual frequencies, but at ten-to-the-minus-six
annual frequencies, there are other seismologists who don't
agree that those limits apply. Livermore hazard curves tend
to -- in their mean, tend to weight that evidence -- not
evidence, but those suppositions stronger than the EPRI ones
do.
At ten-to-the-minus-six ground motions, I think
it's a wild guess what the ground motion might be in the
east at ten-to-the-minus-six annual frequencies, and that
shows up in these hazard curves.
DR. KRESS: Okay. Can we hear from Jason now on
the source term?
MR. SCHAPEROW: I am Jason Schaperow, from the
Office of Research. I have put together a short
presentation on our consequence assessment, including some
work done after the draft report was issued in response to
ACRS comments.
The object of our consequence evaluation was to
assess the effect of one year of decay on off-site
consequences. We also assess the effect of early versus
late evacuation.
DR. KRESS: Why did you choose one year, Jason?
MR. SCHAPEROW: One year was chosen as the point
where NRR was going to start considering reductions in
emergency planning.
DR. KRESS: You had an idea that that might be
about the timeframe for the risk as reduced to a level you
could live with.
MR. RUBIN: If I could add, Dr. Kress. This is
Mark Rubin, again. Yes. Obviously, you have a decreasing
heat load. The key was increasing time for operator
response, because so many of these sequences were operator
response driven, at least at the very beginning. So one
year was a convenient time when we thought we'd probably
have enough time to get pretty robust remedial actions
taken, and that was where the calc was done.
MR. SCHAPEROW: Our evaluation was an extension of
a previous evaluation performed by Brookhaven on operating
reactor spent fuel pool accidents performed in support of
resolution of Generic Safety Issue 82, and the Brookhaven
work was done for 30 days after shutdown.
DR. KRESS: My comment, Jason, that this was an
operating reactor source term had to do with the release
fractions.
MR. SCHAPEROW: Yes, I'll get to that.
DR. KRESS: Not to decay or the inventory. But am
I correct that you use release fractions that would have
been appropriate for operating reactors?
MR. SCHAPEROW: As I will discuss here in the
second and third slides, we used the operating reactor type
source term and then we took a look at it in light of the
possibility of air ingression. For example, for a seismic
event, where the bottom falls off the pool, the air can come
up inside and --
DR. KRESS: But that air ingression wasn't part of
the original document.
MR. SCHAPEROW: That's correct. You read it
correctly. There's no discussion of air ingression or
ruthenium in there.
To perform our evaluation, we performed in-house
MAACS calculations with fission product inventories for 30
days and one year after final shutdown.
DR. POWERS: When you used the MAACS code, did you
-- how did you model the plumes coming out?
MR. SCHAPEROW: We modeled it as an early release
from a reactor accident. We used the Surry modeling.
DR. POWERS: Do you use the default parameters for
plumes in MAACS?
MR. SCHAPEROW: Yes, I believe so.
DR. POWERS: The NRC, in cooperation with the
Europeans, has done quite a little uncertainty study on
those plumes and when the default parameters appeared
against the expert elicitations, there's quite a discrepancy
between the two, isn't there?
MR. SCHAPEROW: Is that a draft study? I think I
looked at that recently.
DR. POWERS: It's fully published.
MR. SCHAPEROW: Okay. I guess I'm not aware of
the study then.
DR. POWERS: I think before you use the MAACS
code, you need to look at this expert elicitation. It's
quite a fine expert elicitation, quite a fine piece of work.
One of the things that I find most striking about
it is how it describes -- how the experts describe plumes as
opposed to how it's done by default parameters within the
MAACS code.
In particular, the experts said, gee, these plumes
spread more rapidly than described by the default
parameters. Now, they had quite a range on their -- it's a
b parameter, I think, that they used to describe the spread
of the plumes.
But I think in all cases, the spreading was a good
deal higher than what you would ordinarily get if you just
sat down and ran the MAACS code. If that's the case, first
of all, there's a substantial uncertainty in your results.
Second of all, you may over-estimate prompt fatalities and
under-estimate the number of latents and the amount of land
contamination associated with these plumes.
MR. SCHAPEROW: As I said, I am not familiar with
that, but we can go back and take a look at that.
DR. KRESS: Did your default parameters include an
energy of the plume release so that you can get the extended
elevation?
MR. SCHAPEROW: That's correct. That's correct.
We used the same as an early release in the Surry deck.
DR. KRESS: So it would be release from a
containment.
MR. SCHAPEROW: We used the large early release
for Surry for NUREG-1150.
DR. KRESS: It had the same energy.
MR. SCHAPEROW: Correct.
DR. KRESS: As opposed to a fire release.
DR. POWERS: I don't think that helps me. I think
it makes it even worse.
DR. KRESS: I think it makes it worse, because a
fire driven release would go higher. It has more energy in
it and it would spread out more and do more of the land
contamination and the latent fatalities that you talked
about. It might help the early fatalities.
DR. POWERS: I think what is most at risk here is estimates
of the prompts and as you know, you and I disagree on the
role of the prompts versus the latents.
DR. KRESS: Maybe not in this case.
DR. POWERS: But maybe not in this case, yes.
Maybe we've found grounds for agreement here. But I think
the issue really boils down to we're doing calculations in
an uncertain area and we're doing them as point values and
maybe sensitivity studies, and then we're being asked to
compare the results against criteria that were developed
that explicitly call out the mean, and I just can't find the
means. I just don't know what they are.
MR. RUBIN: If I could share a perspective with
the committee. Your point certainly we will follow up on.
As far as the actual consequence calculations, results of
the analysis, mean values or even point estimates weren't
used in our actual decision-making process. The results
were used to give us a perspective on the general
characteristics of a spent fuel pool fire consequences.
We wanted to get an appreciation --
DR. POWERS: If I can interrupt you.
MR. RUBIN: Yes, sir.
DR. POWERS: And say you're not helping. When you
tell me that I didn't do the uncertainty study, and so I
didn't have that information to help me make my decision,
I'm not sure that helps.
DR. KRESS: What he's also telling me is that
these consequence analyses were interesting, but weren't
used anywhere in the assessment or the results of the study,
because what they did is just said, all right, we'll just
back off to the one-times-ten-to-the-minus-five LERF and not
use these anyway.
DR. POWERS: Your point is that maybe this is a
higher risk than --
DR. KRESS: Maybe this risk is higher than the
1.1-times-ten-to-the-minus-five or maybe the LERF, the
surrogate LERF ought to be higher based on the fact that --
MR. KELLY: What we looked at in doing this is
that we did the risk -- we asked Research to do the
consequence analysis because we wanted to find out and
confirm what we expected, that the consequences were going
to be very, very significant, and it would be a type of
thing that we wanted to make sure that the frequency of it
occurring would be relatively low and we were not worried
about getting an exact number.
We were interested in confirming that the
consequences were significant or perhaps being surprised and
finding out they weren't significant. But the bottom line
is when we came out, they were very significant, done
several different ways, they keep turning up to be very
high.
DR. KRESS: But I think the point is that they may
be more significant than the reactor accident.
MR. KELLY: The potential is there, particularly
when you have multiple cores available.
DR. KRESS: That's what is concerning us, yes.
DR. POWERS: I guess there is this intuition that
says, gee, you've decayed out for a long time, how much can
you possibly release, and what should come back is maybe a
lot.
MR. SCHAPEROW: Our evaluation, which was
described in the report, which did not include large
ruthenium releases, showed the short-term consequences, the
early fatalities were reduced by a factor of two in going
from 30 days to one year.
It also showed that early evacuation reduces early
fatalities by up to a factor of 100. Long-term
consequences, the cancer fatalities and the societal dose,
were less affected by the additional decay and early
evacuation.
Here is a summary of our calculated results. We
did do a lot of MAACS calculations. We performed over three
dozen MAACS calculations, which included variations in the
radionuclide inventory, evacuation start time and population
density. The results shown here are for the Surry
population density.
Again, our results show about a factor of two
reduction in early fatalities and going from 30 days to one
year. Cesium, with it's long half-life of 30 years, is
responsible for limiting the reduction. For populations
within 100 miles of the site, 97 percent of the societal
dose was from cesium.
Our results also show a large reduction in early
fatalities when evacuation takes place before the release.
MR. KELLY: I just wanted to comment that these
results that are up there are the ones that represent those
that were reported in our draft report.
MR. SCHAPEROW: Now, I'd like to discuss the
effect of ruthenium, which is an important issue which was
identified by the committee. We spoke with Canadian
research staff responsible for experimentally determining or
investigating air ingression to better understand their
experimental research.
We also performed MAACS calculations using a 100
percent ruthenium release.
DR. KRESS: From how many spent fuel cores?
MR. SCHAPEROW: About three-and-a-half cores.
Basically, the whole pool. Assuming the whole pool heated
up to high temperatures under air conditions and released
all of its ruthenium.
And our calculations show that release of all
ruthenium increases early fatalities by up to a factor of
100 because of the assumed form ruthenium oxide has a large
dose per curie inhaled, due to its long clearance time from
the lung to the blood.
DR. KRESS: Which would say that the LERF value
might ought to be increased by a factor of 100. Instead of
one-times-ten-to-the-minus-five, it ought to be --
MR. SCHAPEROW: Ruthenium does have a very large
impact.
DR. KRESS: The other way around.
MR. SCHAPEROW: And we did identify a couple of
mitigating factors. While the effect of ruthenium is, as
you say, potentially quite large, as our calculations show,
there are a couple of mitigating factors.
Rubbling of the fuel may limit air ingression and
reduce the release fraction.
DR. KRESS: Maybe.
DR. POWERS: I can't resist but jump in and just
point out that when we looked at the substantial ruthenium
releases that occurred from Chernobyl, that by and large,
the particles, we believe, were released in the oxide form,
but they promptly converted to the metallic form at some
point in their transport, because we only collect them after
they've gone quite a little ways.
And I don't know why they were reduced to the
metallic form, but I do know it does appear that most of
them are back into the metallic form. That will change the
clearance.
DR. KRESS: That will change the consequence, yes.
DR. POWERS: Their capability and their dose
effectiveness. It may change them only in the organ that's
affected rather than the --
DR. KRESS: It would have some effect on the
analysis.
DR. POWERS: It will have some effect on the
analysis, nor do I have any confidence that what was
observed at Chernobyl would be observed at any other
accident.
DR. KRESS: The other thing is ruthenium, a
one-year half-life, will be a significant effect on latent
cancers and land contamination, also.
MR. SCHAPEROW: That's correct.
DR. KRESS: You don't mention it here, but --
MR. SCHAPEROW: That's correct, and that's --
DR. KRESS: -- it would increase those
considerably.
MR. SCHAPEROW: The next slide indicates that.
DR. KRESS: I'm sorry.
MR. SCHAPEROW: Also, with regard to the
mitigating effect of the one-year half-life of ruthenium is
the fuel is going to be in spent fuel pools for at least
five years. That's what the casks are -- the dry casks are
not now approved for less than five years.
So with a one-year half-life, this is going to --
you're going to see a big fall-off in the early fatalities
as you get further out in time.
Finally, I would like to note that a PHEBUS test
is planned to examine the effect of air ingression on a
larger scale in an integral facility. We believe that this
test is a good opportunity to get integral data on ruthenium
releases.
DR. POWERS: Since you mentioned ruthenium, my
recollection of the work done at Oak Ridge back in the '60s
as they saw enhanced release of other radionuclides. Did
you look at anything else?
MR. SCHAPEROW: No, we did not. I think generally
you will find that those nuclides have very short
half-lives, is my suspicion. If there are any that you're
aware of, I will certainly be happy to look into them.
DR. POWERS: What I suspect is that everything
gets dominated by the ruthenium and it really didn't matter
what else you release, just because of its peculiarity.
Ruthenium is a most intriguing radionuclide from a
radiological point of view.
MR. SCHAPEROW: My last side provides a summary of
our calculated dose for the ruthenium issue. The first two
cases are for late evacuation. The first case only uses a
small ruthenium release, and that's our -- we had a very
small number of early fatalities, that's one.
The second case uses a 100 percent ruthenium
release, which results in a very large increase in early
fatalities. In the third case, which is for early
evacuation, the prompt fatalities drop back down.
As a result of our calculations, we conclude that
the effect of ruthenium can be very significant, but can be
offset by early evacuation.
Those are my prepared remarks to address the ACRS
issue.
DR. KRESS: Well, if you only focused on prompt
fatalities, but evacuation is not going to help the land
contamination. It's not going to help the latent fatalities
too much.
MR. SCHAPEROW: That's correct. Those numbers do
go up some, as you can see.
DR. KRESS: So I'm not sure which is the
controlling regulatory objective here, right? I don't know
whether without the -- we think whether prompt fatalities is
what we are -- just as Davis says, maybe we should look at a
risk acceptance criteria on land contamination or on total
deaths or latent cancers and see if you get a different
worth.
MR. SCHAPEROW: That's true.
DR. KRESS: It wouldn't be the same LERF and it
wouldn't even be called a LERF. It would be called
something else.
MR. SCHAPEROW: The societal dose and the cancer
fatalities are affected by the relocation criteria, the
relocation criteria of 25 rem for distances beyond ten
miles. So we can do something to reduce some of those
consequence numbers.
DR. APOSTOLAKIS: Why do you have to assume that
100 percent of the ruthenium is released?
DR. KRESS: You can't get more than that.
DR. POWERS: Conservative, George.
DR. APOSTOLAKIS: When you say mean consequences
not to do things like that.
DR. POWERS: Unfortunately, George, that may well
be the mean. The Canadian studies that Mr. Schaperow
referred to sees astronomical releases, fractions, that get
up to 100 percent when fuel is exposed to air at relatively
modest temperatures, around, what, 1,800 degrees -- I don't
even remember -- 1,800 degrees Centigrade, I think they were
getting 100 percent releases.
MR. SCHAPEROW: That sounds like the right
ballpark to me.
DR. POWERS: And that some work done back in the
'60s at Oak Ridge where they put chunks of fuel into the
air, they were seeing like 100 percent releases at about
1,100 degrees Centigrade.
Now, I would hasten to add, as Mr. Schaperow has,
that if we put this stuff in a big pile, it's going to be
different than these small tests that they've done and maybe
don't get to 100 percent release, but I'm not sure there's a
big difference between 80 and 90.
DR. KRESS: Now, we're pointing to the ruthenium
possibly changing our perspective on what the 1.174 risk
acceptance criteria. I would like to repeat something I
said earlier, to be sure it got in. Those acceptance
criteria were for reactors, of which there are about 100,
that are going to operate for 40 to 60 years or at least
operate the remaining lifetime.
Here we're talking about a small number of
decommissioned plants that are only at risk for a short
time, like five years or so, and that thinking ought to
enter into your acceptance criteria. This will bring it
down the other way.
It's going to allow -- I mean, it's going to allow
you to have a higher risk acceptance criteria than the
one-times-ten-to-the-minus-five. So that is a mediating
concept that you should have in your thinking about this,
even though we're saying that the ruthenium puts into doubt
the value in the other direction, these other things put it
into doubt in the other direction.
So this might offset things. I think what we need
to do is re-think what is our acceptance criteria and get
the right factors in there, rather than just pick the value
out of 1.174.
DR. APOSTOLAKIS: Is it really reasonable to ask
--
DR. KRESS: If you're going to have a
risk-informed regulation, which is what we're talking about,
we're talking about making a rule on decommissioning and
making it risk-informed.
If you're going to do that, the first place you
have to start, according to one of the ACRS letters, is that
you have to have risk acceptance criteria.
DR. APOSTOLAKIS: Yes.
DR. KRESS: And these may be prompt fatalities,
they may be latent fatalities, they may be land
contamination, they may be lots of things. You ought to
decide what they are before you even start a risk-informed
regulation.
DR. APOSTOLAKIS: My question is really whether
this particular project should think about it or the other
one where --
DR. KRESS: I think this would be a great place
because it's the one place that looks like land
contamination or latent fatalities may be the controller and
it may be a place where you don't want to think prompt
fatalities.
DR. APOSTOLAKIS: But there is another project
that is thinking about revising.
DR. KRESS: I don't know. I don't like to think
of NRC being fractured into different parts. They ought to
get their act together. But my point is that risk
acceptance criteria ought to involve other things, like how
many things are at risk, how many things are we talking
about, and how long are they at risk, and those don't seem
to be showing up here properly and both of those show up --
all of those show up in this particular place.
It's a good place to re-think how we do our risk
acceptance criteria.
DR. APOSTOLAKIS: I agree.
MR. KELLY: Some time ago, we were talking about
changes, the technical results we had gotten, and we talked
about human errors, reduction in human errors. The other
significant reduction from the June report to our current
report was the heavy load sequences, reduced to about
two-times-ten-to-the-minus-seven per year.
This frequency reduction was primarily based on
use of better statistics on our part. We didn't get
significantly more information, but when we went to a
statistician to help us with our uncertainties, we came up
with, we believe, better numbers.
We have already talked about the seismic failures
and the overall frequency was reduced by about an order of
magnitude between the June report and our current report
today.
DR. APOSTOLAKIS: This first bullet is a little
bit of a mystery. First of all, Dr. Kennedy told us that
they are not bounded, but there are eight sites. So why use
the word bounded?
MR. KELLY: In the report, it is bounded for all
but those eight sites and we indicate in the report that
there are eight sites where this doesn't apply, plus the
western sites, and in those cases, we have slightly
different criteria.
Again, we're -- I'm trying to give you a high
overview and there's little asterisks associated with a lot
of these things.
DR. APOSTOLAKIS: But then, again, I ask you
what's so magical about the ten-to-the-minus-six, and you
said nothing. So you could have said here seismic failure
frequency bounded by four-ten-to-the-minus-six, but eight
sites are greater.
I'm trying to understand what is the purpose of
this bullet.
DR. KRESS: It's a result of the calculation,
George.
DR. APOSTOLAKIS: But I could have picked another
number and said seismic failure frequency bounded by
ten-to-the-minus-six, but nine sites are above it.
MR. RUBIN: This is Mark Rubin, again. I should
let someone who knows something about this, rather than I,
speak out, but it wasn't back calculated from
3E-to-the-minus-six and these are the plants that meet it.
Rather, there were -- originally, there were some simplified
seismic techniques that were used, two times, three times
SSE for some sites, that were felt to be appropriate for
initial evaluation and then this was refined by Dr. Kennedy.
When that was done, a calculation came out of
3E-to-the-minus-six for those assumptions. Well, those
assumptions didn't apply to these eight sites. So it went
the other direction and indeed another number could be
chosen and then different sites could be defined as meeting
it or not meeting it. But it fell out of some early
assumptions that went into the calculation.
DR. APOSTOLAKIS: So there is then some
significance to the three-ten-to-the-minus-six. It
represents that set of assumptions.
MR. RUBIN: I believe that's correct.
DR. APOSTOLAKIS: With that, I can see what that
means. But the earlier statement was there is nothing
magical about it.
MR. RUBIN: There most certainly is not magical.
DR. APOSTOLAKIS: You can pick a number and say
yes, but ten of them are above it.
MR. KELLY: It's based on a .5 g HCLPF value.
DR. APOSTOLAKIS: Okay. But not fully quantified
to the seismic checklist approach. What does that mean?
MR. KELLY: We did not do a plant-specific
evaluation for any of the plants. We used hazard curves for
the particular sites, but we don't have fragilities for
everybody's spent fuel pool. We used some generic
fragilities there and that's, in part, why, when Dr. Kennedy
was talking about we had the checklist which would provide
assurance that you're at least as good as .5 g.
MR. RUBIN: Mr. Kelly, I think Dr. Kennedy wanted
to add something to this.
DR. KENNEDY: Bob Kennedy, again. To the best of
my knowledge, there are only two plants where seismic
fragility estimates have actually been made. What was done
for all of these plants was that NEI committed that they
would do a seismic review of the spent fuel pools in
accordance with the checklist and this checklist was aimed
at providing high confidence of a low probability of failure
capacity of .5 g.
So all of these annual frequency estimates,
seismic-induced annual frequency of failure estimates, are
based on the plant having a HCLPF capacity of 0.5 g.
Now, certainly plants -- certain plants are likely
to have HCLPF capacity significantly greater than that.
There's, in my judgment, the likelihood that some of the
plants won't pass the checklist and will have HCLPF
capacities less than 0.5 g, but those will be identified by
having gone through the checklist.
So if the plant passes the checklist, has a HCLPF capacity
in excess of 0.5 g, the probabilities or annual frequencies
that I computed could be conservative because they assume
that is the HCLPF capacity of the plant, and a detailed
study of the plant might justify higher capacities.
The three-times-ten-to-the-minus-six number for
Livermore was selected because a large number of plants come
up fairly close to that number and there is no real gap in
the data below that number.
As you get to the
three-times-ten-to-the-minus-six, there are still eight
sites above that number, but there's a significant, starting
to become a significant gap from site to site. So the
three-times-ten-to-the-minus-six covers the vast majority of
the central and east coast sites, with a HCLPF capacity of
.5 g, and then the next site up goes all the way to four,
and they start bouncing up.
It was an arbitrary choice as a spot where there
seemed to be a spot that you could draw a line.
DR. APOSTOLAKIS: Okay. Fine. Thank you.
MR. KELLY: Okay. We had a number of stakeholder
comments that we addressed in the draft report.
Unfortunately, we missed some. Mr. Lochbaum, in particular,
brought up worker safety and we did not directly address
that in the report, but we will be addressing that directly
in the final report.
There was also a comment that we received on
partial drain-down leading to zirc fires potentially greater
than five years after plant shutdown. We are looking at
that now and we will be addressing that in the final report.
DR. WALLIS: Can I ask about this bottom bullet
here? You said the criticality issue was addressed.
MR. KELLY: Yes.
DR. WALLIS: And I read what Tony Elsies wrote and
it looks as if the low density GE racks, if squashed
suitably, could go critical.
MR. KELLY: That's correct.
DR. WALLIS: And there's a sort of suggestion that
this could be avoided if you scatter the most reactive
assemblies throughout the pool and so on. There is no
conclusion that certain things should or should not be done,
simply as a tentative statement that this might happen and
certain things could be done to make the consequences less
severe. That's all.
MR. KELLY: I believe it also discussed the
frequency of the occurrence, which was --
DR. WALLIS: What's the conclusion? Does this
require any action or you're just going to leave it up to
the licensees to decide if they want to scatter their
reactor assemblies throughout the pool or what?
MR. RUBIN: As you point out, there is no direct
requirement or recommendation for action, namely, because,
as Glenn was pointing out, it didn't seem to be a
significant risk driver, significant risk contributor.
We had thought that in a qualitative sense at the
beginning of the study, for the draft study, but we didn't
have at least a halfway complete thought process to consider
the various scenarios, the possible vulnerabilities, what
assemblies might be vulnerable to criticality, and, in fact,
what initiators or forcing functions could cause the event.
So we tried to go through a logical thought
process to develop these, looking at the frequency of the
event that could cause a criticality and the most likely
criticality result. It didn't seem to be a concern and
that's what we're pointing out in the report, and we wanted
the thought process to be visible for these kinds of
dialogues.
DR. WALLIS: Well, I think you have to say that
somewhere, that here is a technical analysis and it leads up
to something that looks interesting, and then it goes away
and it's gone away on the basis of risk, presumably, the way
so many of these technical things do.
I mean, someone gets interested in something,
works it out, and then, gee whiz, it doesn't matter anyway.
I think you need to point that out, why you did the work and
it didn't matter or something. It needs to be concluded and
rounded off in some way.
MR. RUBIN: We'll look at that. I thought there
was a punch line at the end, but we may have lost it in the
trees of the forest. So thank you.
MR. KELLY: Based on our analysis to date and the
sequences we evaluated, we found that the zirconium fires
would generally not be possible after five years after
shutdown.
DR. POWERS: When you conclude that a zirconium
fire is not possible --
MR. KELLY: Pardon?
DR. POWERS: When you conclude that a zirconium
fire is not possible, I think that means that -- I'm not
sure what that means. It means that the oxidation of
zirconium is not very rapid after five years.
MR. KELLY: What that means is that we believe
that if you were to remove cooling from the fuel in the
state that it exists five years after shutdown, that the
fuel would not heat up to the point that the cladding would
begin a rapid oxidation at about 800 degrees Centigrade,
eventually leading to a zirconium fire.
We believe that the heat transfer mechanisms and
the reduced decay heat would be adequate at that point to
prevent the fuel from heating up to that 800 degrees
Centigrade.
DR. WALLIS: Are you going to give a presentation
on this or is this your -- all you intend to say about the
fire issue?
MR. RUBIN: I believe this was the level that Mr.
Kelly's presentation was going to go into, but, of course,
we have the thermal hydraulic analyst, Joe Staudenmeier, who
completed this study, who is available for more details.
Should we call him up?
DR. WALLIS: Well, I read Appendix A a couple of
times and there are lots of uncertainties in these analyses
which are pointed out, about well mixed atmospheres and the
natural circulation and what are the resistances to the
various pods, through the matrices and the spacers and
everything.
And it seems as if your conclusion is pretty iffy
and then you get to the end of the appendix and it looks as
if the licensee is required to do an analysis of all these
things, and the requirements are that it be a pretty good
analysis. It looks as if it's got to be better than
anything which was reviewed by the NRC.
So I have a question about how do you expect the
licensee to do an analysis which is better than anyone has
done before.
MR. KELLY: That's one of the areas that --
DR. WALLIS: And if he does or she does or they
do, is the NRC competent to tell if it's a good analysis or
not, not having any sort of knowledge any better than
anybody else?
MR. KELLY: I believe this is part of the -- our
-- if you go up to the main body of the report and you look
at the areas that we feel that things that a licensee has to
do to qualify for an exemption or things that we're
proposing would be done for the technical justification for
rulemaking, you're not going to find that they have to
perform a calculation of their heat-up.
We did say that if someone wanted to come in and
request for less than the five years, that there were
recommendations in there on what needed to be done.
It may be that we need to go back and look at what
we've asked for there in Appendix A to determine whether or
not we're asking for more than can really be provided.
But I think that Joe Staudenmeier would be the
best person to answer those questions, because he's the one
who is intimately involved with the appendix.
MR. RUBIN: We can call him up and I think --
DR. WALLIS: Can I ask you why you're there and he
isn't? I tend to look at these technical analyses and then
spend less time on the PRA. We spend a lot of time here on
the risk. And there are uncertainties in the technical
analysis which are pointed out, but they involve factors of
two, four or something like that, here and there.
The significance of it all goes away because of
some risk analysis which has uncertainties which are maybe
factors of ten or 100 in it. I just get a little queasy
feeling about making the technical understanding on the
importance based on risk which seems to have a lot of
uncertainty in it.
I don't know quite what I should conclude, but I
do have a queasy feeling about that approach to the problem.
DR. KRESS: Might I ask where this piece of
information would be used in the rulemaking anyway?
MR. RUBIN: I could address that very briefly. I
don't know if we have any of our rulemaking people here
today -- yes, we do. But I'll shoot my mouth off first and
let someone more knowledgeable correct me.
You're very correct on that there are a certain
amount of uncertainties in the thermal hydraulic
calculations. We were hoping to identify a time that would
be a high confidence, lower threshold where we would not be
concerned with the rapid oxidation occurring.
This is our high confidence number right now. In
the rulemaking, this would identify a point beyond the, say,
one year to five year point where there would be a benefit
from keeping some evacuation capability, maybe a less
formal, somewhat ad hoc, but still some evacuation planning
available to get the reduction of early fatalities that Mr.
Schaperow indicated were possible.
Beyond the five years -- let me correct that.
Beyond the point where you can get the zirconium fire, you
really don't have a significant public risk driver, and so
there may be a step change at that point, wherever it be,
where evacuation, even insurance, you're not buying yourself
anything from keeping those programs in place because you
don't have the potential for risk.
Now, I'll allow the real expert to contribute
anything.
DR. WALLIS: Well, maybe we should see the real
expert, and I'd like to know about this five years, because
there are some numbers here about six kilowatts per MTU and
then this is someone's prediction, but there was something
wrong with the model; therefore, it could actually be as low
as three. Now, that just seems to me that someone guesses
it may be off by a factor of two, but there isn't any
technical justification for whether the factor is two or
four or 25.
And if you start getting factors of four or five
on this power, then it doesn't decay that rapidly after five
years anyway. So it stretches out to maybe ten years.
MR. RUBIN: Well, there are a lot of plants, a lot
of fuel assembly designs, a lot of burnup levels, but I will
let Mr. Staudenmeier contribute his views on that.
DR. WALLIS: It would be nice to hear from him.
MR. RUBIN: Joe?
DR. POWERS: In the course of discussing what
impact the hydride, zirconium hydrides in the high burnup
fuels will have on the ignition probability.
DR. KRESS: Particularly the 800 degree choice.
DR. POWERS: And I would also appreciate knowing
whether in analyzing the heat source, that we also took into
account that the nitrogen itself will react with the clad
exothermically. It's not just the oxygen.
And indeed, some of the recent work in this field
has shown that not only the nitrogen does react, but it also
reacts to create a duplex corrosion structure and
differences in crystal structure between the dioxide and the
mononitride are such that it makes the oxide susceptible to
breakaway exfoliation and consequently deviations from
parabolic reaction kinetics and substantial accelerations in
the rate of reaction.
MR. STAUDENMEIER: Joe Staudenmeier, Reactor
Systems Branch. The five year number was based on
calculations where we were assuming a normal building
ventilation rate and fixed geometry of the fuel in the
calculations. The reason that the number, the decay heat
number went down from six kilowatts per metric ton to three
kilowatts per metric ton, went from changing the assumption
from this perfect ventilation, which is pretty much that the
building doesn't exist at all and it can get an infinite
amount of air from outside, to restricting the ventilation
that can get into the building and provide cooling for the
building.
Obviously, every possible scenario wasn't
calculated because of the fixed geometry assumption.
There's situations you could get into with a cask drop of
compression of the fuel or changing the geometry totally
that we wouldn't know even what the geometry was. I guess
that can cause heat-up rates to get up higher, but it can
also restrict air flow that could fuel the runaway reaction
also.
DR. WALLIS: This six or three kilowatts per
metric ton is some average for the bundle.
MR. STAUDENMEIER: That's right.
DR. WALLIS: There are not spots in the bundle and
you're not worried about an ignition in the local area in
the bundle.
MR. STAUDENMEIER: There is a power profile that's
input into the calculations. It doesn't assume average
power all throughout the calculations. Some things that
still haven't been evaluated yet are sub-channel analysis
for BWR assemblies, which have a very high peaking factor of
between peaked averaged rods. That can get up to like 1.4
peak rod compared to an average rod. PWRs are down in the
range of 1.1. So it's not going to be a big driver there.
And also where the pin is within the assembly can
alter that. The hottest pins are going to be in the middle
of the assembly because they can radiate out to the channels
at the edge. That hasn't been totally evaluated yet. The
previous calculations were just assumed using an average rod
and with a normal type of power shape.
DR. WALLIS: Some of your folks did a fluent
analysis where you actually looked at more realistic natural
circulation within a closed building and so on.
MR. STAUDENMEIER: Yes.
DR. WALLIS: Yes. There's a fluent analysis and
then there's another analysis done by PNNL.
They made things look a little more iffy, I think.
I get the impression that they said things could be worse
than with the other assumptions before.
MR. STAUDENMEIER: Originally, it was looking like
that, but with some more recent calculations, it's been
looking better with the fluent analyses.
DR. WALLIS: Are the recent calculations made to
look better or are they somehow more realistic on a
different criterion?
MR. STAUDENMEIER: I guess I'd have to see which
calculations you were talking about, if they are the most
recent.
DR. WALLIS: But I don't know. The difficulty is
I'm looking through a glass very darkly here, because I get
references to things, but there's no details of what's in
there and I don't want to see a huge stack of paper, but I
just want to get confidence that it was done right.
MR. STAUDENMEIER: These fluent calculations are
very sensitive to input assumptions and things like that.
Actually, we've just -- Chris Boyd is just finishing up some
sensitivity studies on input assumptions and they're
sensitive, but not overly. I mean, it goes in the way you
might think it does. It's not real sensitive to heat
transfer coefficients, because there isn't a big film
temperature drop between the cladding and the air.
It's sensitive to flow resistance. It's not very
sensitive at all for heat transfer between the building and
the outside because it's very low heat transfer to begin
with for that. There's other things that we're looking at
with some other calculations. Pacific Northwest National
Lab is doing some sub-channel calculations for us. They
have a little different approach. They don't solve the
whole problem all together. They have a CFD code that looks
at the zone from the top of the bundles up, assuming an
average power source across there and then doing some
detailed sub-channel calculations within the bundles.
DR. WALLIS: There's still a lot of work in
progress.
MR. STAUDENMEIER: There's still some work going
on, in progress, yes. And also looking at the partial
drain-down at PNNL with the sub-channel analysis.
DR. WALLIS: Could I ask whether this is going to
make any difference? There's work in progress, but PRA says
it's okay. So is it going to make any difference?
MR. STAUDENMEIER: I think the work in progress is
showing that the time will probably be less than five years
if we went and -- but you could come up with assumptions
that -- I mean, if you made the assumption that instead of
the rated building flow through the building, that there was
no building flow at all, then you could obviously come up
with a different number for how many years it's going to
take, because then your only heat removal from the building
is through convection and radiation from the wall out to the
atmosphere.
DR. WALLIS: So it's unlikely that someone would
leave the building closed for that long if something was
going on.
MR. STAUDENMEIER: Right. Yes. That would also
bring in a thing that you have very little air to fuel a
reaction then, too. It becomes fuel limited.
DR. WALLIS: Maybe the best thing is to blanket
the whole thing.
MR. STAUDENMEIER: Yes. If you could seal off all
air, that would be the best thing. It would get hot and not
do anything, except just get hot.
DR. SHACK: When you make the assumption of the
three kilowatt decay heat, what temperatures are you
reaching in the clad?
MR. STAUDENMEIER: Well, that's an input, that
three kilowatts. We look at different times and the power
goes down as --
DR. SHACK: When you use that input, then how hot
does the clad get?
MR. STAUDENMEIER: Well, our acceptance
temperature was below 800 degrees or below runaway reaction.
DR. SHACK: You did go all the way to the 800.
MR. STAUDENMEIER: Actually, the most recent
calculations we've been doing at that heat load level have
been turning out quite a bit below the 800 degrees, like
down in the 600 range.
DR. POWERS: If I think about it, air flowing over
a surface and reacting exothermically with it, and I keep
the nitrogen in, it seems to me that I have temperatures
that are pretty hot.
MR. STAUDENMEIER: The nitrogen, I think the heat
of reaction from the nitrogen is about a quarter or what the
oxygen reaction is and the reaction rate is, I think, down a
couple orders of magnitude at a given temperature from what
the oxygen reaction is, but there's -- in the previous
studies, there's some weird effects that go on apparently
that pure oxygen doesn't give you the highest reaction rate;
that if you mix in some nitrogen, you can get a higher
reaction rate because of something going on with, I guess,
the crystal chemistry and the metal that people really
didn't seem to understand.
It was just something that was pointed out.
DR. POWERS: At least the micrographs that I've
seen show that the -- in the duplex, you get a delamination
at the interface between the dioxide, which has a fluoride
monoclinic type structure, but based on a fluoride, and the
basically cubic zirconium nitrate, there's just no epitaxy
there.
So you get delamination very easily and that's why
you get this orange peel effect in the corrosion product,
where it's dimpled, and instead of a nice, smooth oxide, the
way you do in steam or pure oxygen.
MR. STAUDENMEIER: Yes, and as far as --
DR. POWERS: And that's exasperated, by the way,
in the cylindrical geometry.
MR. STAUDENMEIER: And as far as a parabolic
reaction rate, I agree with you that that's very iffy. The
data that I've seen in what these reaction curves are based
on only go out to an hour exposure. And there is evidence
that going out longer than that, the oxide does start to
peel off and as you know, the parabolic oxidation rate is
diffusion limited. So the thicker the oxide layer, the
lower it goes down.
If you peel that off, then you jump the reaction
rate right back up again.
DR. POWERS: Have you looked at the problem of
hydride inclusions?
MR. STAUDENMEIER: The Office of Research was
looking at -- I asked them to look at hydride. They
couldn't find anything that they thought would change things
significantly. I'm not familiar with the chemistry aspects
at all beyond knowing what parabolic reaction rate curves
mean and assumptions that go into them.
DR. POWERS: I'm not either. I know what uranium
hydrides do in air and other heavy metal hydrides.
Zirconium hydride I have no experience with. The other
heavy metal hydrides are spontaneously combustible.
DR. WALLIS: Glenn, this is interesting, but it
doesn't matter. Is that --
MR. KELLY: No. It does matter. As a matter of
fact, one of the inputs that the calculations, the heat
transfer calculations give us is the amount of time at which
you can get a zirconium fire and we've indicated that we
think, and eventually we'll come to it in the slides here,
where we believe that there are times when it's appropriate
to -- there's a technical basis for saying that you don't
need an EP, potentially you don't need indemnification, and
that clearly is independent of the risk assessment numbers.
DR. POWERS: Dr. Kress, you need to move this
discussion along.
DR. KRESS: I think we're getting near the end.
MR. KELLY: Just quickly, I wanted to take this
opportunity to indicate how the numbers that we chose for
pool performance guidance, guideline, and the numbers that
we got as bottom line risk numbers from our report, how they
compare to other risk measures.
I understand we've already had a lot of discussion
about how people aren't necessarily happy with the numbers
that -- the measures that we've chosen, but just to give you
an idea, in comparison with some others.
Again, our frequency that we came up with for
getting a zirconium fire was less than
three-times-ten-to-the-minus-six per year. Reg Guide 1.174
tried to release baseline guideline, where below that you
can -- we would allow a small increase in risk for licensing
type activities. That guideline was
one-times-ten-to-the-minus-five per year.
The LERF numbers coming out of the IPEs ranged
somewhere between two-to-the-minus-six and
two-times-ten-to-the-minus-five per year, and our pool
performance guideline that we documented in our report, we
chose ten-to-the-minus-five per year.
And I understand, based on your comments, we'll
probably be giving some more consideration to what, if
anything, we need to do with that proposed guideline.
DR. SHACK: There's not a whole lot you can do
about that three-times-ten-to-the-minus-six, though. I
mean, it's seismically dominated and unless you eliminate
earthquakes or rebuild the spent fuel pool.
MR. KELLY: We're not talking about changing that.
That number is what it is. But the question is, is that
okay, is that enough, is that low enough that I feel that I
have a technical basis on a risk-informed basis to want to
give an exemption or want to do something a little
differently with the rule.
DR. SHACK: But my point is if it isn't okay,
there's not a whole lot you can do about it from a
regulatory standpoint.
MR. KELLY: In which case, you would keep --
DR. KRESS: You could ask what it is after two
years, for example.
DR. APOSTOLAKIS: Or you can fund another study at
Livermore to change the seismic curves.
MR. KELLY: What we found is that we believe
there's three basic phases at the spent fuel pool.
Immediately after shutdown, you have a period for about the
first year where you can get large early releases off-site,
early here meaning several hours, releases due to zirconium
fire.
We expect that for the first few years, the design
basis systems, for the first year or so, that the design
basis system will be in place. The operating practices will
be retained. They'll have the operators there, most likely,
and this is a period where we'd expect there would be the
full requirements of EP indemnification and security.
The early decommissioning phase, you get large
releases, would be late, that's ten hours or later after the
pool was uncovered. Relaxation of EP requirements probably
could be justified technically.
To get these exemptions, industry would need to
meet its commitments, the seismic checklist and the staff
assumptions, and we expect that these would be factored into
the proposed rule.
DR. WALLIS: These staff assumptions are about
what industry will do?
MR. KELLY: Just as we -- in our report, we
documented the commitments that were made, we've also
documented those areas which were really clarifications that
we felt, when we went ahead and did the assessment, what was
it we needed to assume.
The frequency of the large releases was within the
Reg Guide 1.174 guidance. We do believe that consideration
for insurance relief during an intermediate phase would be
appropriate.
And the third phase is where we believe that
zirconium fires are no longer possible. We've talked about
the five years and we believe that there is a -- once you
get into that third phase, that there would be a technical
justification for the elimination of off-site EP.
DR. WALLIS: It's interesting you say no longer
possible, whereas your whole slant earlier was the
probabilistic.
MR. KELLY: Well, no.
MR. PARRY: This is a deterministic calculation, I
think.
DR. WALLIS: I doubt if you could say it's
complete and absolutely impossible.
MR. PARRY: Okay. Small.
MR. KELLY: Based on the thinking in Reg Guide
1.174 about how you go about making a good risk-informed
decision, we've looked at the baseline risk and here we feel
that the numbers that we've seen, again, come out within our
PPG guideline, which was based on Reg Guide 1.174.
The thermal inertia and the large volume of water
in the pool give us significant margins, in particular, with
respect to time for the heat-up of a zirconium fire.
Given there's large margins in the spent fuel
pools, we felt the defense-in-depth was not a major issue.
But the risk findings say that it's not an -- that having a
zirconium fire is not an insignificant issue. If you were
to have one, it would be very significant.
Therefore, given these risk analysis findings and
including the uncertainties associated with that, we believe
that there is technical justification for retaining a
baseline level of emergency preparedness, which would
include procedures to classify the accidents and to notify
the off-site authorities.
DR. WALLIS: Defense-in-depth not a major issue,
your report, I think, stated that sabotage is not included,
presumably in looking at security and sabotage,
defense-in-depth does become an issue. That if something is
-- that if some saboteur manages to do something, that
something else doesn't follow into some kind of backup and
so on.
MR. KELLY: That's correct, right. Our report
does not directly address sabotage. What we have done and
when we've completed our final report, we will be providing
information to the security people about those areas that we
feel are most important to protect. So that will be dealt
with separately from the indemnification and emergency
preparedness.
DR. APOSTOLAKIS: I wonder what the sentence
defense-in-depth is not a major issue means. That doesn't
mean that I know what it should mean.
MR. KELLY: Can I tell you what we meant, George?
DR. APOSTOLAKIS: Sure.
MR. KELLY: What we meant here is that we clearly
understand that the traditional measures of defense-in-depth
that you have in an operating reactor, where you have --
DR. APOSTOLAKIS: Multiple.
MR. KELLY: -- all these automatic systems,
containment. You don't have them here for spent fuel pools.
Whether or not it's even decommissioning spent fuel pool.
What you do have is a long time before you're going to get
yourself in trouble. You have the water that's in the pool,
you have the time that it takes for the fuel itself to heat
up.
So from the point of view, again, understanding
the context that we did this assessment, which is to
determine whether or not there was technical justification
for providing exemptions for EP and indemnification, under
those circumstances, we felt that defense-in-depth was not a
significant issue because of the time.
DR. APOSTOLAKIS: I think what you mean is that
the uncertainties are not large enough to justify a
structuralist approach application of defense-in-depth.
That's what you're saying, because you applied the
rationalist already. You told me earlier, Gareth, that it
would put level indicators, temperature indicators and all
that. Why? Because your earlier studies showed that human
error was a major contributor and it led to unacceptable
numbers.
So now you did these extra things to reduce the
number and you are happy that the uncertainties are
reasonably spread out over the thing. So this is a classic
application of -- it's rapidly becoming a classic
application over the rationalist approach to
defense-in-depth.
So what you really mean is that the structuralist
interpretation of what if I am wrong really is not a major
issue here, because you have reasonable confidence that you
are not wrong. Is that the correct interpretation?
MR. KELLY: It's true that I have reasonable
confidence I'm not wrong.
DR. APOSTOLAKIS: Five innocent words. What?
MR. KELLY: Yes.
DR. APOSTOLAKIS: Because you did a lot of things
and Dr. Kennedy talked about the checklist and criteria and
so on.
MR. KELLY: We tried to cross the T's and dot the
I's.
DR. APOSTOLAKIS: This is really -- I hate to say
that -- it's almost risk-based. I mean, everything you've
done is really based on risk numbers.
MR. KELLY: Well, not the areas for
indemnification and that, because we recognize that there
are -- our recommendations, we expect, as we go forward,
will be based on the deterministic analysis of the times for
the --
DR. APOSTOLAKIS: All these things are input to
the PRA.
MR. KELLY: That's correct, they are.
DR. APOSTOLAKIS: The only thing is that you say
the technical results provide justification for retaining a
baseline level of EP, but you have not quantified the impact
of that. We saw some sensitivity studies essentially.
MR. RUBIN: Excuse me. In essence, that was the
justification. It shows -- Mr. Schaperow's work shows that.
DR. APOSTOLAKIS: So that makes it risk-informed.
MR. RUBIN: I guess I should say yes.
DR. APOSTOLAKIS: Everybody wants to go to a
break. Dr. Kress, do you agree with what I said?
DR. KRESS: Partially.
DR. APOSTOLAKIS: Where do you disagree? That's
what I want to know. It seems to me this is almost
risk-based.
DR. KRESS: It is.
DR. APOSTOLAKIS: And the rationalist approach has
been applied.
DR. KRESS: The five years is not risk-based.
DR. WALLIS: I thought you said it very well and
it helped, I think, to steer us in the right direction in
thinking about defense-in-depth. We've been going in that
direction for some time.
DR. APOSTOLAKIS: I think what Gareth said earlier
is really application of the rationalist approach. Anyway,
I've said enough.
DR. KRESS: Where I disagree with you is where you
said that the uncertainties are small enough. I don't think
we know what the uncertainties are. We haven't quantified
them very well and --
DR. APOSTOLAKIS: Do you disagree with the
statement up there?
DR. KRESS: Yes. I would have said that the
structuralist what if we're wrong still applies here, to
some extent, and, therefore, we will have some sort of
emergency plan.
DR. APOSTOLAKIS: That's fine.
MR. RUBIN: And indeed we agree with you, Dr.
Kress.
DR. KRESS: So I think you characterized it just a
little wrong.
DR. APOSTOLAKIS: You agree with too many people,
Mark.
MR. RUBIN: I'll try to do better.
DR. APOSTOLAKIS: You agree with that statement,
too, right?
DR. KRESS: I think we would like to see your
slide on the impact on rulemaking before we close.
MR. KELLY: The slow time it takes for the
releases to occur, we believe, justifies reducing EP
requirements during that one to five year period.
The risk analysis, we believe, does not justify a
reduction in security function directly. It may be that it
may be justified based on -- a reduction might be justified
based on as much less complexity needed for a spent fuel
pool than for the entire operating reactor.
The current report does not take a position on
indemnification. We note that the frequency of a zirconium
fire is not incredible. The Commission may decide that it's
low enough that they want to get rid of indemnification. We
note that operating plants do have large releases at
frequencies similar to what we've calculated.
DR. KRESS: And the implication of that is they
are required to have indemnification even at those values.
MR. KELLY: One might take that to be the case.
DR. POWERS: What does the frequency have to be
before it's incredible?
MR. KELLY: I am the one to make that judgment,
I'm afraid.
DR. WALLIS: Well, you made the statement.
MR. KELLY: Well, we felt that it was high enough
that it was credible, so therefore --
DR. APOSTOLAKIS: The age of the earth's crust,
three-ten-to-the-ninth years. Three-ten-to-the-ninth. So
you decide what is incredible and what is not.
DR. POWERS: There has been a long tradition,
beginning in the time of the AEC, that accidents that have
probabilities of less than about ten-to-the-minus-six are
considered incredible.
DR. KRESS: Yes. I think that's the answer.
MR. KELLY: We believe the rulemaking should
include a requirement to monitor performance scenarios such
as industry commitments, staff assumptions and the seismic
checklist. And lastly, in the late decommissioning phase,
after it's no longer possible to have a zirconium fire, we
believe there isn't a technical basis for retaining
emergency preparedness.
That concludes my presentation. Are there any
questions?
DR. KRESS: I might ask if there's anybody in the
audience or public who wish to make any comments or give us
a reaction to this. Seeing none, I guess I can turn it back
to you.
DR. POWERS: Thank you, Dr. Kress. I will declare
a recess until three of the hour.
[Recess.]
DR. POWERS: Let's come back into session. I
neglected to remind members at the conclusion of the last
session that the Commission has asked us to do not just a
report on this, the work on the spent fuel pool, but a
technical review; not only a technical review of the
phenomenology, but an assessment of the risk goals of the
study, as well.
So it's one of the reasons that we plunged into
the details so much on that previous study.
We turn now to a very interesting topic, one
that's been near and dear to the heart of the ACRS for
loathe these many years, and that's digital instrumentation
and control at nuclear power plants. Professor Uhrig, I
think you're going to lead us into this and plans for
research in this area.
DR. UHRIG: Thank you, Mr. Chairman. Digital
instrumentation systems are not new to nuclear power plants.
They've been around since the late '60s, early '70s, in the
Canadian CANDU reactors. They came into use in the United
States in the late '80s and early '90s, such things as the
GE VTEC instruments, the Eagle-21 safety and control system.
The staff's original position on digital systems
was that all of these were unreviewed safety problems and
had to have Commission approval. This was subsequently
modified, but the discussions led to the Commission asking
the National Academy of Sciences to undertake an extensive
study of the use of digital instrumentation in nuclear power
plants.
During the two years of this study, the staff
resolved many of the issues associated with licensing
digital systems. They upgraded the standard review plan,
NUREG-0800, to include digital systems, and, in a sense,
this plan that we're going to hear today is the first real
formal response to the Academy study of some of the things
that -- the recommendations that they made.
So we're looking forward to hearing and at this, I would
call on Sher Bahadur to introduce the topic and the
presenters.
MR. BAHADUR: Thank you, Bob. I'm Sher Bahadur,
Branch Chief of the Engineering Research Applications Branch
in the Office of Research. Last year, about this time,
Research had gone through a major reorganization. Then the
digital I&C was separated from human factors and came into
my branch.
The purpose of this briefing is to present to you
the plan that we have developed on the research part of the
digital I&C. With me are Steve Arndt and Matt Chiramal, and
I'll go through some of the basic components of this
briefing this morning and which parts will be taken by Steve
and Matt.
As I was saying, the purpose is to bring this plan
to you for your review and what you will hear today is the
regulatory framework in which the research in I&C will be
applied and the plan, its background, challenges, issues and
planned activities, and the schedule.
I want to thank the committee, before I go any
further, for allowing us to come here and present this to
you on such a short notice, because this, in turn, will
allow us to finalize the plan before the budget cycle for
2002 gets final. So it will be very timely, from our point
of view.
I also thank Bob and Dana in particular for
allowing us to let you give the preliminary draft of this
plan earlier to get your feedback at that time in a formal
manner and what you will hear today is not much different
than what you had seen, except that from that time on to
today, this plan has been coordinated with NRR at the
division level.
As you had mentioned, we have had this plan on our
plate for a long time, and what you will hear today is our
effort in-house to develop this plan, and we have developed
this plan based on input from a number of sources.
What I have done is I have listed that for you.
For example, we had had a workshop as early as 1989, the
workshop on the man-machine interface, and the findings of
that workshop were then summarized into NUREG CR-5348.
Later we also had a workshop on the digital system
reliability that was sponsored by NIST and NRC jointly and
the findings of that were also summarized in another NUREG.
Just about the same time, the agency had started
looking deeper into the digital I&C issues and we requested
the National Research Council, Academy of Sciences
subcommittee to look into this more in detail and a report
was produced in 1997 on the instrumentation and control
systems in nuclear power plants.
In 1998, this committee has returned a report on
the research activity and there, again, underlying the
necessity for us to develop the plan. With that in mind, we
held an expert panel meeting last fall on the digital system
research and that meeting was attended by about 12 experts
all over the country, including Doug Chapin, who came from
NPR Associates; Barry Johnson, who came from University of
Virginia; and then there were representatives from SOHAR,
University of Pennsylvania, Duke and other industries.
DR. APOSTOLAKIS: Were these mainly computer
experts, computer science experts?
MR. BAHADUR: These were mostly the software and
the hardware people who were working on problems associated
with safety-related issues for other industries.
For example, Barry Johnson from University of
Virginia had been working in the railroad industry, also
with NASA and Boeing. Similarly, the Penn State people had
been working with some NASA projects.
So I would say they were mostly the
hardware/software people, but they had application
experience in other areas.
MR. ARNDT: Some of the members also had nuclear
experience. For example, Doug Chapin, who does a lot of
work in the nuclear area and was the chairman of the --
DR. APOSTOLAKIS: Was Dr. Levinson a member of the
group?
MR. ARNDT: Pardon me?
DR. APOSTOLAKIS: Dr. Levinson, was she a member?
MR. BAHADUR: Dr. Levinson came as an observer. I
don't remember -- I don't think -- she was not there, no.
Dr. Levinson could not make -- she was invited, but could
not make the meeting.
In addition to getting the information from these
sources, the staff also worked very closely with NRR to see
their needs, because this plan is outlining research which
will assist NRR to make its review function efficient,
timely and realistic. So we worked very closely with the
NRR staff and as a result of our work, while this plan was
being developed, we also received a consolidated user need
from NRR, which ties very closely with the plan which we
presented to you.
DR. POWERS: The plan that you developed, of
course, speaks frequently of the need to develop tools to
aid the line organizations in doing their review.
I wonder if a -- I know that NRR has some explicit
and quantitative guidelines for the timeframes that they
want to handle licensing actions and the amount of manpower
they want to devote to those. Did they give you some
quantitative guidance on what their aspirations were as far
as things like the amount of time that they wanted -- your
plan makes the point that these are very resource-intensive
reviews that they have to do.
Did they give you quantitative guidance on the
kinds of resources they want to devote to each review?
MR. BAHADUR: Yes. Let me -- I don't want to
steal Matt's thunder and also Steve's, but let me just give
you a very general answer to that.
What you would see in this plan is three-pronged
activities. Activities that we are proposing to meet the
immediate requirements of NRR, what I call the short-term
goals; the activities that we do to meet the requirements of
NRR in the intermediate, plus, also, what anticipated
problems could be at that time, and, lastly, something to
keep pace with the ever changing technology so that the NRC
keeps abreast of the information that the industry might
bring to us.
You would see that kind of thread coming out of
both Matt's presentation of the regulatory framework and
then Steve's presentation on the plan itself.
I think with that in mind, let me just say that
this briefing would be in two parts, as I said before. Matt
Chiramal, who is a senior level staff member from the
Division of Engineering, NRR, will present that. Then the
description of the plan itself will be presented by Steve
Arndt. Steve is in the Division of System Analysis and
Regulatory Effectiveness, but he has been on a special
assignment, rotation assignment in my branch, specifically
to work on this plan, which he has done in the last three
months.
So with that in mind, if you don't have any other
general questions, I will request Matt to cover the
regulatory framework.
DR. APOSTOLAKIS: So Steve will leave in three
months, this project?
MR. BAHADUR: Pardon?
DR. APOSTOLAKIS: Steve will leave this project in
three or four months?
MR. BAHADUR: Steve's major function was to work
on this plan and to bring it to its completion. As you and
I speak today, his rotation has been already up last week.
However, he will continue to work on the plan until it's
finalized.
DR. POWERS: Is he going to come back and work on
human performance then?
MR. BAHADUR: No comment.
MR. CHIRAMAL: My name is Matt Chiramal. I'm a
senior level advisor in NRR. Now I'm in the Electrical
Instrumentation Branch. I'm with the Division of
Engineering, Electrical Instrumentation Branch, called EEIB.
I've been here before when we developed the SRP Chapter 7
and that's what we are using for guidance in the review of
both the advanced reactor, which we haven't had since the
1997 version of the SRP was issued, and any retrofits,
including the topical reports we are reviewing at this time.
The SRP Chapter 7 is based upon IEEE 603, which is
now part of the regulations, and its companion standard,
IEEE 7432 on digital systems. That's the framework on which
the SRP was based.
We have at this point five topical reports
in-house, of which number one the Siemen's Teleperm XS and
the ABB-CE Common Q are the ones that will be completed this
year.
The others are in waiting in the wings. Some of
them are on hold. For example, the Triconex will come in
probably later this year, but the Westinghouse units are on
hold because Westinghouse has not decided what to do as yet.
We expect to see applications based on these
platforms coming in in the near future from different
licensees and different utilities.
DR. APOSTOLAKIS: What does platform mean in this
context?
MR. CHIRAMAL: Platform means it's a basic
structure, it's a hardware/software operating system
platform on which you can build specific applications into
that system. So they can use it for a reactor protection
system, engineering safety feature, actuation system, or an
information system, any digital application, as long as it
works on the -- it can be built into the platform.
DR. UHRIG: Would you consider that an integrated
system, the platform constitutes an integrated system?
MR. CHIRAMAL: Yes. It's a system on which you
can integrate other modules and applications.
DR. APOSTOLAKIS: But it does not give also an
assessment of the reliability or whatever. Is that done
off-line or it's not done at all?
MR. CHIRAMAL: The quality and the reliability of
the platform and its operating system and the way it could
interface with the thing, that part of it is done in our
review at this stage.
DR. APOSTOLAKIS: Okay.
MR. CHIRAMAL: At this point, we believe that we
are using the SRP Chapter 7 in parts, depending on what we
get, which part of the system is being modified. We
consider it adequate for the review of the systems that come
in at this time.
DR. POWERS: When you say it's adequate, do you
mean it's technically adequate or it's operationally
adequate or both?
MR. CHIRAMAL: Both.
DR. POWERS: Both. So you don't mind spending
these -- I mean, the plan speaks of resource-intensive
reviews.
MR. CHIRAMAL: Yes. We do spend a lot of time
interacting with the vendor and trying to get into the deep
-- deeply into the process as well as the applications, how
it works, and we spent quite a bit of time in that area and
one of the areas we're looking for is to try and expedite
that review process a little bit so that when we frame it at
this time, give that to the public to review, then we don't
have to ask that many questions and RAIs and things like
that.
DR. POWERS: I guess what I'm trying to understand
is you have a user need over there, but you find the
guidance is adequate. Well, it must be inadequate in some
respect and I assumed it was inadequate -- based on the
plan, I assumed it was inadequate because it takes too much
time and resources to carry out one of these reviews.
MR. CHIRAMAL: The thing is, at this point in
time, we believe that's the best we have.
DR. POWERS: I understand that.
MR. CHIRAMAL: Right.
DR. POWERS: But there must be some inadequacy to
it. Something that you'd like to be better.
MR. CHIRAMAL: Yes. It's a little convoluted and
so we'd like to sort of smooth it out a little bit by having
some sort of a tool that can do some of the reviews that we
do.
DR. POWERS: I think what you're telling me or at
least what I think I understand is what you've got now is
technically adequate. I mean, having gone through the
review, you're confident that the digital I&C system will
perform adequately, provide adequate protection to the
health and safety of the public. But it takes a lot of time
and time translates into money, time translates into delay,
and you'd like to speed the whole thing up and maybe even do
a better job, but certainly speed the whole thing up.
MR. CHIRAMAL: That's correct.
MR. BAHADUR: Let me see if I can just say it the
way I understand where we are in terms of what's adequate
right now versus why the research is needed in this area.
The industry is gradually and slowly changing from
the analog to digital. While this activity is going on, as
Matt said, the existing review process is there to take care
of this.
But there's an ever changing technology and NRC
will continue to get into areas in which perhaps we are not
ready right now to do an adequate review.
So the way I see the function of this research is
to help NRR make its review effective and realistic in the
face of the ever changing technology.
Yes, I agree that there's no overlying safety
issue at this time that we have to worry about, but we don't
know what and how the changes in the industry is going to
be. And for us to be in a position where we can provide
that realistic review, we need to understand and prepare
ourselves, and that's what you will see the plan is leading
towards.
DR. POWERS: Well, I caution you that if I'm in
the role of the Commissioner and I have finite numbers of
dollars to dole out to research and somebody tells me, well,
the chapter is adequate, then I may not read the rest of
this. I say, fine, we don't need to do anything there right
now, let me move on to these things where somebody says it's
inadequate.
Whereas I think you're telling me that there's an
inadequacy here in two things. It takes a lot of time and
effort to do things and the digital systems you see today
are going to be obsolete and unavailable next week.
MR. BAHADUR: And there is something in between,
as well, and that is that while it's taking us a longer time
to do the review, it's also possible that the review we are
conducting is not realistic and, therefore, we have to make
certain assumptions which bring an awful lot of conservatism
into it, because we don't understand right now what's
happening.
One of the issues --
DR. POWERS: So you're introducing a third point
which says it's adequate all right to protect the health and
safety of the public, but it's very conservative.
MR. BAHADUR: For example, one of the issues which
was brought out by Doug Chapin in the expert panel, from NPR
Associates, was exactly this. His point was that in all
these environmental qualification requirement, I think the
NRC needs to gain an understanding of realism, what's
actually happening out there, so that we can accordingly
modify our review criteria.
DR. APOSTOLAKIS: But I thought there were two
more reasons. One is that when SRP Chapter 7 was written,
naturally you had to rely on the state-of-the-art at the
time.
MR. CHIRAMAL: That's correct.
DR. APOSTOLAKIS: And it is too much perhaps
process oriented rather than product investigation and
analysis. And now perhaps we can do better and combine
process with product.
MR. CHIRAMAL: Actually, the emphasis on process
for digital I&C was mainly added to review the quality of
the software. The product orientation was always there in
the SRP, the testing, the --
DR. APOSTOLAKIS: Yes, but I remember our debate
at the time that one of the arguments you made was that
there aren't -- there were no really good methods, proven
methods for investigating the adequacy of the software
itself.
MR. CHIRAMAL: Right. And that's why we emphasize
process a lot in the SRP.
DR. APOSTOLAKIS: But now things may be changing,
that's my point. That's an additional reason for you to
investigate possibility of --
MR. BAHADUR: That's an excellent point, George,
and I think we're almost coming to a point where if I speak
anything more, I might as well be giving the presentation,
and Steve is rearing to go. So once Matt completes his, I
think Steve is going to be on and touching on this.
DR. APOSTOLAKIS: And the third, maybe a fourth
point. I think it's important to say up front why you have
to do --
MR. CHIRAMAL: Yes.
DR. APOSTOLAKIS: The last one is to support the
major initiative of the agency to risk-inform the
regulations.
MR. CHIRAMAL: That's correct.
DR. APOSTOLAKIS: Right now we don't know how to
handle digital software embedded in systems in the plant and
as I saw in your plan, you have some projects there that
will help us quantify the availabilities of systems perhaps
and so on.
MR. CHIRAMAL: And that's part of the user need,
is to keep up with the emerging technology, and the
technology is advancing more rapidly than the standards are
keeping up with it.
DR. APOSTOLAKIS: Do you think risk-informing the
regulation -- to support risk-informing the regulations is a
major argument why you want to do this.
MR. BAHADUR: And I think Matthew has that on this
slide, on your second bullet.
MR. CHIRAMAL: Right.
MR. BAHADUR: It outlines those things. So why
don't we move to that.
DR. WALLIS: I'm a little concerned that the ACRS
is explaining the need for this work. I agree with the
Chairman. I would like to see a much better discussion of
what's adequate, what do you mean by adequate, what could be
done better or what measure and what could be better this
year, what do we see coming down the road, what do we need
to anticipate in five years and so on.
MR. CHIRAMAL: And that's part of the keeping pace
of the technology, is that we do not know what --
DR. WALLIS: But be more specific about it.
MR. CHIRAMAL: For example, you could have a
network system put into place. Again, it depends entirely
upon what's available out there, what is going to be
applicable to nuclear plants.
DR. WALLIS: But you want to anticipate that. You
don't want to be caught off guard and you don't know what to
do with it.
MR. CHIRAMAL: And that's part of the user need
which the plan will address. As Dr. Apostolakis was saying,
we also would like to have -- address the reliability and
risk concentrations of digital I&C, which, at this point, is
not -- is purely qualitative.
And the last bullet is keep current with the
industry standards, because here, again, one of the things
we would like to do is to work with the industry and come up
with the standards that can be used in the application of
digital systems and nuclear plants.
In that sense, that's why the SRP Chapter 7
guidance is based upon the two standards that we mentioned
earlier.
And that concludes my part of the presentation.
Steve?
MR. ARNDT: Good morning. AS my colleagues have
previously noted, my name is Steven Arndt. I am currently
assigned to a different part of Research, but have been
working in this area for the past couple months to help the
various experts in this area to put together this plan.
Since my colleagues and the committee have already
done a lot of my work, this should be pretty easy for me
today. I'm going to go through very quickly the program
goals, outlines and outputs, and then get into the specific
applications of the research that we're trying to do to
support NRR.
You will have noticed in the actual paperwork that
we sent you to look at references to the user need. It was,
unfortunately, not available when we sent you the documents.
It is now available and is appended to the back of this.
DR. WALLIS: This program goal is presumably an
NRR goal.
MR. ARNDT: It's an agency goal.
DR. WALLIS: Okay. It's not the goal of your
research.
MR. ARNDT: No. It's the program as opposed to
the plan goal. It's a program of proving digital I&C
systems.
DR. APOSTOLAKIS: But you really don't care
whether unanticipated failures in digital systems will
occur, because some of these failures may do nothing to the
reactor.
MR. ARNDT: Yes.
DR. APOSTOLAKIS: So this can't be the goal of the
program.
MR. ARNDT: It's not -- it's unanticipated
failures that really affect safety of the plants.
DR. APOSTOLAKIS: The safety of the plant, yes.
MR. ARNDT: Right.
DR. APOSTOLAKIS: Again, I'm always confused by
this need to list goals, but it seems to me one of your
major goals is to support the agency's initiative to
risk-inform the regulation.
MR. ARNDT: Right. And the outputs of the plan,
the outputs of the program that we're putting forth are the
three areas that we previously discussed.
DR. APOSTOLAKIS: I think you need to reword the
program goal.
MR. ARNDT: Okay.
DR. APOSTOLAKIS: And I don't think you should
refer to the ever changing technology in the goal. In other
words, if the technology did not change, are you assured
right now that the introduction of digital systems would not
do something bad? I don't think it's the changing nature of
the technology that is really driving the program. It's an
important element.
MR. ARNDT: We'll look at that goal. The outputs
are, first of all, to improve the methods and tools needed
to support the improvements to the regulatory process.
That's what Sher and Matt have already mentioned. The
process is not as good as it could be. It's not as
efficient as it could be and a lot of the programs are
specifically designed to address user needs from NRR to make
things better.
DR. POWERS: What I don't understand, I mean, this
word improved shows up in your plan frequently and I can
improve things incrementally. I can improve things hugely.
I can improve things somewhere in between.
And without having some understanding of what kind
of improvements or what magnitude of improvement that you're
trying to make and in which ways, I mean, you could print
the SRP in bolder, bigger type face so it would be easier to
read.
MR. ARNDT: Right.
DR. POWERS: And you could print the IEEE standard
in bigger type face so it would be readable, period, and
that would improve the process. Can you give me some idea
of what magnitude of improvement? Reduce the number of man
hours by 50 percent, ten percent, one percent?
MR. ARNDT: Certainly we intend for it to be
considerably more than one percent. An example is the
topical reports that Matt was talking about are taking
something more than one man year per to review, which is
considered too long.
Once those are approved, we'll have the individual
plant-specific applications of the topical reports, which
could be a lot of them considering where the technology is
going, and if they are some significant fraction of that,
then it's also going to be too high.
So we're looking at significant improvements based
on more, in many cases, definite standards. One example
that Matt mentioned is that in some of the software QA
issues, they are more process oriented than product
oriented, a lot of them, parts of the SRP talk about review
the test coverage, review the plan to be used, determine
whether or not various methodologies are appropriate.
And many of the tasks in the plan say look at
these methodologies that people are using, compare them to
specific criteria and give us a tool that says, yes, if they
use CMM and they use a test coverage of 50 percent, then
that's reasonable or not reasonable or industry typical or
something like that.
DR. POWERS: If, at some point, in your plan, if
only to say very qualitatively what you said, you're looking
for significant improvement, that would help me understand
the work a lot better.
MR. ARNDT: Okay.
DR. POWERS: I find appealing the kinds of goals
that NRR sets out to itself, where it says it wants the
process licensing actions and so many per year, so fast,
things like that, a much more quantitative statement of
their own personal goals, that kind of comparison to that
kind of information would considerably aid you in
understanding what's being achieved.
MR. ARNDT: Okay.
DR. POWERS: And the magnitude. Because otherwise
I can read it as, well, we're going to muck around with this
for a while and we'll probably find some way to improve it,
if no more than to put it on bolder type face or something
like that.
MR. ARNDT: Right. Well, as you noted in reading
the plan, each of the activities has a background, a
specific task and a specific outcome, and I think one of the
things that we could do to address that is to be a little
bit more specific in our outcomes.
DR. POWERS: That would really help a lot.
MR. ARNDT: Okay.
DR. POWERS: So I know the magnitude of effort and
some idea on what I'm getting for the money.
MR. ARNDT: Okay.
DR. POWERS: Even if research is difficult to
quantify, but anything that helps me understand the
magnitude of the effort just makes it a lot more palatable
to understand.
MR. ARNDT: Okay.
DR. WALLIS: Do they have tools today?
MR. ARNDT: In some cases, they have qualitative
tools. In some cases, they have quantitative tools.
DR. WALLIS: Do they have computer programs or
something that they can apply?
MR. CHIRAMAL: Yes, we do have a program where we
have a list of -- a checklist and we check off whether
that's available, have we looked at this.
DR. WALLIS: You have a checklist.
MR. BAHADUR: Yes.
DR. WALLIS: It's a tool. Is that the level of a
checklist?
MR. BAHADUR: It's actually making sure that all
the requirements are traceable to the outputs of the system
and it's like --
DR. WALLIS: How do you improve a checklist? I'm
wondering if you have any tools today, maybe the thing is
that you need improved people rather than tools. Do you
know what they're doing at all or -- we're not convinced the
tools are appropriate in this business and what the tools
are today.
MR. ARNDT: We'll talk a little bit about that,
explain that at a later point in the plan. But
specifically, for example, a tool may be referenced, did
they follow this particular standard, and an approved tool
maybe how they followed it or what particular methodology
they used.
DR. WALLIS: It's a bureaucratic level, though.
It's not an intellectual tool level or engineering tool
level where you get one person to do the job of ten by
having some computerization of the checklist or something.
MR. ARNDT: In some cases, that's true. In other
cases, we're specifically looking at, for example, metrics
to say the software testing is acceptable if you have met
these particular metrics, which is an advancement and an
engineering kind of tool.
Let me go on and talk about, again, the outcomes.
We want to, by doing the things in the plant, improve the
regulatory review process, reduce the time it takes to
develop new regulatory guidance, and this really goes to
what Bob and Sher were mentioning earlier -- be ready when
these new technologies become available and to improve the
capabilities of addressing digital I&C systems and digital
systems as a whole in PRAs and other risk-informed tools.
Very important in that very immature industry.
This is just basically a chart of what we're
doing, how it's organized, the research programs, outputs
and outcomes, and the various programs we have are divided
into four categories. These are somewhat arbitrary because
the specifics of the individual tasks sometimes cross the
organizational boundaries.
We have, in particular, the systems aspects of
digital I&C, which reference back to the National Academy
study. It's an area that affects the general design and
applicability of the systems. Software quality assurance
issues, qualitative and quantitative. The digital risk
issues, the assessment of the risk of the systems, and how
that affects our regulatory review process, and the whole
issue of emerging technologies.
One of the things that we're planning is that as
the emerging technologies become actual things that people
are designing and planning on putting into plants, they will
move from this category as the plant evolves in time into
one of these categories.
So, for example, smart sensors is something that
people are just starting to look at for nuclear
applications. As those become things that Westinghouse and
Siemen's are actually building and starting to write topical
reports on and things like that, then they would move into
the more immediate type research.
DR. WALLIS: It looks to me like a generic
transparency. If I put thermal hydraulics every time I saw
I&C here, it would still be valid.
DR. BONACA: Yes.
MR. ARNDT: Yes. And as you will hear this
afternoon, the human factors plan is similar.
DR. APOSTOLAKIS: The difference is that in
thermal hydraulics, you know how to do these things and
there are no emerging technologies. How are the system
aspects of these technologies different from risk assessment
of digital technology?
MR. ARNDT: The systems -- well, let's go about it
from this way. The digital risk assessment issues have to
do specifically with the technology associated with putting
numbers to the risk of these systems, the reliability of the
systems, the data associated with that, the ability to
understand the failure modes with respect to a PRA, those
kind of aspects.
The systems aspects, although applicable to the
digital -- rather the risk, as I mentioned earlier, these
things cross boundaries, look at the more traditional
engineering type issues, like what characteristics of
software operating systems, what characteristics of
environmental qualification do we need to ensure that
particular system will not endanger the health and safety of
the public.
So basically these are more deterministic type
issues as opposed to failure mode and failure rate issues.
DR. APOSTOLAKIS: So it overlaps a lot with the
quality assurance part, too.
MR. ARNDT: Yes, it does.
DR. APOSTOLAKIS: Anyway, fine.
MR. ARNDT: This is basically just a reiteration
of the four areas. This next graphic is just a tool to talk
a little bit about the fact that the issues associated with
this have implications to other agency programs, including,
for example, the human performance plan, the risk-informed
regulation implementation plan.
For example, what is known as a front of the panel
type issue, the man-machine interface issues, the
computerized procedure issues, things like that that we're
dealing with in our human performance plan also have
implications to the digital system issues, as well as vice
versa.
For example, the human factors program that you
will hear about this afternoon has got several issues on
high burnup control systems. That also affects the kinds of
things that we will be interested in in the digital I&C
plan. So we have a close coordination with them.
Also, as the risk-informed regulation plan gets
developed, the kinds of issues, like, for example, the
digital systems are very poorly modeled, if at are, in PRAs
and whether or not that should be part of the standard, for
exactly, the research into that becomes an important issue.
So we work with Nathan and various other people in
the risk area to coordinate the program.
I would like, at this time, to go through briefly
--
DR. APOSTOLAKIS: Are you using -- what kind of
font are you using, Steve?
DR. POWERS: Steve, you don't need to answer that
question.
MR. ARNDT: Thank you, Chairman Powers.
DR. APOSTOLAKIS: This is not current technology.
MR. ARNDT: Speak to the CIO.
DR. APOSTOLAKIS: You used the typewriter perhaps?
DR. SHACK: It's digital, George. It's probably
colored it and they printed it in gray scale.
DR. APOSTOLAKIS: Increased management attention
is needed here.
DR. SHACK: It's a technical problem.
MR. ARNDT: What I would like to do now is go
through briefly the specific areas that we have highlighted
and talk a little bit about what we're doing.
The first several areas under system aspect have
to do with environmental stressors. These areas,
particularly the EMI/RFI and the environmental qualification
areas, things that have been ongoing and if you'll reference
the discussion of schedule in Chapter 5, you will see the
start dates for most of this work has been some time ago and
we're in the process of wrapping up the particular issues,
but they exist in the plan because it's a from now forward
plan.
DR. POWERS: In the plan itself, you come along
and say we're going to complete these.
MR. ARNDT: Yes.
DR. POWERS: Several times in this particular
column here, you're going to complete these. But you don't
explain to me what completion means.
MR. ARNDT: Okay.
DR. POWERS: I'm sitting there -- I'm totally at
sea, because my initial reaction, of course, was EMI/RFI,
oh, I've heard more about that than I want to hear about, I
thought it was done. So I was really ready to hatchet up my
coffee on that, and then I read, well, it's just completing
this, but I didn't know what that meant. I mean, that could
be ten years worth of activity. That could be two and a
half hours.
MR. ARNDT: In Chapter 3 and Chapter 5, we
reference what the tasks are to complete it and the
schedule. For example, the EMI/RFI will be finished in
about 18 months. And where we are currently is we've
completed the research, we've completed the experimental
program. We presented this at a public workshop in
September and we got some comments on it.
Most of those had to do with these are good
guidelines, but it seems to be overly restrictive in certain
areas. So one of the things we did when we met with the
public in that was to look at how can we find ways without
causing problems to be less restrictive and reduce the
burden on the public.
The particular issue was that the guidelines as
they exist today basically say for this kind of equipment
and this kind of environment that we have measured in the
plants, you need this level of protection, and what we're
currently looking at is segmenting that and saying, all
right, if you can show us that this particular area has a
much lower environmental stress in the EMI area, then we can
relax those regulations.
So this is something that has been specifically
requested by the industry and we're looking at that aspect
of it. We're also looking at some other aspects; for
example, signal line EMI. RFI was not completely addressed
in the original research. So we're looking at that, as
well.
DR. POWERS: This whole task on EMI, I know RFI
was very puzzling to me, because you want to complete some
work, then you want to look and see if you've been overly
conservative, and then you want to investigate the potential
of endorsing an IEC consensus standard.
Why not just go to there?
MR. BAHADUR: If the question is why don't we go
to the standards directly and endorse them, I think it would
be a little bit difficult to take any standards and just
blindly endorse them without understanding.
DR. POWERS: You wouldn't be blind. You've done
quite a lot of work in this area.
MR. BAHADUR: We have done work, but I think there
are a couple of issues which were raised in the public
meeting that we need to look into before we go back and look
at the standards and endorse them.
DR. POWERS: Why don't you look at the standard
and say, okay, the standard covers all these things, I
really don't know whether it's adequate in two out of 16
that it covers, and it does cover these other things that I
found in the course of my research, and use the standard as
the guide on what you do on finishing this work up rather
than finishing up the work.
I mean, the way it's organized now is you do the
work and then you endorse the standard. I'm wondering why
don't you do it the other way around or look at the standard
and then decide what you have to finish up.
MR. BAHADUR: I have Christina Antonescue here in
the audience and she is the project manager on this project,
and I would like to see if she can add some details.
MS. ANTONESCUE: We have used consensus standards
and the consensus standards that we used were the military
standards and IEEE standards. At this time, we are moving
towards IEC standards and that's the future. We have not
reviewed the standards in detail, so we cannot really
endorse them at this point.
That's why we need to extend the project and look
at these IEC standards.
DR. POWERS: What I'm asking is --
MS. ANTONESCUE: And they're not balloted yet.
They're not finalized standards.
DR. POWERS: What I'm asking is why do you need to
go through these task A and task B under verify EMI/RFI
qualification, then look at the standard. Why not look at
the standard now and then decide how much needs to be done
in A and B?
MS. ANTONESCUE: The over-conservative nature was
brought up by the industry at one of our meetings we had
with NPR, and the question was raised that the existing I&C
standards that we're using that time, that includes the SER.
So the reg guide that we have provided since is an
improvement and the industry has acknowledged that.
So we are revisiting this over-conservative nature
that was brought up at the NPR meeting and we're looking at
additional data that we're going to collect in the future
and just to address this specific question that was raised.
DR. POWERS: Suppose you go in and you say you've
convinced me -- I don't know who from the industry is trying
to convince you, but Joe has convinced me that indeed we
were over-conservative and you change the process and the
IEC consensus standard, when it finally comes out, ratchets
it up even tighter than you were before, now what happens?
MR. BAHADUR: I think that's a very interesting
issue, because every time we say we're going to improve the
methods, there's always an underlying assumption, like we
are going to reduce the burden, which isn't always true.
In this particular case, when we endorsed the mil
standards, in my mind, the mil standards were way too high.
It's easy to endorse an existing standard. It's very
difficult to unendorse a particular standard, unless we have
something over pressing against that particular standard.
What is happening right now is when we went to the
public meeting and we got this sentiment from the industry
that the mil standards are really way too high for us to
apply, and the question is, okay, if it's too high, what is
there which is representative of the actual condition in the
industry.
DR. POWERS: I would be interested in knowing how
they persuaded you they were way too high. Did they do a
risk analysis?
MR. BAHADUR: And these are the kind of questions
that we need to answer and the result of that public meeting
was that we should get in touch with EPRI, see if there is a
cooperative possibility of looking into this issue, see
whether the mil standards indeed are really higher than the
industry needs, and come to realistic standards and endorse
them.
So it's a three-step process, in my mind, which is
what we plan to do in this particular activity.
MS. ANTONESCUE: These tasks are all in parallel,
we're working in parallel, and basically the IEC standards
are more superior than the military standards and that's the
reason we are investigating now the potential endorsing of
these IEC standards.
MR. BAHADUR: And you will see in the schedule, on
page 39, against this activity, the start date for activity
-- or tasks one and two are second quarter in the financial
year 2000. And then in the third -- and then while these
are going on, when the results start flowing in, then we
move to task three.
DR. POWERS: Well, I'm reminded, it is not
surprising that the industry would think that the standards
were too strict and that may be true of any standard that
you set up.
MR. ARNDT: Just to clarify something. The
question you asked a few minutes ago, what studies were
done, both the industry did a study on this and the agency
did a study on this, both for levels of susceptibility of
equipment and levels in the plant.
So they did a study. We thought some additional
information needed to be independently developed and we did
a study as well. So we have a different --
DR. POWERS: So it wasn't based on risk. It was a
comparison between the study that will affect things and the
level of EMI/RFI that you actually have.
MR. ARNDT: Right.
DR. POWERS: For the deterministic side.
MR. ARNDT: Yes.
MR. CHIRAMAL: Actually, EPRI did a survey of a
number of the operating nuclear plants and came up with an
envelope of EMI/RFI levels at each plant and that obviated
the need for testing at each plant site before installing
equipment. So we have used that as guidance in the SRP at
this point.
DR. POWERS: But it seems to me that an EPRI study
on a few plants is not going to be adequate for modifying
the reg guide. If EPRI had done a survey of all the EMI/RFI
activities that ever show up at all plants, maybe you could
modify the reg guide.
MR. CHIRAMAL: I think part of the research effort
was to do just that.
MR. ARNDT: The environmental qualification area
is one that's also an ongoing activity, looking at
environmental stressors, such as smoke and others. A
significant amount of research was done in that area at Oak
Ridge and Sandia. We're in the process of wrapping that up
now.
The area of lightning protection is one that's
been discussed extensively, both at this committee and
various other areas. We are in the process right now if
putting the other research plan for that and that effort
will be looking at those issues.
The requirements assessment program is a very,
very interesting area. It's something that's been
articulated by many of the experts in the field, both in the
nuclear applications and in general fields, as one of the
most potentially significant areas of improvement. Root
cause analyses have found a significant fraction of digital
system failures can be traced directly back to either poorly
articulated system requirements or poorly interpreted system
requirements as it goes through the life cycle of the plant,
of the development of these systems.
DR. POWERS: Steve, I always love it when you talk
to us, because you build such enthusiasm for the things that
you support. Now, he says, gee, most significant,
tremendous leaps can be made here, sounds good to me.
Then I read what's written, a technical report
will be written. Maybe you could work a little on the
language here to build the same sort of enthusiasm for this
work that you yourself have so that I really honestly think
-- and, you know, if I get another NUREG coming to me, it's
probably not going to change my life a whole lot.
MR. ARNDT: Okay.
DR. POWERS: If I get a new understanding and a
new way of doing business that you're talking about here,
that will change my life, and that kind of enthusiasm just
doesn't come through in it.
MR. ARNDT: It doesn't come through.
MR. CHIRAMAL: The point is well taken. The next
area is the diagnostics and fault tolerance. Again,
something that is an issue that has really come to the
forefront with the increased use of digital applications.
One of the real advantages of digital applications
is you have these on-line diagnostics, fault tolerance
systems. Also one of the issues associated with is the fact
that as you have these complex --
DR. POWERS: This whole area just perplexes me.
Throughout the beginning of your document, you repeatedly
say one of the problems that you encounter is the high level
complexity. And though I can sit here and argue with you
over what you mean by complexity and whatnot and be picky, I
didn't.
I just accepted that these things were complex.
And then I come to this area and it says, gee, these things
self-diagnose themselves, they can recalibrate themselves,
they do all that. I say now I know why they're complex.
They put all this extra garbage that we never had in the
analog systems and it made them very complex and,
consequently, difficult.
Why don't we just get rid of that stuff, do the
simple job that the analytic -- or make them very simple
systems.
MR. ARNDT: Well, there's two or three different
reasons. First of all, we don't design these things, they
do. They bring them to us and there's a lot of potential
advantages to having diagnostics in these systems, both in
reliability, stability. They can do drift correction and
these kinds of things.
We also have the issue that a lot of these things
are either COTS or modified COTS and the other industries
are not as concerned about safety critical failures as we
are, so they are very interested in reliability, and the
COTS systems are becoming more and more embedded with these
kinds of systems.
DR. WALLIS: What is a COT?
MR. ARNDT: I'm sorry. Commercial off-the-shelf
system.
DR. SEALE: One of the interesting things about
built-in diagnostics is they don't shut the plant down when
you do the test.
MR. ARNDT: Right.
DR. SEALE: That has some remedial value.
DR. POWERS: Well, sometimes they do. I mean, we
have -- you have one, I think, in this report, a lovely
example where the system is off doing its self-diagnostic
and then it gets a signal that says do something and it
promptly ignores it.
So there's a real problem here and obviously
there's some happy medium that m analysis exist. But I
think what you're telling me is much as I would like nice,
simple digital systems, I can't get them.
MR. ARNDT: Right.
DR. POWERS: That the complexity is built in
there. I think that's a thought that needs to come across
in the report when you come in. It's not that it's really
missing. I think that if you add two and two between
various sections, you come up to this conclusion that the
complexity is indeed the source of the problem, but you're
stuck with it.
But it would be nice if you made that very clear,
that you just can't go out and buy a simple digital system
that only does the activities of the analog system. That
just doesn't exist out there, and I think that's a very
crucial point, that you're stuck with the complexity.
MR. ARNDT: Right.
DR. UHRIG: Well, isn't there a distinction here
in that regard with reference to safety systems versus I&C?
Safety systems fundamentally are as simple as you can make
them. Below this level, operate; above this level, trip.
It's go/no-go.
Whereas the I&C systems have all the bells and
whistles that we've been discussing here. So you need to
make a distinction between the two systems.
MR. ARNDT: By and large, that's correct, although
we're starting to see not necessarily in applications, but
in vendor designs, even things as critical as SFAS, they're
starting to have these kinds of things in them. So, yes, by
and large, RPS, SFAS, these kinds of systems are much
simpler. They are, by and large, specifically designed for
nuclear application.
But even in those systems, we're starting to see
this problem. The operating systems is something that's a
significant issue in all digital systems. Everyone who has
a PC understands that operating systems can have significant
impact on your applications. That's true for real-time
digital systems, as well.
The operating systems are much simpler and have
less overhead, but they still have significant impact on the
actual operation of what's going on and we need a better way
of dealing with them.
The next area is the whole issue of software
quality. We've broken it down into two basic areas,
software engineering criteria and software testing. The
engineering criteria is basically the design aspects, if you
will, of the process, understanding the difficulties of
setting the limits in the design process, trying to figure
out how to measure these kinds of things, trading off,
process oriented QA issues, like the capability maturity
model versus specific application metrics.
The current SRP has a laundry list of things you
need to look at to see whether or not they've done a good
job, but no metrics on how good the job is.
This is an example where we're trying to provide
methods and tools to improve the efficiency and
effectiveness of that process.
DR. UHRIG: Are you beginning to see any tendency
towards automatic code generation?
MR. CHIRAMAL: Yes. The Siemen's Teleperm XS does
that. They have a tool called SPACE that takes the
requirements and translates them to code through a machine
and then verifies it through the same machine backwards.
DR. UHRIG: And this is part of the review that
you have to go through.
MR. CHIRAMAL: Yes.
DR. UHRIG: So this is part of the evolving
technology.
MR. CHIRAMAL: That's correct.
MR. ARNDT: Right. One of the real issues in
emerging technology, involving technology is not only are we
seeing new applications, new systems, but we're seeing
changes and improvements in the current systems.
AS you well know, the issues of artificial
intelligence have been around for a long time, but up until
very recently, they haven't actually been getting into the
plants. The issue of code generation and automated
processes have been around for a long time, but they're only
now starting to really get into the process of designing
these systems and going into the plants.
So we have both the issues of the changing of
technology, the new technology, and the changing methods,
all of which we have to try and deal with.
Software testing is the specific issues of what
needs to be done in software testing. The current SRP
basically says you have to do software testing, but it
doesn't have a set of metrics by which we say how much
testing, what kind of testing, what kind of results. The
research in this area is to try and understand what is good
software testing, what is a good set of numbers, what are
the standard methodologies.
It's a very immature, if I can say, issue because
when we had our expert panel meeting, we said here's -- I
forget the number -- it was like 40 different software
testing methodologies and we asked the experts to rank them
and come up with some kind of qualitative valuation, and
they weren't able to do it. It's something --
DR. UHRIG: Was it simply a divergence of opinion
or is it that everybody was not familiar with 40 different
types of techniques or what was the problem?
MR. ARNDT: I think it was a combination of all
those things and also the applicability. We are primarily
interested in real-time safety critical type application,
which is a small, but significant part of the industry and
most of the software testing metrics out there are not
designed for that kind of application. They're designed for
general application.
DR. APOSTOLAKIS: Did the fact that the software
is usually part of a system come into this? In other words,
there are inputs that are generated by the rest of the
system.
MR. ARNDT: That's right, and how it functions on
the particular piece of hardware and things like that. The
issue of whether or not software testing and software
assurance needs to be integrated with the particular
application, which is what you're mentioning, and the
particular platform that it was using is another issue,
because perturbation based on flipping a bit here or there
or having this various input is something that most software
testing doesn't look at.
Some does, some of the metrics look at that.
DR. APOSTOLAKIS: What do you mean by metrics?
MR. ARNDT: Things like --
DR. APOSTOLAKIS: Give me an example of a metric
that applies here.
MR. ARNDT: Percentage coverage of a software
test.
DR. APOSTOLAKIS: Percentage of what?
MR. ARNDT: Theoretically, you have some large
countable infinity of different paths you can go through
with a piece of software.
DR. APOSTOLAKIS: Do you know those?
MR. ARNDT: Most people don't know them. In
theory, you can know them. Software test coverage is a
metric on how many of those you were actually trying. So 20
percent, 50 percent, one percent of different ways you can
work your way through the software.
DR. APOSTOLAKIS: Now, these ways are determined
by the rest of the system, aren't they?
MR. ARNDT: In most cases, yes.
DR. APOSTOLAKIS: So if you don't know those,
you're talking about 20 percent of something you don't know.
MR. ARNDT: For example --
DR. POWERS: We've had one speaker from the
reviewers come in and say he can't do this on the software,
you've got to do it on the integrated system.
MR. ARNDT: Right.
DR. APOSTOLAKIS: Yes.
MR. ARNDT: That's Dr. Johnson's theory. And if
you look at, for example, the software testing review that
NRR did on the platforms that it did, they looked at
platform -- that is to say, the hardware, the software, and
the general application, did a review. In point of fact, if
you want to do it completely, you then go back and look at
the specific applications again.
At some point, you have to make a decision about
how general do you want the specific metrics to be. But
right now, the review guidance is very general and we want
to make it more specific.
DR. APOSTOLAKIS: Maybe you can explain to me
better if I give you an example of something that actually
happened.
MR. ARNDT: Okay.
DR. APOSTOLAKIS: In one of the fighter planes,
the following thing happened. The pilot, of course, can
raise the landing gear when the plane is flying.
MR. ARNDT: Right.
DR. APOSTOLAKIS: And it just so happened that he
did it accidentally when the plane was on the ground and the
gear was lifted, raised. Is that a software error?
MR. ARNDT: In all likelihood, it's actually a
design specification error.
DR. APOSTOLAKIS: But how would software testing
find this? The software did what it was designed to do.
MR. ARNDT: Yes. And if the --
DR. APOSTOLAKIS: But the context was different.
MR. ARNDT: Right, the context was different. The
requirement was poorly interpreted. And if you look at
software testing procedures, they are generally generated
specifically from the requirements. For example, you'll say
these are the requirements for this airplane, and I'm a
pilot and I take this personally, you want the landing gear
part of the software to go down when it's supposed to, come
up when it's supposed to, operate properly.
There may be an interlock that says when you're
below so many feet, it automatically comes down.
DR. APOSTOLAKIS: Right.
MR. ARNDT: And there will be a set of
requirements, and one of them is don't lift it while you're
on the ground.
DR. APOSTOLAKIS: Right.
MR. ARNDT: And that requirement will then be fed
into the preliminary design report, the design reviews and
the testing requirements and a good testing program will
specifically test all the design requirements and all the
testing.
DR. APOSTOLAKIS: So you don't do that here in
this box.
MR. ARNDT: No. You do that in the --
DR. APOSTOLAKIS: The requirements box.
MR. ARNDT: -- requirements box.
DR. SEALE: But to test the software, you have to
try to raise it while it's on the ground to see whether or
not the requirement got properly put in it.
MR. ARNDT: Right, exactly. And like I say, all
of these are interrelated.
DR. SEALE: Successfully, I hope. Unsuccessfully,
I hope.
MR. ARNDT: And depending upon the sophistication
of the program, a lot of people build simulations of the
software and hardware and test them both in simulation
space, as well as in real hardware space.
DR. APOSTOLAKIS: So here then you would test only
the ability of the software to raise the landing gear.
MR. ARNDT: Right.
DR. APOSTOLAKIS: Independently of the context.
MR. ARNDT: Independently.
DR. APOSTOLAKIS: And in another box you would
worry about that.
MR. ARNDT: Whether or not the requirements had
been appropriately written into the software.
MR. CHIRAMAL: Dr. Apostolakis, software testing
is two methods. One is you check the functions that are
performed by the software. The other is things like in
designing, you test the modules for its input and outputs
and then you join the modules and you do some testing, and
that's part of software testing also.
At present, in the SRP, what we have done is we
referenced the IEEE standards on unit testing and functional
testing.
DR. APOSTOLAKIS: So then the whole discussion
shifts back to the box that says requirements assessment.
MR. ARNDT: Right.
DR. APOSTOLAKIS: Because, again, you have to look
at the whole system to be able to test those requirements.
MR. ARNDT: That's right.
DR. APOSTOLAKIS: And its various uses.
MR. ARNDT: And there's a lot of controversy in
that area as to what the best ways of doing that are.
DR. APOSTOLAKIS: You have to look at the system.
There can't be a controversy about that.
MR. ARNDT: No, there's not.
MR. CHIRAMAL: But to assure the quality of
software, you've got to do some software unit testing,
module by module, part by part, to make sure that it does
its function.
DR. APOSTOLAKIS: But when you are testing the
requirement or you're checking the validity of the
requirements.
MR. CHIRAMAL: That's integrated testing when you
put the software and the hardware together.
MR. ARNDT: The next area is the whole issue
associated with improving, in many cases developing our
ability to understand digital systems in a risk context.
The first area --
DR. APOSTOLAKIS: I'm a little bit puzzled myself
here.
MR. ARNDT: Okay.
DR. APOSTOLAKIS: We're talking about the plan,
but some elements of this plan are already being
implemented, right? I understand University of Virginia has
been doing work in this area.
MR. ARNDT: Yes, that's correct.
DR. APOSTOLAKIS: So the plan is coming after we
started doing certain things.
MR. ARNDT: The plan is a from today forward plan
and some of the activities are being integrated, have
already started. For example, the University of Virginia is
doing work in digital system failure assessment, looking at
how things work, primarily hardware/software assessment and
that kind of issue.
We have done some in data analysis in-house, and
we intend to do some more.
One of the little short studies we did is appended
to the plan.
So, in several of these areas, including this
area, we've started doing work in some areas, we've done a
lot of work, and we'll be completing it within this
five-year plan, and in some areas, we haven't even started
yet, and we'll only just begin to start working by the end
of our five-year planning horizon.
DR. POWERS: In your document itself, you have a
task list in here that says something to the effect,
identify and develop risk analysis models for I&C systems.
Are there such beasts to be identified?
MR. ARNDT: There have been various proposals put
forth -- for example, software, a fault tree analysis, and
things like that.
DR. POWERS: It would have been useful in the
report, and actually interesting, if you'd give me some
indication of the range of proposals that existed, because
this has always been an area of such substantial heat
between the analogs and the digitals --
MR. ARNDT: Yes.
DR. POWERS: -- and saying that, well, digitals
just are incompatible with the concepts of risk because of
their peculiar failure characteristics, they either don't
work when you get them or they last forever and immune to
everything except a GI with a hammer or something like that.
You know, in reading the plan, especially in those
activities where you have things like I'm going to identify
and I'm going to look at what the possibilities are out
there to give some indication of what the -- there was some
hope to find things --
MR. ARNDT: Okay.
DR. POWERS: -- and some indication of the range
and the variability of things, then make it much more
obvious that a research program is merited here, as opposed
to me just pulling out the handbook and saying do this.
MR. ARNDT: Okay.
So, for example, in this area, we could articulate
we're going to look at areas such as --
DR. POWERS: Uh-huh.
MR. ARNDT: -- and list three or four.
DR. POWERS: Yeah. I don't think anybody is
asking you be comprehensive in a plan, because you haven't
done the work, but it would have been useful to understand
that there were -- where there were technical capabilities
out there and differences of opinions among experts that you
really needed to assess --
MR. ARNDT: Right.
DR. POWERS: I think that would have helped me a
lot more to understand the nitty-gritty of the thing,
because I knew the product I was getting was just a report.
MR. ARNDT: Right.
DR. POWERS: I'd really rather have some
understanding.
DR. APOSTOLAKIS: Actually, one of the methods, I
guess, the words in the box would not be acceptable, because
you are not assessing the risk of digital I&C systems; you
are assessing the risk of the high-pressure injection system
if it includes digital I&C, because that method completely
rejects the idea of assessing the risk from software.
The software drives the hardware, gets information
from the process, and comes back and tells the process do
this.
So, what you want to know is what is the risk or
the reliability from the process itself when you have
embedded software to do all these things, but the software
itself, you know, is just buried there, and that work was
funded by the NRC, by the way.
MR. ARNDT: Yes.
The whole issue of whether you talk about how
software affects or digital systems affects the reliability
of plant systems or the plant itself or talk about software
reliability, which is, depending upon who you ask, a
misnomer, to a certain extent it's semantics, but it does --
DR. APOSTOLAKIS: It's very important. Semantics
is very important sometimes.
MR. ARNDT: It is important in many cases.
DR. APOSTOLAKIS: Because there isn't such a thing
as a reliability of a computer code.
MR. ARNDT: Right.
DR. APOSTOLAKIS: Professor Levinson has given
several talks in the public where she said, you know, people
come to me and they give me a software package and they say
can you tell me how reliable it is, and my answer, without
looking at it, is that the reliability is one.
MR. ARNDT: Yeah.
DR. APOSTOLAKIS: Unless you tell me where you're
going to use it, the question is meaningless.
MR. ARNDT: Right.
The various areas we have here include, of course,
the basics that you want to do for any kind of risk
assessment kind of thing, look at the data, understand the
failure modes.
DR. APOSTOLAKIS: That intrigues me. What kind of
data? I mean I thought this was a new technology.
MR. ARNDT: It is a new technology, and you get
into the issue that we do in many cases, what is the
applicability of the data, how much data is available, if
any?
DR. APOSTOLAKIS: I gave you a piece of data with
a fighter plane.
MR. ARNDT: That's right.
DR. APOSTOLAKIS: What do you do with that?
MR. ARNDT: Well, you have to assess whether or
not it makes any sense to include it into a database that
we're going to care about, and a more difficult issue than
your particular example is all these commercial
off-the-shelf systems, many of which exist in the process
industry, in the petrochemical industry, in the medical
industry, in the railroads.
Lots of different people do real-time processing,
and the have done it more than we have, not a lot more,
because it's a new technology, and also the whole
applicability of do we care what the failure rate of a
system that's based on an 8088 is compared to what we're
going to do with a new system today?
It's a tough issue, but unless we start looking at
it and trying to assess it -- failure models without data
don't mean anything.
DR. POWERS: To some extent, you actually
articulate that in the existing report, but to the extent
that you can make it can clear in the data analysis --
there's a lot of data out there that would not contribute to
a useful database.
MR. ARNDT: Right.
DR. WALLIS: To go back to what we talked about at
the beginning, I get the impression you're working on a very
extensive problem, and you have -- there isn't that much to
start with, and if you can say we know all these things
today and these are the areas we really need improvements in
or something, but you're starting with the whole problem
laid out as a burgeoning problem, and that makes me worry
about how much you really need to do to understand it. It
is really like that, or is a lot already known and there are
just certain weak areas?
MR. CHIRAMAL: One of the problems the experts say
is that you cannot -- it's very difficult to quantitatively
assess the reliability of digital systems.
DR. WALLIS: But you say everything is adequate.
You started off saying everything is adequate. I get the
impression it's adequate because no one knows much at all.
You don't know where it's inadequate. Am I getting the
wrong impression?
MR. ARNDT: Not entirely.
One of the issues that Matt highlighted is that if
you're trying to bring this into a risk-informed-type
application, then you have to be able to put a number to it,
and if you don't know a whole lot and you don't have a lot
of experience, you have to put the number quite high, and
depending upon the experts you talk to, most people say an
integrated software system cannot be shown to be any better
than maybe 10 to the minus 3.
There's a lot of people who believe that
integrated software systems are much more reliable than that
in particular applications.
In point of fact, we have a lot of examples where
we have high-demand systems that are software-driven that
operate for years without problems, but you cannot
theoretically or analytically show that they're anymore
reliable than 10 minus 3.
That's a big problem when you're trying to deal
with systems in a risk perspective that have these embedded
software-based systems in them.
In a lot of ways, the issue is understanding how
to assess these with some kind of certainty when people are
claiming --
DR. APOSTOLAKIS: Well, again, there are other
people who think that statements of this type don't make
sense.
MR. ARNDT: That's right. I mean the concept
we're talking about, reliability for software --
DR. APOSTOLAKIS: In isolation.
MR. ARNDT: -- in isolation --
DR. APOSTOLAKIS: Guess which side I'm on?
MR. ARNDT: Yeah, I can take a guess. Hotly
contested discussion.
DR. APOSTOLAKIS: Now, risk importance -- you say
in the report that you are really looking at the risk
importance of some, again, positions in the PRA, whether
it's an analog system or digital, it doesn't matter, it's a
function, right?
MR. ARNDT: Right.
DR. APOSTOLAKIS: Again, I wonder whether that's
consistent with the view that the software will be embedded
everywhere, in the whole system.
Can you really say this is now an event that's
caused by the software? I don't know.
MR. ARNDT: In some cases, you can. In some
cases, you can't.
DR. APOSTOLAKIS: Yeah. I think that's a good
answer, because it's all-inclusive.
MR. ARNDT: That's right. And if you look at the
database that we're working on and some of the study work we
did, a lot of stuff, even in a detailed analysis, you don't
know, because the root cause didn't go far enough to do it.
DR. APOSTOLAKIS: That's right.
MR. ARNDT: In some cases, we have very specific
examples that it was a problem because of that.
DR. APOSTOLAKIS: And the other thing that we
worry about is that the digital software may introduce new
failure modes --
MR. ARNDT: That's right.
DR. APOSTOLAKIS: -- which are not in the PRA.
Therefore, you cannot assess the risk importance.
MR. ARNDT: That's right.
DR. POWERS: It seems to me that one of the
problems -- correct me if I'm wrong.
MR. ARNDT: Okay.
DR. POWERS: Dealing with these digital systems,
you have the problem that, unlike lots of components, it
either works or it doesn't.
You can have the intermediate situation of it
works badly.
MR. ARNDT: Yes.
DR. POWERS: Okay.
So, the kind of binary logic that's inherent to
most of the trees that we use in PRAs is really not -- may
not be applicable to these things.
MR. ARNDT: It's not applicable. It's not as bad
as all that, because most safety-critical real-time systems
have -- assuming they work properly -- self-diagnostics and
things like that, that if you have an unexpected state, it
will tell it to turn off or whatever.
Of course, you can run into the whole problem of
the diagnostics or the air correction or the watchdog timer
or whatever it is causing an additional problem, but the
whole issue of how it fails, what it looks like when it
fails, does that failure promulgate along an information
pathway that we have not looked at, the whole issue of how
are these things connected or not connected is something
that is different from what we've looked at before.
DR. UHRIG: Steve, we're going to have to wind up
here fairly soon.
Could you sort of hit what you consider the high
points?
MR. ARNDT: Okay.
Well, if George will let me --
DR. APOSTOLAKIS: You're going so slowly.
DR. POWERS: You've really been bogged down here,
Steve.
MR. ARNDT: Okay.
DR. APOSTOLAKIS: It takes you 10 minutes to go
through one view-graph.
MR. ARNDT: The last and probably most important
part is the whole issue of emerging technology, and the
first block is technology review and infrastructure -- I'll
talk about more in a second. So, let's skip that briefly.
We've looked at three or four general classes of
emerging technologies that we know or have a pretty good
understanding are going to have specific applications to
nuclear technology, and we want to look at them
specifically.
They include the whole issue of on-line
maintenance and predictive maintenance, advance
instrumentation, be it for improved accuracy or improved
reliability or just new systems, I mean new different types
of sensors that we haven't looked at before, smart
transmitters.
DR. POWERS: Let me ask this question. You're
saying let's go off and look at all these systems that may
or may not be used.
MR. ARNDT: Right.
DR. POWERS: Okay.
Maybe you're a pretty bright guy and you can say
these probably will and these probably won't.
Why doesn't the NRC take a different tack and say
we want the integrated system to have such-and-such a
reliability, and you, industry, if you want to use these new
stuff, show us that this is here, that it does that, it
meets this standard, and not go off in the emerging
technologies, because it's such a vast and speculative
field, and it changes every week.
I mean my impression is it changes0every week.
How do you keep this from proliferating -- I mean
I understand firewalls of security, that you might want to
look at that, but --
MR. ARNDT: Yeah.
DR. POWERS: -- the rest of it -- why not say it's
hopeless for me to stay abreast of this?
MR. ARNDT: Okay. Well, there's a couple reasons.
First of all, from a programmatic standpoint,
these are all out-year-type issues, and this plan will get
revised, etcetera.
So, as we know more, we'll either add resources or
delete resources from things that are important, but the
more important issue is, because it moves so quickly, two-,
three-year kind of timeframes, if we don't start looking at
these issues before the industry drops a topical report on
us, we cannot stay efficient and effective.
More importantly, we may not be able to catch
everything, because we simply don't understand the
particular applications that they're bringing to us, and
even if we set a standard, that standard may not be
applicable to a new technology.
DR. POWERS: I think it's an issue you need to
confront directly in the report, and I think I understand
your answer --
MR. ARNDT: Okay.
DR. POWERS: -- that you couldn't -- you wouldn't
know how to read the topical report if you remain ignorant
on everything.
MR. ARNDT: Right.
DR. POWERS: It would be fate-based regulation,
one that I'm particularly in favor of, but that's not maybe
what the agency wants.
DR. APOSTOLAKIS: Would this be anticipatory
research?
MR. ARNDT: Yes.
DR. APOSTOLAKIS: And the Office of Research is
supposed to do that, right?
MR. ARNDT: That's right. And specifically, we
have a task in emergency technologies for technology review
and infrastructure.
DR. APOSTOLAKIS: So, what is the smart equipment
again?
MR. ARNDT: Pardon me?
DR. APOSTOLAKIS: Smart equipment?
MR. ARNDT: Smart transmitters, smart sensors.
This area is of particular importance because
we're trying to do three things.
We're trying to deal with the standards as they
change and endorse them or determine what they are and
influence them, something that is near and dear to my NRR
colleagues.
DR. POWERS: Let me ask you about these standards.
MR. ARNDT: Yes.
DR. POWERS: I'm not an electrical engineer and
I'm not a digital person, but I do trouble myself to go look
at these IEEE standards in this area, because it's such an
important area, and they're immense, and they're all written
in 8-point type and dual column, and they seem to be written
for digital systems that control airplanes, for feedback and
control systems, and the kind of systems we have in nuclear
power plants are, Professor Uhrig pointed out, you know,
safety systems, they're supposed to be simple.
Do we look to these standards and say what is the
ore out of this needed for the simpler systems we have, or
are they just not dissectable, it's an all or nothing kind
of thing?
MR. CHIRAMAL: It's the latter, it's all or
nothing, because these particular IEEE standards are not
designed for nuclear engineering; it's for all applications
of software, and hardware, of course, and it's general IEEE
standards.
When we endorsed it with the reg guide, we put
some caveats on certain aspects of it.
MR. ARNDT: And one of the biggest issues is some
of these are very good and very applicable, some of them are
not real great and not very applicable, and all combinations
thereof, and they also change a lot quicker than ASME code
standards and things like that.
So, working on and keeping abreast of the ones
we're interested in is something that we have to consciously
do if we want to do a good job, and it's part of the
infrastructure of dealing with this issue.
Another part that we need to be more conscious of
than we have in the past, even though we've done a lot of
work in this area, is dealing with interacting and
interfacing with other people, and one of the tasks is to
specifically do that, and we have a whole section of things
that we want to do in that area, and we'll continue to
expand that area, as appropriate.
And the third area is we want to physically look
at what's going on in the industry and determine what we
need to do, read the reports, go to the conferences,
understand what's involved from that activity, new
activities and emerging technology will be added or deleted,
and from emerging technologies, those activities will be
pushed into the short-term activities in the other major
categories.
So, we see this as a flow from reviewing what's
going on to looking at it as an emergent technology to
looking at it as a specific application.
DR. POWERS: Let me ask a question that you may
not -- you may be the wrong person to ask, and that's okay.
I get this guy, he is on top of the latest and the
greatest coming out of this field, that's been his job.
Am not just creating a guy who promptly gets
offered millions of dollars to go off and work for a Silicon
Valley company where he can retire at the age of 35 and
things like that?
MR. ARNDT: I haven't got any offers recently.
DR. APOSTOLAKIS: The guys who can go and do that
are not the right people to do this kind of work.
MR. ARNDT: It is an issue.
Mike, do you want to talk to that a little bit?
MR. MAYFIELD: Mike Mayfield from Division of
Engineering Technology.
Maintaining the staff, training the staff, as you
develop very skilled, high-quality folk, you always run the
risk that they're going to go elsewhere. That's just a fact
of life.
I don't know of any way of chaining them to their
desk.
DR. POWERS: This is a real problem.
MR. MAYFIELD: It is a problem. It's a problem
for us. It's a problem in the industry, as well, because
highly-trained people are a sought-after commodity, and they
tend to move.
I don't see this as any different problem than,
frankly, the one Sandia faces or any of the other national
labs face.
DR. SEALE: Stupidity is not the answer.
MR. MAYFIELD: Yeah. Keeping a dim-witted set of
folks is not the direction we want to go, and it's just the
realities of business today.
DR. POWERS: So, this digital system is just going
to be a big manpower problem.
DR. APOSTOLAKIS: When are we going to actually
see the work that is being done under these boxes? I'm
seeing two plans today, and I'm OD'ing on plans.
DR. POWERS: He has completion dates on some of
these things.
DR. APOSTOLAKIS: I don't want to see it after
they're completed.
Can we have a subcommittee meeting, Mr. Chairman
of the subcommittee, where we're going to have some detailed
exposition of what is going on?
DR. UHRIG: That's one of the possibilities. We
have not considered that at this point.
As Steve mentioned earlier on, this came up
relatively rapidly, got on the agenda, and the plan has been
developed over a relatively short period of time, and it was
put on the agenda without a subcommittee meeting.
DR. APOSTOLAKIS: I'm not talking about the
subcommittee on the plan itself. I mean the boxes, what's
going on inside the boxes.
MR. ARNDT: We had a subcommittee meeting, I
think, about a year-and-a-half ago that talked about several
of the issues.
I think Don was the subcommittee chair then, and
we, of course, brought Dr. Johnson from Virginia in to talk
about some of the work that he was doing, but --
DR. APOSTOLAKIS: When is that work going to be
completed, by the way?
MR. ARNDT: I don't recall the exact date, but we
hope to have a product on that.
John?
MR. CALVERT: John Calvert from Research.
We're going to have -- right now we're working on
a pilot project with Calvert Cliffs, taking a digital
feedwater system and applying the University of Virginia
methodology and come up with is their methodology workable
and applicable to a project, and based on that, then we can
decide what we have to do from there, but --
DR. UHRIG: What's the timeframe of that? A year?
MR. CALVERT: The timeframe is -- it's scheduled
for the end of this year, but we're running into -- or we
have run into proprietary agreements and things like that
that we had to work out.
DR. APOSTOLAKIS: Are there any interim reports
that we can review?
MR. CALVERT: Not at this time.
MR. ARNDT: We have some interim reports on some
of the other projects, some of the software QA issues and
things like that, that we could forward to the committee if
they're interested in looking at them.
DR. POWERS: Maybe we can just go ahead and wrap
up the presentation and discuss the schedules off-line.
MR. ARNDT: Okay.
DR. POWERS: Because I promised to let you go to
lunch.
The committee is willing to sit here till late
hours at night, but I did promise you.
DR. APOSTOLAKIS: You are not.
MR. BAHADUR: This brings me to the last slide of
the presentation. I know we are in a time crunch right now.
I just want to bring a couple of things for your
information.
As you have noted, this is a forward-looking plan,
but not really just forward-looking plan. In the past,
digital I&C effort have been mostly to respond to NRR user
need, and work has been conducted in that area for the last
five, six years.
So, what this plan does is it brings us to the
level where we are today and then how we can fit whatever we
have done plus what we propose to do in the overall picture
of the goal that Steve had presented.
We came to subcommittee last fall, where we talked
about the work with the University of Virginia, and I agree
with George that I think this committee needs to see what
all we have done so far in the digital I&C, and we can
discuss that off-line, but my one suggestion would be that
whatever work we have done so far as a user need response in
the I&C area.
We can make a full package out of that and send it
for your review, and if you see the need, we can come and
discuss that with you, or if you just want to read, then
that's okay, as well.
What I plan to do is, once this plan gets approved
and I get the necessary resources, I will continue to work
the approach that we have taken in the past but supplement
that a little more by conducting the work not only with the
contractors and the DOE labs but also bringing an awful lot
of work in-house.
We plan to do work, cooperative research with
EPRI, and then, of course, we'll be doing work with the
universities. The University of Virginia is one example
where we have been doing some work, and on the international
arena, we will do the work with Holland, which will give us
a window to see what other countries are doing in this area.
DR. APOSTOLAKIS: Is Livermore your main
laboratory on this?
MR. BAHADUR: Livermore did some work in the past.
Right now, I don't have any active project with Livermore.
What we're looking for for this briefing would be,
after the committee has a chance to review and see the
report, is to -- a letter to the Commission, because we plan
to send this plan to Commission as information.
DR. UHRIG: When?
MR. BAHADUR: The plan is to send it -- I was
shooting for May. The end of May, we would be sending the
information paper to the Commission.
If the committee were to write in its letter the
importance of this area being acceptable to the committee,
the approach that the staff has taken is in the right
direction, and then if the improvements are needed in this
plan, then specific suggestions as to what need to be done.
I mean I heard an awful lot about where the plan
could be improved in terms of putting the similar enthusiasm
into the plan as was in the presentation, and those kind of
views are very useful to us, but in more philosophical
sense, have we taken the approach that we should have taken,
keeping in mind that the goal is not only to meet the NRR's
immediate user need but to also keep in mind that the agency
is going in the risk-informed direction, whether we are
responsive to that, and also the imaging technology, which,
whether we like or not, is here to stay, and if we accept
that the emerging technology is here to stay, then the NRC's
preparedness is a question, and whether this particular plan
leads to a research activity which help NRC to get prepared
to respond to the industry.
That's the kind of comments we are seeking from
the committee.
DR. APOSTOLAKIS: Well, there are two things that
come to mind, especially judging from a recent experience
with another area, LPSD, low-power and shutdown, where they
produced a report, as well, with all sorts of research
needs.
I think you need to do two things.
One is to go back and work on the goals and the
importance of this and not leave it up to us to define them
and maybe identify them in bullet form, you know, and the
other one is I think you need to prioritize these things.
MR. BARTON: A lot of work here.
DR. APOSTOLAKIS: You have a hell of a lot of work
there, and what do you expect the Commission to do, to give
you the resources to attack all of these?
MR. BARTON: A zillion dollars.
DR. APOSTOLAKIS: Probably you will get a five to
zero decision against.
MR. ARNDT: The plan is that the resources
associated with this will be prioritized within the research
prioritization scheme.
The plan will go to the Commission as a
information notice as to this is what we're planning on
doing and this is why we're planning on doing it, and the
resources will be dealt with as part of the research
prioritization process.
DR. UHRIG: You're going to put all the resources
at risk.
DR. APOSTOLAKIS: It didn't work out that way for
other areas.
DR. POWERS: Well, that's a strategy I think we're
going to have to let them wrestle with.
DR. UHRIG: I think we've reached the witching
hour.
DR. POWERS: Okay.
MR. ARNDT: Thank you again.
MR. BARTON: No, that's 7:30 tonight.
DR. POWERS: I will recess until 1:40. Maybe I'll
recess until 1:30.
[Whereupon, at 12:40 p.m., the meeting was
recessed, to reconvene at 1:30 p.m., this same day.]
. A F T E R N O O N S E S S I O N
[1:30 p.m.]
DR. POWERS: Let's come back into session.
We are now going to discuss performance
indicators, but this is a new breed of performance
indicators, not the ones that we've discussed in the past,
and these are to be risk-based performance indicators, and
George, I guess you'll take us through this?
DR. APOSTOLAKIS: Yeah.
We had a subcommittee meeting in December. The
staff presented to us what they are trying to do in this
area and also the results of some data collection of
unavailabilities of systems and so on. So, today, they are
briefing the committee on that work.
I guess you're requesting a letter, right?
MR. BARANOWSKY: Yeah, or at least some comments
to us that we can use to see what the opinion of the
committee is, because we do have plans to go to the
Commission in June, and we need to say that we have
discussed this and how we are addressing ACRS comments, if
any.
DR. APOSTOLAKIS: So, the floor is yours.
MR. BARANOWSKY: Okay.
So, for the record, I am Patrick Baranowsky, chief
of the Operating Experience Risk Analysis Branch, and we are
going to be talking about risk-based performance indicators
and industry-wide performance measures today, which are sort
of integrated aspects of the same thing but somewhat
different.
With me is Steve Mays, assistant branch chief, and
Hossein Hamzehee, who is the project lead for this technical
work that we have ongoing.
We use Idaho National Engineering and
Environmental Laboratory, and what's the name of the other
group?
MR. HAMZEHEE: ISL.
MR. BARANOWSKY: ISL. It's the split-up of
Scientech into piece-parts, and the one that's working
regulatory matters is called ISL.
This topic, by the way, in addition to having
discussed it with you in December, we've also had a meeting
with our counterparts, including the office director of NRR
and the Executive Director for Operations, and the second
chart here shows what I want to cover today.
In addition to the purpose of talking about
risk-based performance indicators, I want to give some
background, and it's important for us to talk about what are
risk-based performance indicators, because I think there
might be some misperceptions as to what we mean by those,
what are the benefits of risk-based performance indicators,
how do they fit into the revised reactor oversight process,
and the process for developing risk-based performance
indicators, including some early results of work that's not
really at the stage where we need technical review but we
want to give you a flavor of how that's starting to come
out.
Just as a bit of background, you know, this idea
of risk-based performance indicators was brought to the
attention of the Commission back in August of 1995, and we
put together a plan for risk-based analysis of reactor
operating experience which was going to lead up to the
development of risk-based performance indicators back in
December of 1995, and we have talked to the ACRS
conceptually about this, and the concepts are getting a
little bit more detailed now, and so we're back again.
Risk-based performance indicators were also
indicated as a future development activity in the EDO
tasking memorandum of August 1998 that identified steps for
developing the revised reactor oversight process, and the
old AEOD, now defunct, sent out a risk-based performance
indicator proposed program plan in October 1998 to both NRR
and RES.
Not very much happened until about -- maybe a year
ago, and then, recently, the Chairman's tracking system
recognized this as a separate activity to be tracked and
part of the evolutionary aspects of the revised reactor
oversight program, and so, we have issued this so-called
white paper, which provides some concepts and philosophies
and how we plan to go about developing risk-based
performance indicators and provided an early draft to you
about a month or so ago.
It's had a few changes, and then it was sent out
to the public a couple of days ago.
Those changes are fairly small and involve various
technicalities that I don't think impact the thrust, the
gist of the paper.
DR. APOSTOLAKIS: On that point, as you have up
there, we discussed this two years ago, or something like
that.
I was a bit surprised when I read the white paper
that it talks about it at a higher level, talks about how
you plan to go about doing those things.
I thought it was going to be more specific as to
how you're actually doing it.
Is it because people weren't paying attention and
now there is revived interest?
MR. BARANOWSKY: That's quite a bit of it.
I think what we found was, in talking to both
folks in NRR and in industry, there were a lot of
misconceptions about the high-level concepts and that we
didn't do a really good job in laying those out in some of
our earlier discussions.
So, we tried to take a lot of care, in talking to
people about their perceptions of this, in putting this
conceptual paper together.
Now, as it turns out, a lot of the technical work
that's the foundation for the risk-based performance
indicators, is work that's been done in my branch on the
risk-based analysis of reactor operating experience for
which we have a briefing scheduled tomorrow morning, and so,
I think the technology isn't something that's new.
We're trying to use things that we have already
worked on and had a fair amount of interaction with you, as
well as NRR and other public organizations on in the past,
and just adapt them to the performance indicator business.
In fact, that's why we expect to be able to
produce a technical report, which I think is the meat that
you're looking for, in July of this year.
Now, that's not really very much time.
So, we're going to give you a flavor of the kind
of technical work that we're doing, but I don't think we
want to go over our incomplete papers, if you will, in an
ACRS meeting.
When we talk to you in July -- Steve will go over
the schedule, but that's the beginning of a more significant
technical review that will involve external and internal
stakeholders, then coming back to the ACRS again for more
technical discussion, maybe some workshops, and then finally
going to the Commission after all those technical reviews
and views are incorporated and revisions made and whatever
comes out comes out.
DR. APOSTOLAKIS: So, you have already done, then,
the work -- some of the work that you are proposing in this
white paper.
MR. BARANOWSKY: Yes.
DR. APOSTOLAKIS: It's not like you are starting
now.
MR. BARANOWSKY: Yes.
I mean we have a pretty good idea about what kind
of information needs to be collected, what kind of models,
what kind of statistical approaches need to be taken in
order to be able to have valid indications that have good
statistical characteristics and reflect what we think are
pretty good plant performance characteristics, too.
MR. BONACA: So, in part, you're using some of the
information you presented in December when we were looking
at the risk-based analysis of performance data, right?
MR. BARANOWSKY: Yeah.
MR. BONACA: That's really the information that
you're using.
MR. BARANOWSKY: Right.
MR. BONACA: I'm asking the question because
tomorrow morning we have a presentation, in fact, on that
subject.
MR. BARANOWSKY: Right.
MR. BONACA: The two issues are very much tied
together.
In fact, you presented this in December as a key
element of the other presentation.
MR. BARANOWSKY: Right.
MR. BONACA: Okay.
MR. BARANOWSKY: In fact, the chart that I used
that shows the different program elements that we have, and
right at the top are risk-based performance indicators, and
we'll go over that tomorrow, but all this stuff feeds in
there.
So, there's a lot of technical work that's on the
table that's behind this that we've talked about in the past
that I don't want to go over today mainly because what is a
risk-based performance indicator needs to be, I think,
appreciated, and that was one of the issues that was raised,
in fact, as to what is the definition of a risk-based
performance indicator.
We've changed that definition several times. Now
we think we have the right one, and I think it's important
that we have the perspective of what we mean by risk-based
performance indicators.
It's not tracking plant performance at the public
risk level, but it's more or less having the logic
associated with risk connected to the indicators. Maybe
that's the time for me to leave into Steve, who wants to
pick up on the balance of the presentation, if that's okay.
DR. APOSTOLAKIS: One last question.
MR. BARANOWSKY: Okay.
DR. APOSTOLAKIS: The work that you will complete
or the paper you will complete by July or so -- will that be
the final piece of work?
MR. BARANOWSKY: No. That's the, I guess I would
call it, draft of phase one --
DR. APOSTOLAKIS: Okay.
MR. BARANOWSKY: -- which Steve will explain.
DR. APOSTOLAKIS: Okay.
MR. BARANOWSKY: We're going to do this in phases,
because it's not so easy to do everything at once, and yet
we want to take advantage of what we think is already
available for use.
DR. APOSTOLAKIS: Okay.
MR. BARANOWSKY: Steve?
MR. MAYS: Okay. I'm Steve Mays from the Office
of Research, and for Dr. Seale and Dr. Kress, let the record
show that I am wearing both a coat and a tie.
When we met before with the ACRS --
DR. SEALE: I'm impressed.
MR. MAYS: When we met before with the ACRS
subcommittee in December -- and other comments we got from
other people that Pat has already alluded to about what do
you mean by risk-based PIs, and we've struggled quite a bit
on that, whether we wanted to get down and give definitions
of reliability and availability and frequency or something
more global or something in between, and we've finally come
to a decision about trying to have a more global definition,
because once we can get some agreement on that, I think the
rest of the steps of what you choose or not to choose to
have in a particular risk-based PI program become a little
more clear as to the rationale for why you do that.
So, we've put together a definition in response to
that which is a little more global than our previous one,
and it says risk-based performance indicators are indicators
that reflect changes in a licensee's performance that are
logically related to the risk and associated models, and
that means, quite frankly, that if I don't have a logical
relation between whatever indication I'm measuring and how
that impacts risk, it's not a risk-based performance
indicator.
Now, it may be some other kind of performance
indicator, but it's not going to be a risk-based performance
indicator, and it's not part of the ones we're going to be
working on developing in this process.
DR. APOSTOLAKIS: Why did you feel that you had to
add the word "and associated models"?
MR. MAYS: I think we had to do that to make it
clear to people that there had to be a link between what you
were thinking about indicating and how that would reflect on
risk, and the reason for that is another point that I was
going to bring up in the next slide, but I'll bring it up
now, and that is, risk-based performance indicators are not
something we're doing just for the sake of having risk-based
performance indicators.
They're things to be done in the context of a
regulatory process, and the regulatory process for which
they're supposed to be incorporated is the revised reactor
oversight process.
Part of that process has concepts in there about
being able to measure changes in performance that are
related to risk in certain, basically, decades of risk
contribution, and so, it was important to talk about the
models for risk, as well, in order to be able to go from
here to something that's part of the oversight process,
which is how do you establish thresholds.
So, that was the reason for choosing those words
that way, and that's why thought it was important to make
sure people understood that. That's why we went to the more
global definition.
DR. WALLIS: So, the indicators represent changes?
MR. MAYS: That's right.
DR. WALLIS: The indicators are a measure of
change?
MR. MAYS: That's correct.
DR. APOSTOLAKIS: Reflect change.
MR. MAYS: They reflect changes.
DR. WALLIS: But you actually plot something, and
then you have to take the slope of it. You're not going to
plot the change itself as the indicator?
MR. MAYS: Correct. We will plot performance
that, if it does change, we will be able to detect the
change.
DR. WALLIS: The indicator isn't itself a measure
of change.
MR. MAYS: That's correct.
DR. KRESS: The word "logically" intrigues me.
Did you use that purposely instead of "quantitatively"?
MR. MAYS: Yes. In fact, we used that instead of
"intuitively."
I think risk-informed thinking is usually
intuitive, in addition to being sometimes logical, but not
always logical, and we want to say this will always be
logical, and that's the difference between purely
risk-informed, where I would draw the line, if you will, and
risk-based.
DR. WALLIS: Logically would really mean
mathematically, quantitatively, rather than by word
argument.
MR. MAYS: Yes, and I think he's going to show a
chart that tells you what we're talking about here, and
performance in this context was -- we wanted to indicate,
also, when we say what kind of performance we're measuring,
we're measuring performances of all aspects of plant
operation, as they relate to meeting the cornerstone
objectives.
So, again, this is within a context of the revised
reactor oversight process.
MR. BONACA: So, this really includes almost a
concept of leading indicator.
MR. BARTON: Oh, yeah.
MR. BONACA: Right? I mean because you have
indirect -- I mean you have areas of activities that you're
measuring, and it should give you a lead indication, a
leading indication, is something integrated.
MR. MAYS: That's correct, and I think that's an
important point which I don't have in the slides here, and
I'll address it right now if I can.
Any time you're collecting data and doing analysis
of anything, that data and that analysis is lagging of the
particular thing you're measuring.
That doesn't make it a lagging indicator, because
what we're doing is we're looking at what it is that affects
public risk and what are the pieces that contribute to that.
So, if I have data and analysis and there's some
time period associated with, for example, diesel generator
reliability, then that indicator is lagging for diesel
generator reliability, but diesel generator reliability,
because of the logical model and association, is leading of
public risk.
MR. BARTON: Right.
MR. MAYS: And I think that's a very critical
thing that we've had some discussions that have gotten a
little confused about.
We're talking about disaggregating risk into
constituent parts for which, if we have indication at that
level, that would be leading of the indication of public
risk.
So, that's an important piece, too.
DR. APOSTOLAKIS: So, the first bullet, then,
should read -- I think what you said and what's written
there are a little different.
These performance -- risk-based performance
indicators are indicators that reflect changes, that are
capable of reflecting changes.
Can you put those words there?
MR. MAYS: Certainly.
DR. APOSTOLAKIS: In other words, you are not
defining what the indicator is; you are defining what the
risk-based indicator is.
I think that's more accurate than what you're
trying to do.
MR. MAYS: Okay.
So, getting into how we go about developing the
indicators -- I'm going to over this somewhat briefly now
and in a little more detail later to give you some examples
of where this tends to go.
So, we've got a -- basically a five-step process
of looking at that, and the first thing is to look at the
risk-significant key attributes under each of the
cornerstones of safety, and there's two figures that come
after here that I'll refer to as I go down this chart.
Let's put the key attributes chart up, Figure 1.
This is an example out of SECY 99-007 for the
mitigating system attribute, and you'll see that it is
broken up into key attributes, and the concept here is that,
if we are able to sample -- and I think it's an important
word -- sample performance of the plants under this
cornerstone in these areas and come to the conclusion that
the attribute areas are being satisfactorily maintained,
then we can come to the conclusion that the mitigating
systems cornerstone is being maintained.
MR. MAYS: So what we need to do in terms of
risk-based performance indicators is say which of these key
attributes has the largest risk contribution. We need to be
able to say that because it's fairly obvious that not all
these are going to have exactly the same risk contribution.
So, we want to say what are the most risk important
contributions under each one of these mitigating system
cornerstones, and that's the area we want to go look to see
if we can develop indicators.
DR. APOSTOLAKIS: So, are you going to develop
RBPI's for the most risk significant?
MR. MAYS: That's correct. We're going to look at
these areas. We're going to determine which were the most
risk significant and lay out our logic as to why. We're
going to look at see once we do that -- go back to the first
slide before that. Once we decide which key attribute areas
have the most risk significance, then we say okay, within
those, can we gather data and information to be able to put
together risk based performance indicators. Then we have
several other steps here about identifying the particular
indicators that are capable of detecting performance changes
in a timely manner and are capable of having thresholds
established in accordance with the concepts in 99007.
Now, anything that causes us not to be able to
fulfill one of those will be a reason why that particular
area is not a risk based performance indicator, but the
reason we're doing this is to make sure that as we go down
here, when we find key attribute areas that are risk
significant but for some reason can't -- don't have data,
well then you can make a decision. Do you want to go get
data or do you want to rely on risk informed inspection
activity to cover that part of the cornerstone. Or, if
you've got data but you don't have an indicator that
provides you timely performance, then that's another aspect
of how risk informed baseline and other inspections can be
brought into play.
So, we're trying to map out under the cornerstone
areas the risk significant attributes as well as where we
can get to risk based performance indicators because
whatever we don't cover with indicators, we have said we
were going to indicate for the inspection program what
aspects of that cornerstone are important so that those can
be emphasized there.
MR. BARANOWSKY: Let me just make sure that
there's not a misperception here. Go back to figure 1.
We're not going to have indicators individually for each of
these areas which we did not have when we went through the
framework the first time around, but we want to make sure
that the indicators reflect performance in each one of these
areas. The next figure that Steve's going to talk about are
the areas where the indicators are potentially capable, and
we have to see how all these attributes would map into those
indicators.
MR. MAYS: So, the next slide, figure 2 --
DR. APOSTOLAKIS: Wait, wait, wait, let's look at
this. If I could have a performance indicator at the system
level, why would I care about the lower level? I mean,
shouldn't there be some sort of a rule added to your slide 5
where you say RBPI's are developed by, that says that your
initial effort will be to define those as high in the risk
hierarchy as you can, and only if you can find it, for
example, in this case impossible to come up with something
meaningful at the system level or the train level, then you
go down to human performance and so on.
MR. MAYS: I think that's a good point, and we --
DR. APOSTOLAKIS: Well, you were doing that anyway
without stating.
MR. BARANOWSKY: I think that's a good point.
MR. MAYS: We were doing that anyway. I think
it's also important because we choose to say I'm going to
have a system level indicator, you have to be able to show
that the areas that are a part of the oversight process for
these areas like design and configuration control are
reflected in that indicator. If they're the kinds of
performance that wouldn't be reflected in there, then those
also have to be covered in some other way consistent with
the risk, and so that's why we were breaking it down in that
direction.
DR. APOSTOLAKIS: But I think it's important for
you to emphasize that because that will help also the
reactor oversight process. It help to reduce the baseline
inspection program. If you make a convincing case that this
particular indicator covers a lot of these other things,
then there's no reason for us to cover the baseline
inspection.
MR. MAYS: Well, we have to lay out that logic.
MR. BONACA: Well, you have to have a balance
there, right? I mean, a balance is how far do you want to
make it a living indicator?
MR. MAYS: Well, there will always be a baseline
inspection that either looks at areas that we don't have
good indicators for or that performs some sort of a V in V
function to make sure that the indicators are true.
DR. APOSTOLAKIS: Right.
MR. MAYS: They might be somewhat different in
terms of the inspection activities, but we will cover the
full thing with the inspection program.
Figure 2 is something that you've seen before in
various different configurations before, but this gives you,
again, what are the elements, the logical elements of risk
where an indicator of performance might be related to risk,
and you can see in this particular one we have things like
sequence level indicators which would be at the higher
level, system train and others. When I get to the specific
examples later, which you will see later in the package, you
will see that we looked at, for an example plant, that there
were some that we chose to use system level indicators and
some that we were looking at train and other things, so
you'll see some of that a little bit later one.
DR. APOSTOLAKIS: But again, here you may be
sending the wrong message. Maybe you want to work a little
bit on the right-hand side column there because that sends
the message that there will be indicators at each level, and
basically what we just said is that if you manage to get one
for the sequence level, the others are not needed unless
there is a special reason. I think that logic has to become
transparent.
MR. MAYS: Yeah, and that's true, but what we know
from our past experience is that we're going to have to go
down a fair amount in order to get responsive indicators.
DR. APOSTOLAKIS: Right.
MR. MAYS: We think we're going to be between the
two solid bars mostly.
DR. APOSTOLAKIS: Yes.
MR. MAYS: But we might be below that bar on the
bottom. Now, the reason these will be risk based is all the
information that goes in there is going to be folded through
some sort of a risk model as opposed to saying I'll count up
human errors and ten is too many, 14 is way, way too many.
We don't do stuff like that, okay, but it could be some
human error things or whatever where there is more data
density because it's more responsive. Then it could be
rolled up into higher and higher levels of information.
DR. APOSTOLAKIS: I have a question, it just
occurred to me, that relates also to the definition. There
seems to be, you know, out there near unanimous consensus
that having a learning organization is very important, and
the key thing there is to have feedback from operating
experience, both yours and other people's and use that in a
meaningful way.
Now, let's say you go to a plant, and there isn't
such a formal mechanism for learning, is that an indicator
of some sort, or in your definition, it does not?
MR. MAYS: I'm sure it's an indicator of some
sort. The question is, for our purposes, would it be a risk
based indicator inside the concepts of the revised reactor
oversight process. In order to make that connection, you'd
have to say failure to be a learning organization would be a
causal factor that would lead to something else, which would
then be related to risk. So, you would have to make that
kind of an argument and be able to show that what you were
measuring at the learning organization definition level, if
there was such a thing, had a cause and effect relationship
to get to something that you would be able to relate to
risk.
So, that's the difference between have an
intuitive thing that says, you know, if you have a good
staff you'll do good work and if you have a poor staff
you'll do poor work, to say what kind of a poor staff
constitutes and is logically related to equipment
performances or other things which we know will affect risk.
DR. APOSTOLAKIS: So things that may have an
impact on risk but have not been quantified would not
qualify for you to provide a performance indicator, a risk
based performance indicator?
MR. MAYS: That's correct.
DR. APOSTOLAKIS: The proper place would be
perhaps a baseline inspection or somewhere else. Okay,
that's a good clarification.
MR. BONACA: Okay, but I thought that however,
it's clear at the performance level if you had a number of
situations where you have a train level indicator failure or
branch failure and then you have another basic event level
indicator and the system level, and all three correlate to
below the line on failure over the experience, then at the
specific level, I guess the inspection process would throw
out that conclusion.
MR. MAYS: That's a good point because part of
what you're talking about is basically causal factors.
MR. BONACA: Right, exactly.
MR. MAYS: So, if the causal factors is a learning
organization, we do have in the oversight process inspection
activity aimed at problem identification and corrective
action programs. So, if you were to see that not being a
learning organization, for example, was causing you to have
problems in these areas and you had a problem in your diesel
generators and a problem in your HPI system and a problem in
your such and such, and those all had the same common cause,
that would be a basis for part of, Pat said, verification
and validation, and say do I have a corrective action
program that's capable of being identified and corrected in
these programs because that's what I have to have in order
to be confident that these PI's are telling me what they're
supposed to be telling me.
Also, the residents and the region folks would
then be able to say, based on this information, these are
the areas I need to inspect to make sure that that
supporting feature is being maintained.
DR. APOSTOLAKIS: In figure 2, wouldn't it be more
consistent with what you said earlier, to replace the tier
labeled sequence level indicator by the cornerstones?
MR. MAYS: I don't think it's quite that clean
because if you look underneath there in the sequence level
indicators, that gets input from both initiating events and
system reliabilities, and in the oversight process,
initiating events is its own separate cornerstone from
mitigating systems. So, they start to mix together in a
risk model a little bit differently than they're broken out
in the oversight process, so I'm not sure that that would be
--
DR. APOSTOLAKIS: Well, but you said that you
wanted to support the revised reactor oversight process, and
that's what they do. That's what you should do, too, I
mean, provide them with indicators in the initiating event
cornerstone because now -- I mean, they have four, I think.
In addition to the two you have there, initiating events and
system reliability, availability --
MR. MAYS: It's seven, actually.
DR. APOSTOLAKIS: -- but you expanded this a
little bit. Anyway, they also have the integrity of the --
MR. MAYS: Containment.
DR. APOSTOLAKIS: -- pressure boundary.
MR. MAYS: Barrier integrity, and they have
emergency preparedness and security and safeguards and --
DR. APOSTOLAKIS: I don't know that you want to
get into that, but the boundary, for example, might be
important. I mean, you know that you're not going to get
any indicators of the health effects level. I mean, you
know that the core damage frequency, although that might be
an indicator. Maybe that will be your global indicator.
MR. MAYS: When we talk about the integrated
indicator, that's the reason that was put at that level.
DR. APOSTOLAKIS: That was a secret in the plan.
MR. MAYS: Now, this is spelled out very clearly.
DR. APOSTOLAKIS: I don't think it is. I don't
think it is.
MR. MAYS: The integrated indicator talked about
having indications of core damage frequency which involved
two cornerstones rather than one, so that was why it wasn't
broken out separately.
DR. APOSTOLAKIS: I think you should think about
it.
MR. MAYS: Okay.
DR. APOSTOLAKIS: Maybe the cornerstone should be
there, and then the other stuff you want to do should be in
a separate figure because now we can start arguing why do
you have the containment failure, health effects, and core
damage frequency on the same level when first you have to
have core damage and then containment and so on. But also,
you said that you want to support the revised process, so
that's what they do. That's what you should do, too, but
then you go beyond that and integrate, which is a good
point.
MR. MAYS: Okay. The next --
DR. APOSTOLAKIS: No, wait. Now, if I go to the
very bottom, according to what you said earlier, you will
not have performance indicators for QA, right? You will not
have performance indicators for safety culture because these
things are not quantified in the PRA.
MR. MAYS: No, that's not what we said.
DR. APOSTOLAKIS: I thought that's what --
MR. MAYS: What we said was you had to have a
logical connection to the risk in associated models. I
didn't say you had to have a detailed model of QA or that
relates in the PRA. It says I have to make a logical
connection between what I might measure in QA and how I
might put that into a risk model, so I have to have a
logical connection there that people will agree to that yes,
this thing measures QA and those changes in QA can be
expected to be manifested in these areas of the risk model.
DR. APOSTOLAKIS: But you also said, Steve, that
things that are not in the PRA you will not develop
performance indicators for --
MR. MAYS: No, I think what I --
DR. APOSTOLAKIS: -- and QA is not.
MR. MAYS: What I -- I didn't --
DR. APOSTOLAKIS: You're taking it back?
MR. MAYS: No, that's not what --
DR. APOSTOLAKIS: Because we can do that.
MR. BARANOWSKY: The only thing we said, it has to
be logically connected to risk and the models.
MR. MAYS: Right.
MR. BARANOWSKY: We didn't say it has to be an
existing model.
MR. MAYS: Right.
MR. BARANOWSKY: We have the right to logically
connect it as long as peer review says that's a reasonable
thing to do. Now, what we're not going to do is figure out
advancing the state of the art in PRA and how to relay QA
actions to equipment performance, but if we can figure out
what kind of information to collect and it looks practical
and useful on QA or anything else and relate it through
these models, it would probably be cause/effect kinds of
things.
DR. APOSTOLAKIS: So, if I could make an argument
that the learning organization effects positively the human
performance of a plan, then you would say yeah, maybe we can
find the performance indicator for it.
MR. BARANOWSKY: Yeah, in fact, usually the way we
do that, we get knowledgeable people together and we say
well, how does that happen? They'll say well, this doesn't
work and that causes that to happen. So, we start to work a
progression and therefore a model.
DR. APOSTOLAKIS: Well, then I misunderstood.
MR. MAYS: Okay. The next slide talks about how
these fit in the reactor oversight process. I'm not going
to go over all the details on those right now. This is
really basically a what should risk based PI's be able to do
kind of slide. It's a test when you get down to the end to
say am I able to do these things with this risk based PI as
an intended consequence.
DR. POWERS: Could you explain better to me the
second bullet?
DR. APOSTOLAKIS: I was about to do that.
MR. MAYS: Risk based PI's should cover all modes
of operation.
DR. POWERS: Does that mean each has to cover
everything or that you should have a set that covers
everything?
MR. MAYS: Again, going back to the first concept
of sample, when I say covers all modes, it means in each
mode of plant operations, we should try to have some risk
based PI's that covers that particular area. So, we're
going to look at shutdown. We're going to look at fires or
external events, so those are the kinds of things we're
going to be looking at to see where we can put together risk
based PI's. We're not excluding --
DR. POWERS: So what you're saying is that you
have to have a set that covers all modes of operation, not
that each one covers?
MR. BARANOWSKY: Right.
MR. MAYS: That's correct.
DR. POWERS: I'll be fascinated to hear how you're
going to find -- well, I'm adequately informed. I know from
the past --
MR. MAYS: And so are we.
DR. POWERS: -- that you have superior shutdown
models that everything we need. So, hey.
DR. APOSTOLAKIS: Hey, go back a screen. Can you
elaborate a little bit on the last bullet?
MR. MAYS: Okay. Risk based PI should be minimal.
Establish plant specific thresholds consistent with revised
reactor oversight process. Again, as I said before, we're
not doing risk based PI's just for the sake in doing them.
We're doing them to be able to be part of a regulatory
process which is the revised reactor oversight process. We
want to be able to put together to the extent possible plant
specific thresholds that reflect the risk at that particular
plant for that particular performance change, and to do
those consistent with the -- right now the decades of risk
color scheme that's in the green, white, yellow, red scheme
that's in the revised reactor oversight process.
If we are unable to do that, then that would make
that not a candidate for a risk based performance indicator.
As you'll see when we get down in here later, one of the
things we're looking to do in the industry-wide performance
measures is for things for which we can't do plant specific
performance indicators. Perhaps we can do industry-wide or
group trends, which is another piece of the feedback
mechanism into the oversight process.
MR. BARANOWSKY: Well, I think there's one more
aspect to that, and that is if we have a sort of a global
indicator that is not capable of reflecting significant
differences in plant risk, then we would take that indicator
and make it into two or three slightly different ones that
can reflect plant differences in order to put the risk
context proper for different plant designs. So, the simple
example we always use is two, three or four diesel
generators, how do you want to treat those, especially if
you were at the system level. You would have different
system reliability parameters that would be acceptable in
the different color schemes that we have for the thresholds.
MR. MAYS: Next, the benefits we hope to bring the
revised reactor oversight process with this effort is to
first get reliability indicators at the component system and
train level. That will do two things. One, it will be a
greater coverage of risk perspective in each of the
cornerstones than we currently have, and because of both the
breadth of coverage of those and the nature of them, some of
them being at the component level, would give us more
indication of cross-cutting issue performance. Those are
two things that have been discussed with the ACRS, at the
meetings with the industry and public interest groups.
We're trying to scratch that itch by giving us a little more
capability in that area.
We're also looking to provide shutdown and fire
events. Again, the key words in the bullet is consistent
with the available models, data, and methods. What we're
not going to be doing in this particular program is going
out and defining the definitive shutdown risk model for how
to model risk of shutdown at all the plants. We're not
going to be doing that in this program. We're going to take
the insights and the information from existing risk models
as our basis for saying what's the important areas and what
areas should we be looking for, and develop indicators from
that.
DR. POWERS: Am I understanding the shutdown
models, we've got everything we need?
MR. BARANOWSKY: No, what he's saying is, in fact
--
DR. POWERS: I heard management talked to the
Commission, said that they have all the tools they need in
this area.
MR. MAYS: I can't comment on that specifically
because one, I wasn't there and two, I didn't make the
comment, but I do think there is significant discussion
within the agency and within the commission about what the
shutdown risk levels are and what level of information you
need to --modeling you need to have to understand them and
deal with them in a regulatory atmosphere. The point I
wanted to make here is that what we're not going to be doing
is going out and getting the definitive shutdown risk
models. We're going to take whatever is extant and try to
use that in a logical way to extent that we have models and
information. So, we're not going to be developing the new
models. We'll be developing risk based PI's consistent with
our current understanding.
MR. BARANOWSKY: Including our understanding of
what their limitations are. That's the part that I think is
important. So, if after we go through this and have a
review of the shutdown models, we determine that they have
significant limitations, then that has to be addressed
through other elements at a revised reactor oversight
process, namely inspections.
DR. POWERS: Yes, as far as I know, that must be
great. There's certainly no reason to pursue this any
further --
MR. MAYS: That's good to hear.
DR. POWERS: As I understand it.
MR. MAYS: That's good to hear.
DR. POWERS: That was news to me.
MR. MAYS: Okay. The other thing which was an
ACRS concern that was raised and other people raised the
concern, was about the thresholds and the current reactor
oversight process not being plant specific. We're going to
endeavor to make plant specific thresholds, and as we cited
in the oversight paper, we're not going to be making plant
specific thresholds just for the sake of doing it. If you
have 10 or 15 indicators at every plant and you have 103
plants, you can quickly figure out how many thresholds you
have, especially if you have four bands.
So, we're not interested in putting together 5,000
new thresholds, all of them unique and individual. That
just doesn't make much sense. So, what we're trying to do
is make thresholds that are consistent with how much risk
contribution those things make and again, we gave the
example before about plants with two diesels versus plants
with four diesels, things of that nature.
DR. APOSTOLAKIS: But the question of whether you
have to do it is really an open one. I mean, if you develop
a process here that, you know, after the review and
everything people say well, this is reasonable and then you
come back and say well, gee, you know, I can't develop 5,000
of these numbers. I'll do the best I can. Maybe I'll group
the plans, I'll do this, but you, Mr. Licensee, if you want
to do better, take my method and come back and give me more
specific numbers.
MR. BARANOWSKY: Yes, but I don't think people
will want to do that.
DR. APOSTOLAKIS: Well, you never know. You never
know.
MR. BARANOWSKY: Well, that's why we're going to
have a lot of interaction over the summer.
DR. APOSTOLAKIS: If there is benefit to them
doing it, they will do it.
MR. BARANOWSKY: Yeah. We're not really closing
the door here, but our intent is not to have people nervous
about having this broken down so finely that it takes an
army of people to manage it.
DR. APOSTOLAKIS: Sure.
MR. MAYS: As we alluded to earlier, we're going
to try to put together an integrated indicator to identify
changes in multiple performance areas. One of the things
that the ACRS raised was that the thresholds as they're
current set are based on that one parameter changing and
everything else being in its nominal value. We think
there's some benefit to putting together a model that would
allow us to be able to reflect changes in multiple
parameters as they occur. So, that's part of our phase 2
work, but that's one of the concepts we wanted to have on
here, and that was the reason for the sequence level
indication on the previous chart.
DR. APOSTOLAKIS: Which is really the great
strength of a PRA. I mean, that's what you're saying, let's
go back to the PRA.
MR. MAYS: Well, again, the example I used back in
August of '95 with the Commission was what if your diesel
generator reliability is not as good as it was before in a
BWR, but your loss of off-site power frequency has gone down
and your RCSI and your HPSI are very reliable and you've
increased your battery capacity from four hours to eight
hours. Well, the fact that your diesel generator
reliability went down may not be that significant in light
of those things, or, if it went down and all those other
things were going down, too, well then now you have a bigger
problem than you see with any one of them individually, and
that was the reason for wanting to put together something in
an integrated fashion.
The last one on here, the trending and risk
significant performance, this is an important piece not only
because it provides feedback mechanism and general
perspective for the oversight process. We also have a
strategic plan to report to Congress on our performance and
monitoring and one of which has no statistically significant
adverse industry trends and safety performance. So, this is
intended to also satisfy those kinds of concerns.
DR. APOSTOLAKIS: There are two methodological
issues I want to raise, and I see that we are really talking
about the value and so on later, so maybe this is just as
good a place as any
One is a question that was raised by this
Committee way back when we were talking about -- before
1174. The question is how much analysis or how many -- what
kind -- how much processing of the numbers, the raw data
that you are getting from the plant would you be allowed to
do before you produce a number for the performance
indicator? An extreme is to say well, gee, my performance
indicator is the core damage frequency, and the answer to
that is no, it can't be because there are so many
calculations you have to do from the plant specific data and
so many judgments that it is useless as an indicator. It's
perhaps useful -- not perhaps, it is useful as an integrator
but you don't call that a performance
indicator in this sense.
The other extreme is what we do in the
maintenance. We calculate our -- I mean, we measure hours,
we count hours. It's up or down and then we divide, you
know, divide the number of hours it was unavailable by the
bottom, a very simple mathematic calculation. Now, in
between there is a lot of stuff. How far would you allow
people to go, or in your world, would you be willing to go?
Is it only addition, subtraction, division, and
multiplication that would be allowed? Would you say well,
gee, you know, this is a simple model but everybody accepts
it, so the indicator should be the output of the model, you
know, given certain inputs from the plant? I don't know,
but this is a practical question, I think, that will come up
time and time again.
MR. MAYS: It is a practical question, and we're
working on that right now. That's one of the reasons, as we
stated earlier, for wanting to lay this logic out. The
logic is we want to have these things related to risk at a
level for which we can get data and a level for which we can
detect performance changes in a timely manner and at a level
in which we can produce thresholds that are consistent with
the oversight process. Those criteria will dictate to us
what kinds of methods we have to use.
Now, we've already in previous discussions, we've
shown the ACRS and the Commission and the world the kinds of
things we felt were necessary to come up with realistic
estimations of reliabilities, of trains and things in our
system studies and our component studies. I envision that
we will use similar techniques in the risk based performance
indicators, and that means there will be some analysis of
this information in order to be able to come up with
meaningful results. When I say analysis, I mean mostly
likely some sort of Bayesian updating process. That's why
it's so important to us to get the data from the industry,
in a larger sense, so you have a basis for how you're
constructing priors and how you're doing updates.
Now, once you have that kind of information on the
table and people can see the efficacy of the indicators by
their ability to meet these goals, then the actual
calculational things is really straightforward after that in
terms of once we agree that we're going to use a prior with
this much data in it calculated this way and an update
period of X numbers of years with these kinds of failures
and demands, the mathematics after that is fairly
straightforward. The real issue is gathering together
mechanisms to do analysis that meet these objectives, and
once we meet the objectives, whether we do it or whether
utilities do it, that's an open question, but we're not
intending to make it so complicated that only five PRA gurus
in a smoke-filled room can do this. We're trying to make it
so that it has some transparency --
MR. BONACA: But it seems to me from where you're
going, I mean, and what we heard in December, is that the
EPIX database will give you really information literally at
the maintenance level for all those components. So, the key
issue is, is the industry going to be conscientious in
putting all this information, I mean, because I understand
this is a voluntary reporting system. So, that's an issue
we need to discuss at some point. If you're going to rely
on that information, I have to make some other judgments
here.
MR. MAYS: Right, I agree with you, but the
question I think that George was asking was about what kind
of mechanisms do you assemble that data. There are two
issues. One is that you have to have data that gives you
good information and is part of V and V. You can verify it
that it's giving you the right information that's
appropriate for giving you the indication you're trying to
calculate. I think George's question was what kind of
methods do you use for calculating the indicator, something
really simple, something a little more complex, and my
answer to that is it really doesn't matter to me whether
it's simple or a little more complex, so long as it meets
the objectives. Now, obviously if you can use a simple
means and meet the objectives as opposed to a complex means,
you go with the simple one.
DR. APOSTOLAKIS: Yes, but if it's complex, you
may not have wide acceptance.
MR. MAYS: That's a potential thing we have to
deal with, and part of what we're going to do is put
together of what our concept is of what it takes to do these
indicators to meet these goals, have interactions with you
and with industry and public citizens and other people and
say all right, this is what we think the indicators ought to
look like and how they have to be done and how they ought to
be communicated, and we'll get feedback from them and we'll
find out where we go from there.
MR. BONACA: That's why I made that comment,
however, because I wanted to say that in the spectrum that
George was pointing out to from CDF to maintenance rule,
supposedly you're getting a database that would give you --
you could go all the way to the maintenance rule criteria
for some specific important piece of equipment. I'm saying
that you have a lot of flexibility there.
MR. BARANOWSKY: Well, that's going down -- you
could do that. We could go to a really low level of
information and by knowing its relationship to risk, we
could say, we could bean count things if you want. I don't
think we want to go to that level because then you're going
to have a very custom approach for bean counting at every
plant, and it will not be clear unless you run it through
the models, of what the implications are of going over the
bean count two or three dots. So, we have to figure
something out in between for a balance to be struck.
DR. APOSTOLAKIS: If we go to the proceeding
slide, there is an example, I think, of what we're talking
about. The bottom bullet -- yeah -- says that you will look
at common gross failures, right, the very last two lines,
slide 9.
MR. HAMCEHEE: Trending of risk significant.
MR. MAYS: Oh, okay.
DR. APOSTOLAKIS: Now, as you know very well, what
is a common cause of failure or potential common cause of
failure is not always agreed upon. We have to make
judgments. You know, you caught it in time and it could
have failed the other component and so on. So, I was
wondering how a performance indicator can be created for
that kind of event when there is judgment all over the
place, disagreement and so on.
MR. MAYS: Well, I think for the common cause
failure example, I think we've done enough work in the
common cause failure area and gathered enough data to pretty
well conclude that it is unlikely that common cause failure
would be an individual plant specific indicator. So, what
we're talking here in the trending is we have a pretty well
defined definition of common cause failure and how you code
it as part of the common cause failure database work that
we've been doing, and we've put together trends in that, and
we'll show you some of those a couple of slides later.
So, the issue here is can we give some indication
on the industry level about where something has happened,
and the definition of what we mean by common cause failure,
I agree, it has to be clear and has to be understood so that
the insight one would draw from that trend would be a
correct insight as to what the safety program is doing or
not doing. In all these cases, the definitions of what
these particular indicators are and what the trending
indications are have to be clear so that people will be
clear as to what inferences to draw from them.
DR. POWERS: I think that's one of the issues of
trending, is drawing the right inference, is a very
important one. I see lots of plots of pieces of industry
data in which somebody's put a straight to -- fit it to a
straight line and even put some error bounds around it,
confidence bounds around that. When I look at the data
underlying it, it appears to me that there are maybe
processes involved, one of which clearly has this downward
trend that they're proud of, but there seems to be one that
has -- it's not a straight line. Maybe it's a cyclical
behavior in there. Are you looking at more complicated
trending, more time series analysis than just the straight
line sort of things? I believe you even have an example of
some trends in here and which did show the five year cycle.
MR. BARANOWSKY: Yeah, I think most of the work
that we're doing on trending is not looking for anything too
sophisticated, but the key to doing it correctly is to have
the up-front definitions of what counts in the data that
goes into the trends so that one can then go back and given,
the trend, ask yourself does it have a statistical
significance and does it make sense from an engineering
point of view. I mean, we do that on every single trend
activity that we do. Those answers have to make sense to us
together.
So, the important thing for us in trending is not
whether it has some interesting dips and dives to it because
we're not trying to look at really fast moving changes in
these things normally.
DR. POWERS: Well, what if the dips and dives in
it is indicative of another mechanism? I mean, yes, for the
last ten years you've had a downward trend, but you have
this other mechanism going on and once you get down a little
bit, suddenly that becomes the dominant trend, and so you
have these cycles.
MR. BARANOWSKY: Well, I think you can see the
cycles in some of the plots, and that's when we go into the
causal factors, if you will, that are behind the ups and
downs as opposed to doing a statistical analysis, and so
maybe we're saying the same thing.
DR. POWERS: Maybe we're saying the same thing.
MR. BARANOWSKY: We're trying to analyze the data,
not just mathematically but also, you know, functionally, if
you will.
DR. POWERS: Well, I mean, there are some criteria
that these people are -- to do things like time series
analysis on the stock have for when the dips and dives
actually count and when they're just random variations.
MR. BARANOWSKY: Right.
DR. POWERS: They do more sophisticated than
straight lines.
MR. BARANOWSKY: Right. There are statistical
techniques like doing the fast Fourier transform, finding
the underlying frequency of changes, doing the filter and
see how much that for other -- we're not doing those kinds
of applications in here. We haven't seen that kind of
fluctuations. The one example I would give with respect to
what you were talking about, when we did the ASP results, we
shared with the Commission as well as with the ACRS, and we
bend the frequency of ASP precursors according to their
orders of magnitudes -- you know, 10 to the minus 6 is
fives, fours. We found, you know, decreasing trends on all
of those except for the ones greater than 10 to the minus 3.
We saw that case where we didn't have a significant trend,
where we saw it would be up one year and none for a few
years and then up for a year.
What we did is we went back and looked at that and
said we can't fit a trend line to this particular area, and
furthermore, we went back and looked at what the
contributions were and we found that in the ASP report that
the 10 to the minus 3 type ASP precursors were occurring for
unrelated reasons. There were no common threads between
them, and that gave us the idea that maybe this is just the
random level of occurrence that that particular thing
happens because there were no particular commonalities among
those.
So, I mean, that's the kind of things we do but we
haven't found the need to go into that level of
sophistication on trends.
DR. APOSTOLAKIS: Speaking of that, that leads me
to the second point I wanted to make. I think you have to
develop some way of discriminating between random aleatory
occurrences of something and true changes in the
unavailability or whatever quantities of the epistemic type
because that's really what you want to know. You don't want
to know that for some random thing this unavailability goes
up for a short period of time. Now again, in quality
control, they have ways of handling that, and I think you
ought to spend some time thinking about doing something
similar or analogous to make sure that what you are -- the
changes you refer to in the definition were real changes and
not just something random.
MR. MAYS: Right.
DR. POWERS: I think that -- I think quality
control may not be the right discipline. I think it's
really time series analysis.
DR. APOSTOLAKIS: Okay, the mathematic -- well, in
quality control, they worry about that.
MR. MAYS: right.
DR. APOSTOLAKIS: You know, did they see three
defectives because of some random thing or is it and
underlaying cause?
MR. MAYS: Some systemic cause.
DR. APOSTOLAKIS: Yeah, systemic cause, that kind
of thing. Now, whether there are more sophisticated
methods, sure there are, but I think you have to address
that issue.
MR. MAYS: Well, that's an issue that's going to
come up when we talk about setting performance thresholds,
and one of the goals we had in the white paper was about the
false negative, false positive rate, which is somewhat along
the same lines as when you see variations in your quality
control, is it random or systemic.
DR. APOSTOLAKIS: Right --
MR. MAYS: Those are issues that deal with the
fact that the kinds of things we're going to be measuring
here are quantities that have some uncertainty associated
with them. Whenever you have a quantity with some
uncertainty and you set some sort of bright line limit,
you're always going to have that kind of a problem. I think
the approach taken in the oversight program is to take a
graded approach to response to each one of those areas so
that you don't have to be that terribly precise at, say, the
green, white interface. What you have to be precise about
is the fact that you're not indicating green when it's
really red or that you're not indicating red when it's
really green. That's more along a how precise do you need
to be kind of argument, and we'll talk about that in a
little bit.
DR. APOSTOLAKIS: Okay, so this is something
you're going to address?
MR. MAYS: We are thinking about that.
Let me put up, instead of the next slide, let me
go to the picture here because I think I want to talk --
DR. APOSTOLAKIS: How much time do we have? We
started a little late, and we're supposed to finished at
2:45.
DR. POWERS: Why don't you go ahead and take
another half hour?
MR. MAYS: Okay, let me go to figure 3 here and
we'll talk about some of the issues that are in some of the
intervening slides here. This picture was put together to
try to come up with a simple way to see where we are and how
this activity would fit in the reactor oversight process.
If you look at the bottom of the slide, you see there's a
lot of information that comes out of plant performance.
Some of that goes into the current reactor oversight PI's,
some of that goes into the information for the risk informed
baseline inspections. In the current process, information
going through the current PI's is compared to a threshold
value. If that change in performance is determined to be
significant, that information goes up into the action
matrix. If it's not, then it goes back into the plant's
corrective action program.
Similarly, with the inspection program, we have
inspection findings. Those inspection findings go through
the significance determination process. If they're found to
be significant, then those things go up into the action
matrix. If not, then they go back into the corrective
action programs. That's kind of the existing way that the
oversight process is designed to work.
DR. APOSTOLAKIS: This is your figure or theirs?
MR. MAYS: This is ours. So, what we're trying to
show in the bold areas is where the new things in part of
the risk based indicator program will potentially impact and
interact with that process. So, the first thing you see at
the bottom right-hand side is data information in terms of
EPIX and the rad systems. We're getting more information
through the EPIX information than we currently had available
in the past with LER's alone. That's being put together in
the RAD system to combine LER, monthly operating port, other
information, which will be the basis for our calculations
for the risk-based PI's
What we're showing here is that in a similar way
to the current PI's, risk based PI's will be compared to a
threshold and then have the opportunity to go into
significance and the action matrix. Over time, we would
anticipate the risk-based PI's and the current PI's would
basically be merged into one set so that that complication
in the drawing would eventually go away.
Coming out of the risk based PI's, there are two
key things that we talked about a little bit. One was the
integrated performance indicator, which was to say what was
the overall effect of these things potentially happening
concurrently, and that would involved not only the
indicators but also information from the inspection programs
as well.
We also have a block to show the development of
the industry trends and insights, and the key thing here is
that it provides a feedback mechanism to go back into your
risk informed baseline inspection or into your reactive
inspections at areas where you don't have necessarily a
plant specific indications and say okay, are my loss of
feedwater frequency or my steam generator tube rupture
frequency or my other things that I know to be risk
significant on the whole going up, down or sideways and
therefore, do I need to affect the frequency or the nature
of the kinds of risk informed inspections I do as I sample
the performance in those parts of the cornerstone.
So, that's kind of our concept of what we're going
to be doing and where these things fit.
MR. BONACA: Before you go forth, however, again I
keep coming to that because when I look at this process, the
foundation to it is data.
MR. MAYS: That's correct.
MR. BONACA: Now, the LER's, you know, you have
plenty of LER's out there, but it's always been skewed
information because certain data do not get reported just
because of regulatory quality or aspects of the information.
If you have a single train failure in a multiple train
system, it doesn't get reported.
MR. MAYS: Right.
MR. BONACA: It's a failure that doesn't get
counted.
MR. MAYS: Right.
MR. BONACA: Now, the only question I have, now
the EPIX system is going to provide ten times more
information than the LER system. However, again, I go back
to the issue, how do you deal with the fact that some of
this information still is voluntary? Therefore, some people
will not report it, or they will choose to report what they
want to report.
MR. BARANOWSKY: Tell him about the INPO.
MR. MAYS: Yeah, I think there are two things on
that. When we say it's a voluntary reporting system, I
don't think it means I report it if I feel like it. The way
this has been set up is that the EPIX system was put
together and proposed by the industry to the Commission as
an alternative to the reliability/availability data rule.
There was a commitment made that there was information that
would be put together to support giving the agency
information to move forward in risk informed regulations,
and there are definitions and processes and QA things that
are being put in place in that system so that we'll be able
to tell whether we're getting the right information. That's
part of the validation and verification phase as well.
Now, in addition, there has been a task force put
together through NEI with the agency and INPO to look at on
a more global level what kind of data and information ought
to be presented to the agency, what's the most efficient
ways to get it, what is the information going to be used
for, those kinds of things, so that we have a consistent
understanding of what this information is, what it's used
for and what the basis for it is so that we will get quality
voluntary reporting.
Now, if we don't get -- if we find the plant is
not doing that information or isn't producing, we'll have to
make up for anything we can't get from a data standpoint
through inspection. So, that's kind of the regulatory
hammer, if you will, that if you don't want to provide
information, then you have to be subjected to more
inspection.
MR. BONACA: Okay, but it's still an open issue.
We discussed this in December, and you recognized that you
had a concern yet -- and there were ways to close that issue
but I think, you know, we'd like to hear about it, maybe
tomorrow morning because I mean, it's very important. This
EPIX system is the foundation of much of what you're doing
right now in this area.
MR. BARANOWSKY: Yeah, why don't I cover that
tomorrow morning because I can tell you about some of the
interactions that we're having.
MR. SEALE: In that regard, though, if you have a
case where you have an -- I shouldn't say insufficient -- an
incomplete reporting from the particular plant, it seems to
me that that's a caveat that you can put on the quality of
the answer that you get, and to make that work, if you do
have a lack of data from the plant, that ought to be part of
the record for that plant, that you don't have that data.
Therefore, my answer is incomplete. Treat it with a grain
of salt.
MR. BARANOWSKY: You have the same issue with the
current set of performance indicators. It's voluntarily
provided information, and there's an approach to dealing
with when that information is not being provided. We
wouldn't change that approach. It's just that there's some
additional information being provided for this particular
set of indicators, but the rest of it is the same. We're
not going to a new regulatory approach.
DR. APOSTOLAKIS: Now, your impact on the process
should be more than you're showing there. It seems to me
that as you define RBPI's, there should be an impact on the
baseline inspection. As we said earlier, I mean, as you
defined those and you say this is what's covered here, then
maybe these guys should eliminate some of the areas the
inspect.
MR. MAYS: Let me go to what we talk about on the
industry performance trend, which is on the next slides
where we give some indication on the next few slides about
what we would be doing with that. I mentioned before we
wanted to be able to have a mechanism for answering the
Congressional question of how we're performing about no
statistically significant adverse trends.
We're also looking at industry trends to be able
to help the agency put together a fairly succinct statement
to Congress and public and other stakeholders of what our
view of where reactor safety is and where it's going. This
is something that NRR intends to use as a piece of the
measures of how successful the reactor safety program is in
general.
The next to the last bullet talks about areas and
specific aspects that should be improved or have resources
redirected, including reactor safety research. This is the
feedback mechanism because the industry trends will include
not only the individual plant specific PI's but the other
aspects of performance that we're able to measure, and that
feedback mechanism is where it would go back into the
process and say therefore, these inspection activities can
be changed, either in their frequency or in the nature of
how they're being done.
On the next slide where we talked about how they
could be used, we talk about in the third bullet, updated
risk informed inspection guidance and supporting analysis of
regulatory effectiveness. This is the mechanism by which if
you have an indicator in a particular area that covers
things that you used to have to do inspection on, you could
change the way you were doing inspection.
The other piece that we wanted to do with this
industry trend stuff in addition to providing general
statements to the public is deal with the issue of public
confidence because this information would be being generated
consistent with the general philosophy that AEOD had when it
was a separate organization, which was to provide an
independent look at operating experience and feed that back
to the operating -- the regulator so that those kinds of
things could be taken into account, and there would be an
enhancement, we hope, in the public confidence that the --
there was a group who was going to say that this is good or
this is bad independently of what the individual program
office would be doing, so there's a certain level of
independence there.
The actually ones that we're looking at doing, and
these will evolve over time as the risk based PI's get done
and we are able to make other measurements and calculations
in a more timely manner, is that we would be looking at
industry or group averages from the current reactor
oversight process, risk based PI's as they come along, and
other complimentary things which are some of the things that
we've done in some of the system studies, event studies and
the ASP program in this branch.
DR. APOSTOLAKIS: Now, the ASP again is another
one that really cannot be a performance indicator.
MR. BARANOWSKY: But it can be an industry trend
indication.
DR. APOSTOLAKIS: Yes.
MR. MAYS: And that's what we're talking about
here. On the last page, the next page after that, was
examples of trends which you have seen before in various
reports. These are the kinds of things that we could give
indications of that as a set would be able to say this is
what we think about where performance is going.
MR. BARANOWSKY: And we would not just show
figures, of course, as we were discussing with Dana. We
would try to understand why things are changing and where
they are in time and what the factors are.
DR. POWERS: Yeah, if you look at the one in the
lower right-hand corner, my eyes aren't good enough to tell
you what it is, but you see you got a nice downward trend
there, but in fact, if you look at it in detail, it was a
five-year cycle going through that. It's got to be
something that's causing a five-year cycle in there because
I get three cycles there. That must indicate some other
mechanism. Now, it may be a risk insignificant mechanism
and maybe the way corporate planning is done or something
like that, but it would be more detailed than just a
rudimentary plotting.
MR. BARANOWSKY: Correct.
MR. BONACA: It may be a different accounting
system. There is something.
DR. POWERS: Yeah, it may be every five years, NRC
counts it --
MR. MAYS: Yeah, it can happen. In the interest
of time, what I want to do is go over slide 16 next, and
I'll skip a couple after that.
DR. APOSTOLAKIS: You know, just an administrative
suggestion. For slide 8, I would change the heading, and
instead of saying what are RPBI's, give it a fancier name
--the guiding principles for development, and then make that
part of the process. There you should also mention that you
would like to make it as high as possible.
MR. MAYS: Okay.
DR. APOSTOLAKIS: We don't need 16, you say?
MR. MAYS: No, we'll do 16. I may skip a couple
of other ones. What we intend to do here is we're issuing
the white paper for stakeholder comment. We'll brief you
and the Commission, and then we want to get some agreement
that this is the right philosophy to go off and develop
these indicators. Then the phase 1 report will come in
which will be the first example of trying that process out
and we would have discussions as to how well that went. We
go back to the ACRS again, back to the Commission, and do
the same thing on phase 2.
MR. BARANOWSKY: The only thing you didn't mention
and we didn't put it in here, is that there would be also
substantial interaction with the public, which we would
either have workshops or at least meetings to go over this
stuff and get comments on it after the July report is
issued.
DR. POWERS: In that regard, one of the members of
this Committee has articulated a position on what the public
is that resonates with me a great deal. He has suggested
that there's a technical community that is part of the
public and that, in fact, if you cannot persuade that
technical community in one discipline or another and in this
particular discipline, I'm not sure which technical
community you would identify, but I'm sure there is a
technical community outside the reactor area that
constitutes a public that you have to persuade. Maybe it's
the guys that do time series analysis on the stock market.
I don't know.
When you say we're going to have stakeholder
meetings and meet with the public, are you going to
prostelitize this thing to the appropriate technical
institutions that represents that body of expertise that if
you don't persuade them, you're never going to get --
persuade the average guy on the street that you're doing the
right thing.
MR. BARANOWSKY: I think we plan on having a
pretty wide interaction through a public -- through Federal
Register notices and through targeting --
DR. POWERS: You would be surprised how few people
read the Federal Register. I mean, it may come as a shock
to you, but I bet you it really doesn't have the kind of
circulation in the rest of the country that it does here in
Washington.
MR. BARANOWSKY: Oh, I thought everyone reads it
at 5:00.
DR. APOSTOLAKIS: Even if they read it, they won't
come.
MR. BARANOWSKY: Also, targeting other
organizations, whether they're reactor groups or other
technical groups, public interest groups and so forth, that
have either shown an interest or that as best we can tell,
would have a good input to this process. It's not going to
get everybody in the world involved, but it's going to get
--
MR WALLIS: One thing you can do, though, is
instead of just meeting with what appears to be a meeting
between you and industry to sort of hash things out, call a
public meeting, you can make an effort to present a paper to
some tentacle society where other people are there and
convince them and get feedback from them. Go out of your
way to broaden this public reaction.
MR. MAYS: We have gone more than just meetings
with the industry in what we're intending to do. The
concepts of these risk based PI's and other things have been
presented through technical meetings and in other journals
as we've come along before, so that part of it's been done.
The people that we are primarily focusing on are the people
who have been actively involved in the reactor oversight
process, which is not just the industry. At the latest
Commission briefing on the reactor oversight process, there
were presentations from Dr. Lipoti from the state of New
Jersey, who had sent in concerns. There were presentations
from David Lochbaum from UCS. There have been other
workshops and meetings in which James Riccio from Public
Citizen has been involved. There's been the public
performance-based regulatory workshop in which other people
from Public Citizen and other groups have come. We're
sending this out to them and trying to get those people
involved.
MR WALLIS: Those are not the groups that Dr.
Powers was talking about.
DR. POWERS: Somehow I'm just not matriculating it
very well, Graham. I think we've got to -- I mean, I love
this idea, and I believe it's absolutely true, that if we
cannot persuade these people in professional organizations
outside the nuclear industry that the nuclear industry is
doing a good job, we'll never persuade the man down the
street.
MR WALLIS: Or even you give a seminar to Dr.
Apostolakis' students at MIT and let them have a go at you.
MR. MAYS: I've done that for the last two years.
MR WALLIS: Good, I'm glad. That's great.
MR. MAYS: I'm scheduled to go back again this
year.
DR. POWERS: The students at MIT are way too
reticent. You need to go up to Dartmouth. Dartmouth is
where they're really, really tough.
DR. APOSTOLAKIS: Well, let me -- I think there's
an important practical issue here, Graham, that --
MR. MAYS: As I get older, I go further and
further south.
DR. POWERS: -- the technical communities that
you're talking about as a rule will not come to what we call
public meetings.
MR. MAYS: Of course not. Of course not.
DR. APOSTOLAKIS: So, I can see two ways to get
them. Either you hire them and pay them as they view us.
I'm sorry, a statistician will not come to a Commission
meeting just to listen and express his views. They've got
to pay him.
DR. KRESS: They might be part of a National
Academy study.
DR. APOSTOLAKIS: Or you go to where they are
already, like technical meetings, their journals and so on.
So, I think you have to think about that, what would be the
best way to reach out to these communities. It's a fact of
life that even if you bring the Federal Register notice to
them, they're going to say well, you know, that's
interesting. They're not going to go out of their way to
fly to Washington, right? So, I mean, have that in mind.
MR. BARANOWSKY: I think there are probably fewer
highly technical and mathematical issues that need to be
resolved here than there are philosophical things.
DR. APOSTOLAKIS: Yeah, but for example --
MR. BARANOWSKY: But we would be willing for those
things that should have a high level technical review to get
it, because I'm convinced, as you are, that good quality
work that's been accepted is one of the keys to moving
forward with these kinds of things, and that's been one of
our philosophies for years.
DR. POWERS: I think you'll find a receptive
audience among the political economists. I mean, they're in
the business of doing this kind of stuff. It might be an
interesting way to go.
MR. BARANOWSKY: Yes. We might learn something.
DR. POWERS: Yes, but you'll never want to go to
another political economics meeting, I can say that.
DR. APOSTOLAKIS: Are you presenting this piece on
this one, or it's too recent?
MR. BARANOWSKY: We presented -- well, the charts
that you saw, the clocks, that was a paper I presented on
industry-wide trends.
MR. MAYS: And I made a presentation at PSA '99 as
well.
DR. APOSTOLAKIS: How about this coming November?
Are you going to the meeting?
MR. MAYS: I don't have a paper for that meeting.
DR. APOSTOLAKIS: Oh. I think you have to skip a
few of these.
MR. BARANOWSKY: I think we do. Let's go to the
schedule 1 -- well, we've covered the schedule.
DR. APOSTOLAKIS: No, tell us a little bit of the
roll of the integrated indicator, because that's a
relatively new concept. I'd like to understand.
MR. MAYS: Actually, the integrated indicator, we
can probably, if you will allow us, skip a little bit
because that's really a phase two process that's not going
to be something you're going to see in July anyway.
Basically the concept would be there, we would use a plant
specific model such as the SPAR 3 models of the plants to
combine the information about the indicators and inspection
findings, see what the overall implications were on core
damage frequency as a result of those. That's our initial
concepts on that, and we're not going to have anything more
for you to bite into for awhile yet. So, unless there's
something else you want us to talk about on that, I'd like
to skip that.
DR. APOSTOLAKIS: Okay, fine.
MR. MAYS: I'd like to go to the example
information on slide 2 about some of the stuff that we've
done, to give you kind of a taste of what's coming up. This
is not going to be a comprehensive statement of everything
that's being done. The July report will have a more
comprehensive work, but you'll see the examples includes six
bullets here, five of which you've already seen before,
because those are the things about how we develop a
particular PI going from the key attributes, the risk
elements, and the other one, which is on the next slide,
which a general risk perspective.
So, on slide 22 --
DR. APOSTOLAKIS: I can't believe you did this
here.
MR. MAYS: Yes, you can believe it.
DR. APOSTOLAKIS: So I have to count zeros now,
right?
MR. MAYS: You have to count zeros. Now, George,
I showed you this at your class --
DR. APOSTOLAKIS: Are you aware of the scientific
notation --
MR. MAYS: I'm aware of the scientific notation,
and I specifically did this with zeros for a reason, and we
shared this with your class last summer, so I wanted to --
DR. APOSTOLAKIS: That was the reason.
DR. POWERS: Ooh, Jack.
MR. MAYS: I wish I could take credit for being
that clever, but I can't. The purpose of this slide was to
do some perspective things because one of the things I've
seen in the workshops and discussions with public and the
discussions with the Commission and ACRS is there's two
things that tend to happen. One is there's the forest for
the trees problem. People start worrying about whether
something can actually count and whether something's 2.95
times 10 to the minus 6 or 3.27 times 10 to the minus 6, so
I think there's a perspective issue that needs to be done
there. There's also a perspective from a broader, more
global regulatory framework that I'll want to bring to bear
here.
Now, in doing this particular slide, there's a
couple caveats I'm going to start out with in the first
place, and one is this is talking about individual accident
death risk. This is not risk from cancer. This is not land
interdiction. This is not the complete and total picture of
what risk is, but it's a piece of it.
What we did here is we went back and looked at the
background risk for an individual in the United States dying
from an accident is, and as you can see here, in scientific
notation, that would be five 10 to the minus 4 per year. I
put the zeros on there because of another reason. The NRC's
quantitative health objective is for the risk to an
individual living near a nuclear power plant dying from an
accident should be less than .1 percent of the normal
background risk.
I put that on there to make sure that people didn't think
that 5E minus 4 and 5E minus 7 was just three. It's not
just 3. It's three zeros, okay, because people get confused
when you start talking logorhythmic from where we really
are.
DR. APOSTOLAKIS: A great communicator you are.
MR. MAYS: So, the next piece that I think is
important is okay, let's suppose a plant had a 10 to the
minus 4 core damage frequency from all causes, whatever that
was. We then looked at what the corresponding risk would be
using the worst NUREG 1150 containment and offsite
consequences calculations. When you take that kind of
approach, what you find is the risk to the individual, and
that case would be two 10 to the minus --
MR. BARANOWSKY: Eight.
MR. MAYS: Eight, which is a factor of 25 less
than what the qualitative health objective would be.
Therefore, a risk of 10 to the minus 5 delta CDF would be an
order of magnitude less than that and 10 to the minus 6 an
order of magnitude below that. So, if you happen to live
near a nuclear power plant with a 10 to the minus 4 core
damage frequency and a containment and offsite
characteristics like the worst of the 1150 plants, then the
individual risk would be .00050002. I think that's
important to know how many zeros are between the two of
those from a perspective standpoint.
MR WALLIS: Well, that's false because the
uncertainty in the five swamps any of the two.
MR. BARANOWSKY: That's excellent because what
we're really saying is we have to make sure we don't lose
our heads over that two when the real issue is at the five.
MR WALLIS: But the five should be rounded of. It
should be five two.
DR. APOSTOLAKIS: No, but that also tells you that
you have to have uncertainty that is so broad that the three
zeros there --
MR. MAYS: Exactly.
MR. BARANOWSKY: The three zeros, exactly, and
this perspective needs to come to play in talking about the
precision of the mathematics for any of the stuff we're
talking about. That's why he's showing it, not because
we're going to try and develop indicators at this level, but
how precise do we need to be.
DR. APOSTOLAKIS: Has the Commission seen this
slide?
MR. MAYS: I don't know if the Commission has seen
it. I do know the EDO has seen it.
MR WALLIS: Several times.
MR. MAYS: Several times. As a matter of fact,
earlier once again this morning. The issue here --
MR. BARANOWSKY: We will show this to the
Commission when we go in June.
MR. MAYS: Yes.
MR. BARANOWSKY: We'll probably modify the
presentation because there are certain sensitivities,
particularly about polar perception on land interdiction and
so forth, but nonetheless, I think this gives some risk
perspective that when people talk about being risk informed,
I hardly ever see them talk about risk. The next thing --
so therefore --
MR WALLIS: This has been around for -- since Wash
1400 and before.
MR. BARANOWSKY: Yes, but for some reason, we
stopped talking about it for ten years. I think the
important point from a standpoint of the reactor oversight
process, if you look at the last three lines, the reactor
oversight process is set up so that changes in performance
that will result in a 10 to the minus 4 increase in the CDF
for a plant which is basically the red zone, which has been
defined as the unacceptable performance range, means that
you would go from .00050002 to .00050004. That's the red
zone because that would be an addition one 10 to the minus 4
to what's already 10 to the minus four. It would be
doubling it. So, the red zone is still three orders of
magnitude down from the existing risk. That means the next
level down with yellow and the next level down from white --
DR. POWERS: Excuse me. Do you really think it's
necessary to explain the scientific notation to the
committee?
MR. MAYS: But that's where the process interfaces
with that. That's what I wanted to make clear there.
When we go through this then, started looking at
how we were going to put together individual risk based
PI's. On slide 23, the -- what we're going to give you an
example here is for PWR and for internal events at full
power of the mitigating system cornerstones. So, this is
one slice of the entire process. The issue for us is how do
you pick up and identify what the significant attributes
were, and the approach we're going to use is we're going to
use insights from plant specific IPE's, ASP results, system
component and other studies to say which of these particular
attributes has the most risk significance.
On the next slide, we give you that attribute
slide again, and we bold, highlight in here the equipment
performance piece. It's a pretty good consensus that the
reliability and availability of mitigating systems is
important pieces to the risk. That's pretty simple to do.
The more difficult question is to go in and say okay, what
is it about design or procedure quality or human performance
that's risk significant if it changes from whatever its
baseline is, that we would need to capture, that isn't
reflected in reliability and availability of equipment.
DR. APOSTOLAKIS: Now, Steve, in this particular
issue, it seems to me that you need to explain a little bit
what you're doing on the figure. You're not really looking
at the equipment performance. You're looking at the
equipment performance under certain conditions.
MR. MAYS: That's correct.
DR. APOSTOLAKIS: Because later on, for example,
on slide 27, you separate out AC power, right? So, what
you're doing here is you're saying given that I have, for
example, AC power, due to other reasons, why is -- I would
have a performance indicator that would tell me what is the
reliability of this piece of equipment, given that certain
conditions exists, and for these conditions, I may have
something else. I have another performance indicator or put
them back into the baseline inspection program. That would
be consistent with the PRA, as you know very well. It comes
back to the idea, you know, what is the reliability of this
system. Well, the PRA's tell us that this is not a very
meaningful question because there are so many different
conditions under which that system may be demanded, and its
reliability will be different.
MR. MAYS: Right, and what we have to do is we
have to break the mitigating existing cornerstone up into
the particular areas for which things make sense to be
grouped together to see what the risk based PI's. That's
why I said starting out here, I'm talking about full power,
PWR, mitigating system, internal events only.
DR. APOSTOLAKIS: What I'm saying is make sure
there is somewhere in the figure here where these words
appear, that this is under these specified conditions.
MR. MAYS: That's correct.
DR. APOSTOLAKIS: Because slide 27 really is a
very important one.
MR. MAYS: And so we have to develop the
methodology and come back and show you why we think what
pieces are relevant for developing risk based PI's, and
that's part of our program.
On the next slide, we talked about we're going to
take the reliability block of that previous one, so we've
narrowed down a little bit further and said how do we
identify the risk significant system trains or components
with respect to the reliability attribute, and the approach
we're going to use is using a Fussell-Vesely risk
achievement worth features to look at the plant specific and
generic PRA's to see what particular things we can come up
with are the key systems that we need to look at. Then
we'll go down from the system to train or other levels in
accordance with the criteria that we had for what makes a
good indicator.
So, these are the ones we were looking at for this
particular PWR as the potential systems for which risk based
PI's might be developed.
DR. APOSTOLAKIS: Could you skip to 28?
MR. MAYS: Sure, we can do that. On 28, we talk
about, given that we've decided what level we're going to do
a particular indicator, or how are we going to do the
threshold determination. The approach we intend to use is
to use plant specific spar models and then go back and look
at what changes in the reliability model would produce
changes in the core damage frequency assessor at 10 to the
minus 6, 5 and 4. That's a little bit different from what
the current oversight process has, and the ACRS has raised a
concern about this in the past about why the green/white
interface was set at a 95 percentile of the average
performance.
There's lots of implications associated with that.
The first one is 95 percent may or may not be very risk
significant. The second one is if you're doing it on 95
percent of what the average is, as the performance improves,
what you've basically done is broaden the white zone as the
performance improves, unless you've set it at one particular
year and leave it there.
So, we want to address all those issues, and so
what we've done initially is put it on a risk basis because
this was risk based performance indicators, after all. We
believe we have the address, whether the 95 percent makes
any sense and why and what those are, and we need to lay
that case out.
Now, having said that, the 10 to the minus 6 one
also has some consistencies with respect to the significance
determination process, green/white interfaces, basically at
a 10 to the minus 6 delta CDF level as well. So, that helps
with some of the consistency issues.
So, going back to the last one, to give you an
example of what this might look like for this particular
plant under mitigating systems for internal events at power,
we have in the chart, for each one of these either systems
or trains, the baseline value, what the amount of change
would be necessary to get the delta CDF's as shown, and what
we're looking to see is is there a big enough spread
between, say, the green and the red to believe we have a
graded approach here, that we don't go from -- for example,
if we had the AFW went from 1.6 10 to the minus 4 nominal to
the red zone at 1.8 10 to the minus 4, you'd say that's a
pretty lousy indicator, and we wouldn't use that as an
indicator because there's no difference between red and
green. It's so small that it wouldn't make any sense.
So, we have to look to see if we have a spread in
those values and to do that. The case of that for this
particular plant is another example down here under power
operator relief valves. What you find at this plant is that
even if the PORV's are incapable of operating for the entire
year, are basically not there, you still can't make a delta
CDF with that alone, that 10 to the minus 4. You can
measure it at the lower levels, but not to that. So, this
helps us be able to figure out whether or not that's a good
indicator.
DR. APOSTOLAKIS: There's an important point here
that just occurred to me as you were speaking that I think
has to be made today, and then we'll start. When we -- I
don't know whether you read that ACRS letter where some
members said this and some members said that with regard to
the performance indicator and the threshold values. There
were two opposing viewpoints which come down to two
different views of the objectives of the reactor oversight
process.
In one, the plant is licensed, right. It's
allowed to operate. It has a certain risk profile. The
objective of the oversight process is to make sure that that
profile doesn't change significantly. The other is look at
the industry as a whole and do your oversight in a way that
the industry functions in an acceptable way.
You are really introducing a third view in your
slide 28, and maybe you can reflect a little bit on that.
You're saying with the second bullet yes, the plant is
licensed and so on, in slide 28, but all I care about is
really whether their CDF values, minus 4, 5 and 6, have been
exceeded. This is a third view, because you're not
referring now to the risk profile as it is now, and you know
you are interested in changes. You're saying fine, you can
even change, as long as these CDF values are not changed.
You have to think about it because we are running out of
time, but this is really an important point, and I don't
know whether this is the way we want to go.
MR. MAYS: Well, it was meant to be consistent
with 1.174.
DR. APOSTOLAKIS: You are making it risk based
now. Huh?
MR. MAYS: It was meant to be consistent with Reg
Guide 1.174.
DR. APOSTOLAKIS: I understand that.
MR. MAYS: Okay.
DR. APOSTOLAKIS: I understand that, but I'm tell
you, now three views of what the objectives of the process
are. All I'm saying is I want you to think about it before
July.
MR. BONACA: I'm not sure, however, because I
mean, there was a implication that consistency with Appendix
B requirements will give it a level of reliability. Now, it
was never measured, but this is the first time that I see an
interpretation in writing of what that would mean.
DR. APOSTOLAKIS: Well, that's a subject we
probably need to discuss in a separate meeting.
MR. BARANOWSKY: Our intent is to say that the
nominal performance should be acceptable, and that these
changes, 10 to the minus 6, 5 and 4, from that nominal
performance are the ones that we want to look at in terms --
DR. APOSTOLAKIS: I understand that. Okay, we'll
have to discuss it in another forum because this is a big
subject. I take it you don't have anything else that is
really important to tell the Committee.
MR. MAYS: No.
DR. APOSTOLAKIS: Any members have a question or
comment?
MR. BARANOWSKY: Good work. Thank you.
MR. MAYS: Thank you.
DR. APOSTOLAKIS: Shall we invite the public or
the staff? Okay, back to you, Mr. Chairman.
DR. POWERS: Fine. I will recess us until 20
after the hour.
[Recess.]
DR. POWERS: Let's come back into session. We're
turning to the area of the human performance program plan,
and our cognizant member -- I don't know if he's available,
but I can introduce this subject. Here's Jack.
MR. ROSENTHAL: Thank you. I wore a jacket just
as a sign of respect to the ACRS, but now that -- it's hot,
so I'll take it off.
DR. POWERS: As a sign of good sense, you'll now
take it off, right?
MR. ROSENTHAL: And I'll be making somewhat less
than 40 minutes of the presentation, and we should allow for
NRR, which is both the user of the information and it is an
agency or attempt to be an agency program.
I'm going to talk about the plan itself, and then
Dave Tremble and Dick Echinrode are going to talk about the
reactor oversight process and the relationship of this to
the oversight process. Then I'll take the microphone back
and talk about where we go in the future.
We have been talking, slide 3, about plans for
years, and you know, there was National Academy of Science
reviews way back, and you know, this is pre-TMI. Then the
most recent round are really from about '95 where all that
happens was that three branch chiefs -- I was in AEOD, and
Mary Lee Slossen, I think, we in NRR. Dick was involved,
and I think it was Brian Coffman in RES. All we wanted to
do was get together and see that the programs didn't
overlap.
Well, that turned into a big plan which was
published and rejected, and then more recently there was a
SECY in '98 which the ACRS commented about heavily. Then
there was a presentation in February, '99. I sat in the
audience and Steve Arndt made that presentation. We took
what you said seriously.
DR. POWERS: We all jumped up and applauded that
presentation.
MR. ROSENTHAL: Yes, you did. You liked that
presentation very much on being more risk informed, but Dr.
Seale also tells us that we better be talking to INPO and we
should not be duplicative of the INPO efforts. We were to
see what other federal agencies were doing, as well as risk
informing. I think that we have, in fact, told us to do, so
I'll be talking mostly about the plan.
The plan is the staff's report because we continue
to work on risk informing it. It includes a mission
statement I want to talk a little bit about the basis for
the program elements and future directions. Ideally, one
would have a perfect PRA's and be able to do risk
achievement works or Fussell-Vesely on each element of the
plan and have this absolutely logical coherent instruction.
Well, even within the scope of PRA, I can talk
about human error, but I can't -- it's not at a state of
knowledge where you can separate out training from
procedures, et cetera, but we still try to do a fair amount
of risk work. The reality is that RES develops tools and
methods for user offices like NRR and NMSS, so user needs
are very, very important to us. In the plan, you'll see
that there are several specific elements with a mark of
anticipated user needs. Sam Collins signed that user need,
too. So, at this point, about 90 percent of my program in
fiscal 2000/2001 is user need based and specific
contemporaneous user needs.
We also looked at what the industry is doing where
fellows at FAA and NASA and Coast Guard, et cetera, as a
reality check to see that we're worried about the same
things that they are, and did they have data that might be
useful to us, et cetera. We also want to do our work within
the context of broader agency programs.
Okay, so now -- we did a fair amount of risk
reviews. Let me say that on March 15, we had a 4-1/2 hour
session with the subcommittee in which we went over the
stuff in detail, and I'll try to summarize here. There's
two sources of risk information that we would use. One is
from PRA's that have been performed and PRA risk insights,
and from the accident sequence precursor data. These don't
give you conflicting, they give you complementary views of
reality, but these complementary views are somewhat
different.
Brookhaven wrote a letter report which we did
provide to the Committee in which there's not a table of
employment human factors. There's a table of studies of PRA
effort, PRA, that did sensitivity studies said that human
performance, in whatever level of modeling that they did,
was important.
The things that you typically see in a PRA are
things like going to bleed and feed on a pressurized water
reactor, coping with a steam generator tube rupture, on a
boiler pumping with at atlas event, control of slick,
control of water level, et cetera. These are rare events
which one does not observe, fortunately, in every day
practice in the plant.
If you want to make conclusions about them or look
-- and you thought that was the total set of what you should
be looking at, then we should be looking at the simulator
and EEO PE's and procedures and training, and INPO does that
very well. So, our effort would then be modest. I'm not
proposing that we do that because INPO does it.
If one looks at the ASP data, one gets a somewhat
different view of reality. What did was we looked at the
last five years of accident sequence precursor events. It's
roughly ten a year -- I'm sorry, 10, 20 a year, looked at
those which were greater than 10 to the minus 5.
Conditional core damage probability, there's about 50 of
those events, and said we'll study those roughly 50 events
to say what's important. This becomes important when I talk
about -- and I will later on, about looking at the plant
oversight process. This was the focus and what could we
learn from these events.
In discussion with Dana at the subcommittee and
again in preparation just a few minutes before the official
meeting, he properly brought up I don't have -- I can't say
that 50 events greater than 10 to the minus 5 is no good or
good. I know that we've had there very many events. I know
that the mission goal is no single event in a year greater
than 10 to the minus 3 as a written stated goal, performance
measure, but the fact that we've had 50 might or might not
be acceptable, and what you have to do is somehow match that
up against --
DR. APOSTOLAKIS: You mean 1E-5? 10E-5 Is 10 to
the minus 4.
MR. ROSENTHAL: I'm sorry, 1E-5. Okay, now I'm
embarrassed.
DR. POWERS: Jack, let me ask you a question. You
go through and you look at these events and you see what's
the human contribution to this, and I have a tendency to go
back to the old inspector mentality. There are only four
possibilities on any event. The design was inadequate.
That means a human failing to anticipate the requirements.
There was no procedure -- that's a human failing to
anticipate the need for a procedure. The procedure was
inadequate -- a human failing to prepare a good procedure.
The procedure was not followed properly -- clearly a human
failure. Everything is human failure if you go back far
enough. Okay, so what do you do differently when you look
at these?
MR. ROSENTHAL: We took out the lightning strike
as, you know, the event that -- where, you know, within
milliseconds the plant's reacted. We took out the designer,
like a pressure locking of MOV's. Ultimately it was some
engineer who didn't listen to all the stuff. What we did is
we looked at those in which it was the operator or the
operating organization and the operating organization's
processes or procedures. Let me give some -- let me just
move to the next slide, and then I'll continue talking.
Okay, the most significant event that we've had in
the last few years was Wolf Creek event. It was a shutdown
event, 10 minus 3 conditional core damage probability. That
is an event in which the operating organization chose to do
multiple evolutions at the same time and caused that event
to take place. That was event in which they were under
pressures to do the fastest refueling outage ever. I think
that that's a legitimate look at how human performance in
the broadest scale could affect events. That's an event in
which the organization both caused the event and the
operators quickly closed the valves and ameliorated the
event so that they can do good as well as bad.
I throw -- okay, another important event was at
Oconee where they burnt up two high pressure safety
injection pumps. They actually caused damage to two pumps
and would have damaged the third pump if not for the
operators being swift enough to stop the third pump before
it autostarted and burnt up, too. Underlying that event was
-- it's a balance problem, and that is that you test high
pressure safety injection pumps quarterly and they worked,
and in today's speak, they'd be green and they'd have
adequate reliability, et cetera, et cetera, et cetera, but
you didn't include in your test boundary the sensors on the
refueling water storage tanks. So, if the refueling water
storage tank level is too low, then the pumps cavitated.
I consider that a human performance of the
operating organization. So, that's the kinds of things that
we put in and in contrast to the design of the pressure
locking valve. Now, I guess if I was more intellectual, I
would go from the specific examples to the general, but I
don't know how to quite say it.
Okay, so we do this work, and it tells us,
depending on the event, that the contribution of the human
is very important, only in these 50 events -- I'm sorry, in
these events, I think that we only have six that involve
some aspect of the operators in the control room which is
the typical focus of the PRA. Then we have many, many, many
events involved the operating organization to include a lot
involving maintenance and the ratio of what I call latent
errors -- it's like four to one. What I mean by latent
errors is the Jim Reason sense of the definition of latent
error, this thing that has escaped your ISI, IST,
reliability studies, et cetera. It's something waiting
there to grab you.
That is not to say that this is not an acceptable
number for the industry as a whole over time.
DR. POWERS: What you're telling me --
MR. ROSENTHAL: And that's work to be done.
DR. POWERS: I mean, when we do a PRA, there is
certain unreliability due to maintenance failures. It's a
maintenance failure, not a human error probability built
into there, and that's what you're looking at when you look
at these things. You're looking at human a little more
broadly than you would in any risk achievement worth going
into analysis where I said the operator was either dead or
infallible, depending on which way I did it. You're looking
a little broader than that.
MR. ROSENTHAL: Yes, and now this affects the how
is human considered within the overall plant assessment
process, where if I base it solely on the PRA insights, then
I'm going to use the plant assessment process and look at
things like recall exams, and if I'm worrying about all
these latent errors, then I should look more broadly. Then
ultimately what I'm going to do is see if the plant
assessment process is or is not considering this.
DR. APOSTOLAKIS: The other very important reason
why you look for latent errors is that some of those may
introduce dependencies that we're not aware of. For
example, if the work prioritization process has some
problem, okay, that may result in a coupled failures of
equipment that are not even similar, which is a usual common
cause failure analysis in the PRA. Now, that doesn't mean
that if you look you're going to find that for sure, but
there is a suspicion that this may happen, and at this stage
of the game, what we're saying is let's try to understand it
a little better to make sure that these things will not
happen because that's the problem with latent errors. They
may not lead to a single active failure. They may actually
lead to a number of them at the time when you don't want
them to happen. By the way --
MR. ROSENTHAL: We have a test to look at
corrective actions, and we've been doing some work with Bill
Veseley, and I'll give a hypothetical example of what Dr.
Apostolakis is talking about, or a pretty concrete one.
There's probably like one or two guys at a plant that are
MOVATS qualified now, you know, and they go around to all
the safety related valves over time, and they set up the
valves. So now if you have an error in the procedures that
they're following or an error in the MOBAT software, I mean,
whatever, then they're systematically marching through the
plant messing it up, and it is not revealed through normal
operation because you don't demand the valves. It is not
revealed during the current IST which is typically done
under zero delta P, and it's sitting there latent to catch
you when the real event takes place. So, I think that's a
concrete example of the kind of thing we might we talking
about, and we do intend to explore that in my branch,
although maybe not as part of this.
DR. APOSTOLAKIS: As part of what? It should be
part of the program.
MR. ROSENTHAL: Because I have another task I'm
looking at. It's a research task. Okay, so in any case, if
you take the events, you can split them apart. There's --
among slide 8, there's a zillion ways of cutting and
parceling it. I know that some people like program, some
people like processes, and you can cut it any number of
ways, but the point is that this is the sort of issues that
you see, and you see the latent errors rather than the
so-called active control room errors.
So now I'm supposed to use this information to
risk inform my human performance plan, and we should
continue doing this work because we want to mind this more,
and I think it's -- and my intent is to independently
publish this in depth look at ASP because I think it has
lots of utility.
Now let me get to the element -- the oversight
process. I'm sorry, the human performance plan elements,
and there's essentially four elements of the oversight
process. Licensing and monitoring are really NRR functions,
and we'd be occasionally provide some assistance. There's
risk informing the regulations in an area which we call
emerging technologies. Jay Persinsky has I think ABC flow
charter on his machine, so he's able to make fancier slides
than Steve.
The top block is the vector, really, is the
maintained safety factor, and we decided this is not a
burden reduction of public confidence, though really it
should be labeled maintain safety.
The squares and wigglies are current ongoing
programs within the NRC, and then the wide background boxes
are things that we're doing. The first one, and we just
got, as I say, the signed letter from Sam Collins now, is
characterizing the effects of human performance within the
overall plant oversight process. I think that that, at
least in part, will attempt to answer some of the questions
that the ACRS has raised on how you consider human
performance a cross cutting issue.
Let me give a specific example that we discussed
again at the subcommittee. One might argue that if you have
.96 diesels and you wanted .95 diesels and you have 0001
high pressure safety injection and that's what's in your
PRA, so you're meeting all your hardware goals, why should
you even look at human performance? It's covered within the
hardware, and there is some merit to that.
If I know that the unreliability contribution due
to the human is important for diesels and HPSI and low
pressure injection, whatever it is, if I know in these
several areas that it's important, then I have a cross
cutting issue that may be greater than the individual
elements. To the extent that I'm only monitoring about 20
percent of the risk by looking at the current generation of
PI's, then I want to look at this cross cutting to see these
other issues. So, that's why I think it's important to look
-- what?
DR. APOSTOLAKIS: Why are you limiting yourself to
human performance? It seems to me the problem can
accommodate all three cross cutting issues. The second one
is a safety conscious work environment, and I look at the
inspection manual that you have prepared, and you're asking
questions, you know, about the questioning attitude of
people -- I mean, safety, cultural kind of stuff. Safety
culture. Safety culture. So, why is this limited to human
performance? Corrective action certainly is a human
activity that, you know. It seems to me that your program
should be focusing on all three.
MR. ROSENTHAL: We're surely looking at corrective
actions. We're surely looking at human performance. I have
no specific activity on a safety conscious workplace, but
maybe NRR can think about what they might say about it.
DR. APOSTOLAKIS: You are asking the right
questions, or some of the right questions in the inspection
manual.
MR. ROSENTHAL: Okay, good.
DR. APOSTOLAKIS: I mean, this questioning
attitude, I thought I would never see, but you know, you
have it there. There is a question about that. So, why did
you say you received from Mr. Collins, what was it, a
memorandum or something?
MR. ROSENTHAL: User needs.
DR. APOSTOLAKIS: User need, the specifically
mentioned human performance and not the other two?
MR. ROSENTHAL: Right, and we can -- okay. The
NRC currently hypothesizes that the current generation of
PI's and the current inspections are adequate and that human
performance will be revealed in the equipment reliability.
That was given as a statement of fact, and I would say that
that's a hypothesis or an assumption, and now we've been
charged with the opportunity to explore to what degree that
assumption is or is not true.
DR. APOSTOLAKIS: Well, it's not clear what their
assumption is. I think you're right that the original 007
report said that, although what confused me a little bit was
at the subcomittee meeting. It was stated by the staff that
the performance indicators and inspection findings will
reflect poor performance in any of the cross-cutting issues.
Now, given that you are submitting this manual for part of
the base lying inspection problem, this is a true statement.
We will be reflected somewhere. I mean, you're asking
questions of the safety culture. So, it's not going to
session anymore, and I'm a bit confused now as to whether
this hypothesis be tested, or it's a thing of the past.
MR. ROSENTHAL: I mean, surely we should get on
with the oversight process, you know, let that go. I do
think it appropriate and what we intended to do was to go
back to the operational data with emphasis on these 50 top
ASP events and match it up against the current generation of
PI's and the current plant baseline inspection and ask very
systematically, are the kinds of -- what?
DR. APOSTOLAKIS: You said the current, not the
one that would be enlarged by what you are proposing. When
you say current, do you include the manual you have
prepared?
MR. ROSENTHAL: Double 0 seven.
DR. APOSTOLAKIS: Oh, double O seven is current?
MR. ROSENTHAL: Right.
DR. APOSTOLAKIS: See, with this manual, we're
deceived.
MR. TRIMBLE: We are going to talk about the
supplemental inspection.
DR. APOSTOLAKIS: Yeah, the supplemental
inspection. Is that part of the current process or not?
MR. TRIMBLE: It's part of the new oversight
process.
DR. APOSTOLAKIS: So there is no hypothesis then.
I think the claim is true because you have questions there
that refer to the cross cutting issues. Yes, the baseline
inspection and we've got you. The big question was whether
the component performance -- I mean, you know --
MR. SEALE: I guess I'm totally lost. Are we
discussing the human performance plan or something else?
DR. APOSTOLAKIS: Yes, yes, because there's a big
question whether they are supporting the oversight process.
MR. TRIMBLE: What I'm proposing to do is to take
what we've learned from the PRA's and the ASP work and match
it up against the currently, you know, what we're going to
implement, not the prior lecture that you heard on risk
based PI's, but what we're currently implementing, match
that up against those 50 events, match that up against what
we know from PRA and see if either the PI's or the baseline
inspection process picks up the issues that I'm talking
about.
If they do find, then that's a very interesting
finding. If they don't -- if there are holes, let's
identify the holes, and then you might find out how you
would go about fixing those holes.
DR. APOSTOLAKIS: I guess the question is which
baseline inspection process?
MR. SEALE: The one they're currently
implementing.
DR. APOSTOLAKIS: Not the one that includes this
particular supplemental --
MR. ECKENRODE: First of all, that supplemental
inspection that you're looking at is just that, it's a
supplemental. It's -- in fact, it's probably going to be
used about three levels down from the baseline, two levels
down, at least from the baseline.
DR. POWERS: By the way, the direction down
doesn't mean anything to me.
MR. ECKENRODE: Okay, The inspection program,
first of all, has a baseline -- a group of baseline
inspections. They're done every year or every -- on a
periodic basis. When you find some significant issues that
cause you to have, I believe, it's two whites in a
particular area, and this is the baseline inspection, you do
a supplemental inspection. That's the next level down,
okay?
DR. POWERS: Okay, yeah. The supplemental
inspection, I didn't understand. Okay, from that, you've
got to have a significant number of issues in there. Then
you may use, if they happen to be human performance issues,
you would use the inspection procedure that you're looking
at now.
DR. APOSTOLAKIS: And this is part of the current
revised reactor oversight process?
MR. ECKENRODE: That's correct. By the way, it is
a draft. It's not been approved yet. It's out to the
regions for comment. That's it.
MR. TRIMBLE: So, that's a new process.
DR. APOSTOLAKIS: So, there is a staged approach,
plus the baseline must find it, in which case Jack is right.
You should be testing this.
MR. TRIMBLE: Correct.
DR. APOSTOLAKIS: Yes.
MR. ROSENTHAL: Okay, the second leg is primarily
NRR activities, and I'll let them speak for themselves. I
would point out that policy review includes things like what
we might do on fatigue or staffing or hours worked, that
sort of thing, and that would be an effort in which I would
hope both offices would work together.
DR. POWERS: Well, that's one of the areas that I
didn't quite understand in the plan. It seems to me that
there's been a hypothesis, I guess, if you will, that no
need to worry about these things, that if, in fact, fatigue,
for instance, is a problem, it will surely show up in the
other performance indicators.
Now does the plan work to test the hypothesis or
something else?
MR. ROSENTHAL: You know, there is an active 2206
petition on this and there's been staff meetings, and we've
met with NEI, and I think that it's -- I'm just -- I'm
hesitant to speak too much right now.
MR. TRIMBLE: Let me just speak to the fatigue
issue. We haven't been thinking of it in --
DR. APOSTOLAKIS: Use the microphone.
MR. TRIMBLE: I'm sorry. I'm Dave Trimble,
section chief over at NRR, License and Human Performance.
We have been thinking about the fatigue issue. We haven't
been thinking about it in the context really of the
oversight issue or as perhaps you may be alluding to. It's
been on a separate track. And, as Jack referred to, we have
a petition for ruling that's been sent in, basically, where
the petitioner --
DR. POWERS: We're familiar with it.
MR. TRIMBLE: Right. And so, we're basically
looking at a couple of options how to deal with that. But,
again, we have not been associating it particularly with the
oversight process.
DR. POWERS: But when I look at the plan, I just
can't -- I can't make that disassociation and there are
three or four of those similarly on conduct of operations
and human factors engineering. It seemed to me that if I
were to follow the oversight philosophy, after a while, I
just don't need to worry about these things, because -- I
mean, to say not worry about is over stated. I don't have
to worry about it at the baseline level. I have to worry
about it at one or two levels down, okay. Whichever
direction we go, I worry about it when I find -- when I have
findings. And so, it's not that I don't worry about it at
all; it's that I don't worry about it at the baseline level,
okay.
That, by section, does not come across in the
plan. I think what you're telling me is, yeah, it doesn't.
But, I think it should. I think it should be explained in
the plan, that we're looking at working hours, we're looking
at conduct of operations, we're looking at human factors
engineering; not to do something day-to-day to the poor guy
that's running the plant, but rather when we have findings
and we're trying to understand what the cause of these
multiple findings are, and it does have to be multiple, but
we need to understand working hours, conduct of operation
and human factors engineering. That would make it more
comprehensible to me why we're going into those.
DR. APOSTOLAKIS: Well, that, again, relies on the
untested hypothesis.
DR. POWERS: That we will pick these things up --
DR. APOSTOLAKIS: We will pick these things up.
DR. POWERS: -- in the performance indicators?
DR. APOSTOLAKIS: And the baseline inspection.
DR. POWERS: And/or the baseline inspection.
MR. ROSENTHAL: You've had separate briefings on
the risk-informing regulations. Let me just give examples
of a -- currently, a -- is not a design basis event, but
your PRA would say that they -- on a PWI is an important
sequence, so -- as well as things where you would reduce the
burden. That would be an example where your observations
would drive you to perhaps do more. So that we see that a
contemporaneous look at human performance ought to drive --
or be an input to risk informing the regulations. And then
the other thing is that we see this as a source of data to
provide information for HRA to drive PRA.
The last area, emerging technologies, let me give
a specific example. Beaver Valley, within the last six
months, they lose an electrical bus and they had 130 some
odd alarms come in at the same time. And I would say that
that is unfair to the operator and that -- I will, okay.
DR. POWERS: Well, that's not news. We had TMI,
where we had literally thousands alarms coming in at the
same time.
MR. ROSENTHAL: Okay. So, it is time to do alarm
prioritization. And it's going on, to some extent and
that's good. And I'll surely will take advantage of Steve's
smart sensors and AI, you know, all different sorts of
things. And it's a good thing. You just have to do it
carefully. So, if there's definitely an interface between
the digital INC plan and what we would know about human
performance --we've already looked at some things like alarm
-- and provided -- we've written NUREG CRS on alarm
prioritization and how do you do -- I've got lost on the NRC
Webpage yesterday, looking for generic issues, which is
another one of my responsibilities, no less an outsider.
Well, if you have somebody then responding to an event on a
-- will they be -- they have to be able to navigate through
their multiple pages on their CRT and what not. So, there
are issues.
DR. POWERS: My guess off hand is that the average
operator would be better trained on a CRT --
MR. ROSENTHAL: Absolutely.
DR. POWERS: -- than you were on NCR's Webpage.
MR. ROSENTHAL: Yes, sir.
DR. POWERS: That's my guess. I don't know how
well trained you are on NRC's Webpage.
MR. ROSENTHAL: Yes, sir.
DR. APOSTOLAKIS: Isn't there an untested
hypothesis here again, that the fact that they had 150
alarms is bad? How do you know it's bad? And doesn't it go
against the current regulatory philosophy that the
Commission is promoting, at looking at results, performance?
I mean, why shouldn't we expect that an the oversight
process, with its performance indicators and baseline
oversight process pick up failures that would lead us to the
conclusion after we broke down one level, that the number of
areas was bad -- was high, too high? What is so special
about these that needs to have a separate item? In other
words, shouldn't this problem really focus on the first and
third key program areas: reactor oversight and risk
informing the regulations, and really go out of its way to
support these two activities that are, I think, the two
major initiatives the Commission has stated? And everything
else should be justified, in the context of these two, as
opposed to --
MR. ROSENTHAL: I don't -- I don't know if I'm
doing semantics or what. The reason that we measure this
emerging technology and put it out separately is that this
is work that's going to be done far in advance of stuff that
we're going to see in the oversight process.
DR. APOSTOLAKIS: Which is an untested hypothesis.
MR. ROSENTHAL: And I -- I would -- I'm going to
live to regret that phrase. I would assert that stuff is
going on at real plants, okay. Calvert Cliffs, I know,
would go into its life exemption with pistol grip controls
and fancy flat screen digital displays, of new information
systems and all controls. And then at some point, we ought
to be able to understand and review it. If we're not going
to review it, because it's an information system, then we've
given tacit approval. That is okay, if we know what we're
doing. So, we ought to be able to get out in front on the
emerging technology, so that at a minimum, we understand
what's going on and we shouldn't be an impediment to the
plant modification. So, at least in my mind, I break that
off from the oversight process, which is on a different time
scale.
DR. APOSTOLAKIS: Wouldn't that -- wouldn't it be
better, though, to have ATHEANA out there, say, you know, on
the risk informing the regulations? Right.
MR. ROSENTHAL: That's --
DR. APOSTOLAKIS: Yeah, I know that's what you
mean.
MR. ROSENTHAL: Well --
DR. APOSTOLAKIS: ATHEANA tells us that we have to
worry about the error forcing context and that this
particular issue may be an important element of the error
forcing context. But, at least, you're going through in
logic here that is forcing you to think about how important
is this issue or how important could it be. And, at the
same time by doing this and having those fellows interact
with you -- this is another untested hypothesis, but I think
the ATHEANA guys will be able to limit bit this motion of
things that form an error forcing context, which his
something that this committee complained about last time.
And some of us on the committee don't see this
close collaboration between your group and ATHEANA, so that
both of you will benefit from each other. Because, you
know, you read ATHEANA and so many things -- anything that
anybody can thing about is something that can affect the
error forcing context. And then Jack's group can come in
and say, wait a minute now, here's what the data says, the
evidence, right?
MR. ROSENTHAL: Nathan and I met -- have met a
couple of times. And I just have to give you an IOU
standing up here right now, that, in fact, the intent is
that we work close together.
DR. POWERS: Well, that is one of the things that
I persist in not understanding about the human -- about this
plan is the disassociation between this set of activities
and this ATHEANA development activity. I mean, why isn't it
all part of the same -- or maybe it is, and just does never
get mentioned or hidden under PR model development?
MR. SIU: Yeah. This is Nathan Siu, Office of
Research PRA Branch, in charge of the ATHEANA program. The
-- you're right, Dr. Powers, PRA model development doesn't
cover ATHEANA. It covers HRA, in general. One of the
things that we have to look at is -- one of the things that
you have to do is develop the plan for the HRA model
development that addresses issues that are covered by other
methodologies. I think the question came up this morning
about, for example, the ASP HRA. Then when you get --
DR. POWERS: It came up with SPAR HRA.
MR. SIU: SPAR, sorry.
DR. POWERS: That is identical --
MR. SIU: That's right, okay. But the point is
that we have to come up with a program plan that addresses
the variety of activities going on within HRA, within the
agency, how they look together. And, obviously, part of
that, also, is how we interact with the human performance
program. You're right, we have not done a good job of that
in the past and Jack and I have spoken about that and sworn
internal brotherhood and stuff like that.
[Laughter.]
MR. SIU: We will work on that and get better.
It's obviously not perfect right now.
DR. POWERS: Well, I mean there are other things
that discuss it. For example, the plan discusses the
collection of data. And -- but, we've been collecting data
a long time about human performance and things like that.
It never seems to do anything. And my example of that is
just what we heard about earlier this morning, the same
thing that brought up ASP HRA is that they go through and
they see -- frequently, they do things with THERP tables and
the THERP tables -- the publication date was 17 years ago.
I happen to know that much of them was made up from
information gathered in the 19 -- late 1960s.
MR. ROSENTHAL: It was the early 1960s. I was
there.
DR. POWERS: You were there. This data never
seems to result in any changes in the way we do human
reliability analysis. So, why are we collecting it?
DR. APOSTOLAKIS: Put another way, would it be
time now to revisit the human reliability conduct?
MR. ROSENTHAL: It's time to revisit human
reliability.
MR. SIU: Yeah, and that's -- that's, again, what
we have to do in the development of this plan. Let me
address this point about the data not telling us that we
should change the way we do HRA. Actually, the data have
told us that we should change the way we do HRA and that's
why we are working on ATHEANA, because it is this notion of
looking at the severe accidents -- not severe accidents,
excuse me, the major events that have occurred, what are the
driving factors underlying those events and, therefore, what
are the things that we should be including in the models.
Now, I'm using data in a broader sense than just,
say, the number of successes and failures. We're looking
at, again, what are the factors that have been observed for
these different events and trying to develop a way to tie
those -- that information from those events into the
analysis. Right now, as the committee has noted, our
quantification process is -- what shall I say -- it needs
improvement and --
DR. POWERS: It's geriatric.
[Laughter.]
MR. SIU: We recognize that we have to do
something about that and that's one of the things that we
hope to address as part, frankly, of some of the
applications work we're doing, for example, in PTS.
DR. POWERS: I mean, don't you think that you can
make a more persuasive case in your plan to come in and say:
my THERP tables are geriatric; I've got better data; please,
Commission, give me the money and resources to allow me to
bring these into the 21st century? I honestly don't believe
the Commission understands that the THERP is fine. I mean,
it's been fine for a long time. It's a good way of doing a
lot of things. But, you have the database now to take THERP
out of the weapons community, where it was defined, and make
it specific to the reactor community, and you've got a good
database to do that updating. That seems like it ought to
appear in the plans somehow.
DR. APOSTOLAKIS: Let me give you an example of
that. There is a statement in the human reliability
handbook that human actions separated in time tend to be
independent. Now, with all this evidence about latent
errors, I'm not sure that true anymore. It depends on the
nature of the latent error. And, yet, we are using that.
We have gone to staggered testing of redundant trains now,
as opposed to consecutive test. I'm not saying go back to
consecutive test, but is that an assumption that is still
valued, in light of the evidence of the last 20 years, or
should we do something about it? Okay, I'm not particularly
concerned about the distributions that Alan Swain put there
about individual human actions, because I don't think anyone
can really do much better than that; but, there are some
assumptions there in the model that I think need
questioning.
And then, of course, the other point that we made
in one of the recent letters is the humans, during normal
operations, starting an initiating event, because ATHEANA
has focused on recovery actions and as of commission given
the initiator.
Now, these are issues that, it seems to me, are
really important and should be prominent in this. By the --
another question, Jack. Plant licensing and monitoring,
these are activities that you have to do, you don't choose
to do, because you're supporting other offices?
MR. ECKENRODE: They're NRR activities.
MR. ROSENTHAL: No, these are -- yeah, these are
NRR -- most of this whole leg is NRR.
DR. APOSTOLAKIS: But the "Y" boxes, rectangles,
human factors information system.
MR. ROSENTHAL: That's NRR. No, we'll give him
the floor in a minute.
MR. PERSENSKY: Excuse me, George, this is an
agency-wide plan. It is not a research plan. So, their
activities are, also, included. So, their activities are
included in here and that's where the -- a lot of them are.
The first two boxes are really NRR activities, except for
some things that we're doing under user need.
MR. ROSENTHAL: Let me just pick out one more
bubble and then give it to NRR and then take the floor back
quickly, and that is that -- and it's a very small effort,
but I find it interesting, and that we wanted -- we are
doing some small modest effort on econmic deregulation,
industry consolidation, and we're trying to get out in
front, not waiting for -- to monitor equipment reliability.
And there are issues like: what will this industry look
like with six or eight generators, rather than utilities,
owning multiple plants; what kind of pressures will they be
under; what resources will there or not there be to maintain
safety.
The whole paradigm of a base loaded plant may
change, when in the Midwest on a Saturday August afternoon
at $2,000 per megawatt hour, you make all the profit for the
year on that afternoon. Well -- so, will the very paradigms
for how you run a plant change? And if that's true, then
what are there --
DR. SEALE: Make Shack pay for it.
MR. ROSENTHAL: -- are there technological issues
that we ought to be addressing? And what we'd like to do is
just do a modest amount of thinking, a little bit of
research, to get out -- to try to get out in front of what
these things would be, rather than waiting.
DR. POWERS: You know, it's hard -- hard to be
opposed to thinking and getting out in front. On the other
hand, in this area -- this is another one of those areas
that I say, gee, no matter what they find out, there's
nothing they can do with it, because -- suppose a licensee
says, I am becoming economically competitive; I don't want
all my profit to be on Saturday afternoon in the Midwest in
August; and so, I'm cutting my manpower by 50 percent with
this plan. And the NRC can say, yeah, and let's look at
your performance indicators and they all stay good. They
don't do anything about it. Then, how do you act upon this
information that you get?
MR. ECKENRODE: We're doing a little bit of that
in the license transfer area right now --
DR. POWERS: Yeah.
MR. ECKENRODE: -- in NRR. We're basically
approving the organization charts. We have one now that's
going to have -- one organization is going to have 20 plants
under it and we find that a problem. So, we're doing
licensing reviews now, from that point of view.
MR. ROSENTHAL: It's an area where I think --
before you tell me how I'm going to use the information, I
have to invest at least enough to understand what -- to try
to get a handle on what -- even what are the issues.
DR. APOSTOLAKIS: Let me -- I've got another one.
I'm trying to understand what it is we're trying to do here.
This is an agency-wide problem, so what is NMSS?
MR. ROSENTHAL: I really --
DR. APOSTOLAKIS: They don't involve humans in
their activities or are they involving --
MR. ROSENTHAL: Ideally -- ideally, NMSS would
play -- NMSS is, at this point, too busy reinventing itself
to participate. My assistant branch chief, John Flack, who
is a PRA practitioner is on loan to NMSS, to help them risk
inform NMSS activities. I would anticipate that in future
years --
DR. APOSTOLAKIS: Years.
MR. ROSENTHAL: -- years, they would be more --
they would be part of an agency-wide plan.
DR. POWERS: When we asked this question a
year-and-a-half ago, we were told the same thing. So, years
is apparently the right time frame.
MR. ECKENRODE: Originally, they were players in
this. Yeah, they showed up once.
DR. POWERS: And then they bowed out quickly.
DR. APOSTOLAKIS: Okay. So, that's one
observation. The second observation is: is this supposed
to be a picture -- a figure of the way things are and there
isn't really much we can do about them, or we will try to --
we can try to improve these? And if so, who is the
decisionmaker that will change this? Is it EDO or is just
the three of you? If you agree, then we can change this.
I'd like to understand how much flexibility do I have here.
If you don't --
MR. ROSENTHAL: You have no flexibility.
DR. APOSTOLAKIS: No, no, no. These boxes here
are there, because Congress said that.
MR. ROSENTHAL: No, no, no, no.
DR. APOSTOLAKIS: Comment on that.
MR. ROSENTHAL: You have no flexibility for my
fiscal 2000 efforts. They are being executed right now.
DR. APOSTOLAKIS: Okay, right.
MR. ROSENTHAL: You have influence over 2001. And
your questions that I stood up here and sweated over, we're
going to try to address them. I mean, yeah, we really have
been responsive on the --
DR. APOSTOLAKIS: Who is "we?"
MR. ROSENTHAL: In this case, I know that I can
affect my -- I know that I can affect my fiscal 2001 and I
know that the prodding that you're giving me here will
affect the details of what we do in 2001, and you'll have
more freedom yet for 2002.
DR. APOSTOLAKIS: No, but, the point is --
MR. ROSENTHAL: So, we really do pay attention.
DR. APOSTOLAKIS: -- if NRR is determined to do
some of these things, then it's not entirely up to you to
change your program. So, I'm trying to understand who has
the final authority here to say, yeah, this is a good idea;
we're going to change it. If the three of you agree, it's
going to happen or it has to go higher?
MR. ECKENRODE: Higher.
DR. APOSTOLAKIS: Higher, where?
MR. ECKENRODE: It definitely has to go higher.
This was going to the Commission.
DR. APOSTOLAKIS: All the way to the Commission?
MR. ECKENRODE: This plan was presented to the
Commission.
DR. APOSTOLAKIS: Okay, that's good to know.
MR. ROSENTHAL: Okay, NRR?
MR. TRIMBLE: Yeah, again, our remarks are limited
in nature. And we were going to focus on the elements of
the performance plan that relates to the oversight process
and we are speaking in a support role to that process. And
we've already touched on what I would say our flagship
effort, which was to test this hypothesis through the user
need that we've been talking about. Steve has finally
gotten over to research here, so we can get rolling on that.
The second area is -- again, we've touched on the
supplementary procedure and all these areas, Dick is going
to make a couple additional comments. And that was
developed. As we mentioned earlier, it's in draft form and
it's to be used two levels down or two levels up. We want
to think about it. And that -- the thrust of that is more
for an understanding, if the licensee is not to do the work
for them, but to understand if they are going about it in
the right way.
The third area, we've had some sort of hesitating
starts in the -- coming up with the significance
determination process. We're just -- we came up with a
concept. We were kind of presenting it to the oversight
folks, getting -- the processing of getting beaten back a
little bit and then trying to make advances from a slightly
different direction. One area they certainly want us to
focus on is what to do with multiple problems across the
board. And I have to say that, initially, we were taking
the tack of more of the broken operator. What happens if
somebody returns? In a requalification setting, if they're
found to be lacking in a significant -- risk significant
action, they return to shift unremediated. And,
particularly, if multiple crews do, then you're sort of set
up for failure and you wouldn't pick that up until the event
happens.
DR. POWERS: Yes.
MR. TRIMBLE: So, that was one of the thrusts we
were going down. But, again, we're running into, you know,
the kinds of things we have to deal with: was that already
considered and through some of these other assumptions in
the PRAs. And so, we're starting to interface with the PRA
people on that.
But, enough said on the general overview. Dick's
got a little bit more detail for you.
MR. ECKENRODE: I'll give you this little
presentation. Dave is going to answer all the questions.
DR. POWERS: We hardly ever ask any, so.
MR. ECKENRODE: I know. Back on the user need
that went over to research, the precise words of the
assumption that were in that user need are on the top here.
The effect of human performance in plant safety will be
largely reflected in plant performance indicators and
inspection findings. We agree that it's not -- you're not
going to get them all, but they expect it will be largely.
We are going to try to test that assumption, first
through the research user need from their direction, and NRR
is going to look at it from -- through our human factors
information system program. We intend to -- in that area,
we intend to record information from the inspection reports
that are going to come out in the new process. And assuming
there is sufficient data -- and we're not sure of that yet,
because that process is being changed, also, the 50.72
process -- if there is sufficient data, we intend to attempt
to compare it with the last historic data we have in HFIS
for the last five years.
DR. SEALE: Could I ask you something about your
assumption? I read that and I have difficulty. I don't
know whether you're telling me that superior human
performance will result in green performance indicators and
positive inspection findings, or that efficient human
performance will result in white or yellow performance
indicators and deficiencies in performance -- in inspection
findings.
MR. TRIMBLE: I believe the latter is the case,
that poor performance would show up --
DR. SEALE: Okay.
MR. TRIMBLE: -- in these indicators.
DR. SEALE: Okay.
DR. POWERS: You know, I mean, this is -- I can't
tell you how enthusiastic I am about this idea of testing
hypothesis and what not, and I think that needs to come
across clearly in the plan, that the agency is going back
and testing its hypothesis here. Because, I think -- I
mean, that's the mark of a really first-rate operation here,
and that sense needs to get across.
DR. SEALE: Yes.
DR. POWERS: And then here, you're telling me,
yeah, we're actually going to go use this data for something
that we've been collecting. You know, that makes me excited
about it, too, and I think that's -- I think it's a --
MR. ECKENRODE: We've been using this for a long
time, for a lot of different things. This is the first time
for this kind of a thing.
DR. POWERS: Yes.
MR. ECKENRODE: We don't know if it's going to
work or not, because, frankly, the new reporting process is
going to reduce the amount of data we're going to have and
that's can be a problem.
DR. POWERS: If you knew it was going to work, it
wouldn't be research; it would just be go do it.
MR. ECKENRODE: We don't do research in NRR.
DR. POWERS: I know. But, you would do this. If
you knew this was going to work, you wouldn't have sent it
over to research. You would have just sat down and done it,
right?
MR. ECKENRODE: That's right; sure. The
supplemental inspection procedures that you have copies of,
it's based on this hierarchy of baseline and supplemental,
and then this is a sub-supplemental. So, you understand --
we went through that. I don't think I have to talk about
that at all.
These are the objectives of that. And, again, the
purpose is not for us to do the root cause analysis; it's
for us to do -- to look at the licensee's root cause
analysis process and make sure they're doing it right. Now,
obviously, to do that, we're going to have to go down and
find out ourselves pretty much what happened, and that's why
the detailed procedure that you've seen. As I say, this is
a draft procedure. It's gone through NRR. It's out now to
the regions for comment. We haven't heard anything yet at
all from them. They were quite -- the region were quite
happy to see one. They're interested in seeing one, because
they feel that their human performance is being given sort
of the short shrift in their reports. So, this is the area
we're --
DR. POWERS: And you can't underestimate how
important the corrective action program is. And you've read
more AITs than I have, but I can't remember one that comes
back and doesn't say something about that the licensees
really didn't do this corrective action in the past well, or
something like that. I mean, it's a very common thing in
AIT.
MR. ECKENRODE: Right. These are the areas that
you see in the inspection procedure. We looked at the -- in
the human system interface area, we looked at both the
visual information annex and some mechanization, same as the
control function. We look for whether the control function
is available and whether it's -- the hardware is there to do
the job.
The only other two that we said anything about, I
guess procedure use and adherence is talked about there, but
procedural quality is not. And the reason for that is that
the two inspection procedures are still in place for EOPs
and for plant procedures. It's IP 42001 and 42700 are still
in existence and they would be used for that particular
thing.
Similarly, training and qualifications, we don't
look at all in this procedure. It slipped out, IP 41500.
That's why those two are --
DR. APOSTOLAKIS: What are work practices?
MR. ECKENRODE: Pardon me?
DR. APOSTOLAKIS: Work practices?
MR. ECKENRODE: This has got to do with all of
your work plans and work -- you know --
DR. POWERS: Plan the day sort of thing.
MR. ECKENRODE: Plan a day; yeah, discussions of
-- what do they call them -- tail --
MR. BARTON: Pre-job briefings.
MR. ECKENRODE: Pre-job briefings, right.
DR. APOSTOLAKIS: Where is the work
prioritization?
MR. ECKENRODE: That's in there, too.
DR. APOSTOLAKIS: It's in there?
MR. ECKENRODE: Yes. It's part of it.
DR. APOSTOLAKIS: I don't know, what -- are we
going to have the opportunity to understand the logic behind
this, at some point at the subcommittee meeting? Jack, do
you plan to have a -- oh, no, it's not Jack. I think we
should discuss -- I would like to understand the logic that
led you to this set, as opposed to some others.
MR. ECKENRODE: This set of areas?
DR. APOSTOLAKIS: Yes. I am not saying that --
MR. ECKENRODE: This is a common set of human
performance areas. And, you know, we talked earlier about
safety culture. Yes, there's some of that in here. But,
the whole human performance area has -- you know, is
involved here. It covers just about everything, except
getting into upper management.
DR. POWERS: We don't do that.
DR. APOSTOLAKIS: I would like to understand this,
sir, so maybe in a future subcommittee --
MR. ECKENRODE: I'm not sure I understand your
misunderstanding or your --
DR. APOSTOLAKIS: Why these --
MR. ECKENRODE: What other material --
DR. POWERS: What's missing. What would you put
in?
DR. APOSTOLAKIS: Resource allocation, I don't
know, where is that?
MR. ECKENRODE: Resource allocation is -- it's
there.
DR. POWERS: That's in there, yes.
MR. BARTON: It's not called that, but there's
something else that would go and pick that up in here.
DR. APOSTOLAKIS: Well, I'd like to understand
that. I can replace this by --
MR. BARTON: More sufficient workers assigned,
appropriate materials available, sufficient time allocated
for a job.
MR. TRIMBLE: Well, we are certainly willing to
talk about it in the future.
DR. APOSTOLAKIS: Yeah, I think we should discuss
it. I'm not saying it's bad. All I'm saying is I want to
understand it.
DR. POWERS: We'd be glad to hear your thoughts on
it, because we --
MR. ECKENRODE: Oh, yes.
DR. APOSTOLAKIS: Is this consistent with the
Commission's decision a year ago to stop all work on
organizational?
MR. ECKENRODE: There are no organizational issues
in here.
DR. APOSTOLAKIS: There are no organizational
issues in coordination or work practices.
DR. POWERS: They are all here, George, they're
just not called organizational issues.
MR. TRIMBLE: Let me characterize this. This is,
again, a boundary between the old and the new approaches
here, and it may be dangerously on that boundary. It's --
it is -- actually, it's draft. It was viewed as how on
earth do we check and see if licensees -- at some point,
they're having problems; at some point, you need to make
sure that they are looking at the right spots. You have to
satisfy yourself. So, it's a way of providing guidance to
the inspectors in an easy to use format.
DR. POWERS: Now, my understanding is that I don't
think it's on dangerous -- dangerously on the boundary
there. This is the sub-sub-supplemental -- I mean, this is
after you found things in the new system and you've reached
the point where NRC feels it has to do something.
MR. ECKENRODE: Okay. The next theory is the
significance determination process. This is sort of our
objective, is to be able to cover these functional areas.
It's -- we may not be successful in it, but we're hoping to.
And, again, on the issue areas, the same thing, the seven or
eight issue areas there, we'd like to be able to cover all
of these through the significance determination process, and
to be able to show significance.
We're operating on what we consider to be a very
basic premise in human factors, which is every human action
requires information to initiate the action and control
capability to accomplish the action. It gets pretty simple,
the idea being that we're going to look at all of the kinds
of things that we find and the issues in the expected list.
DR. APOSTOLAKIS: So that's interesting
information. There's nothing else?
MR. SIEBER: Like brain?
DR. APOSTOLAKIS: Yes.
MR. ECKENRODE: Pardon me?
DR. WALLIS: Well, there must be.
DR. APOSTOLAKIS: There must be something there.
DR. WALLIS: There must something that tells you
what to do with the information, in order to initiate it.
DR. APOSTOLAKIS: Yeah. I mean, they operators,
themselves --
MR. ECKENRODE: Not defined through inspections or
PI.
DR. APOSTOLAKIS: I thought the basic premise was
going to be the standard model of your information --
processing information and then action. You seem to be
skipping the processing of information.
MR. ECKENRODE: That's true.
DR. APOSTOLAKIS: And you are doing that
consciously?
DR. WALLIS: He regards that as part of the
information, I think.
DR. APOSTOLAKIS: No, because there is an
assumption that the procedures tell you what to do.
MR. ECKENRODE: Yeah, the response to the
information, as far as training.
DR. APOSTOLAKIS: And I was noticing behavior is
not covered by this.
MR. ECKENRODE: Right.
DR. APOSTOLAKIS: Unusual circumstances that would
require some creative thinking is not covered by this.
MR. ECKENRODE: That's true.
DR. POWERS: This gets -- the significance in
determination process gets triggered when there's a -- I
mean, they have to find a -- it has to be a finding.
MR. ECKENRODE: That's right.
DR. POWERS: And a guy saving the plant is going
to be in somebody else's bailiwick, because it was initiated
by something else.
DR. APOSTOLAKIS: So, if it's in low power
shutdown, in a situation that the operators understand and
they made a mistake, wouldn't go through the SDP?
DR. POWERS: Sure, it would. I mean, if they had
a fault there, they would come back and say the operators
didn't understand that. So, it must be there's inadequate
procedures, procedures weren't followed, they didn't have
the right information, something like that. That's what
he's going to do when he looks at those.
DR. SEALE: Training didn't cover the case.
MR. ECKENRODE: Right.
DR. WALLIS: So, all eventualities are foreseen
ahead of time, you're saying.
MR. ECKENRODE: Well, we would hope so.
DR. POWERS: So, they did not, because the
procedure wasn't written for them. So, in essence --
DR. APOSTOLAKIS: In essence what they are saying
is in this parenthesis up there, display and so on, they
define the context for ATHEANA.
DR. POWERS: Yeah, they do.
DR. APOSTOLAKIS: So, they should stop
investigating the context and take these and run with them.
You need display, training, procedures, and supervise the
action.
MR. TRIMBLE: Yes. One of the things -- part of
the feedback we were getting on this when we first did this,
we sort of thought you need to kind of understand -- go
through the process and understand what was missing, so the
operator couldn't do the function, and then that would
somehow give us a feel for the significance of the problem.
One of the bits of feedback we've gotten is, well, maybe
this approach -- maybe we're getting into trying to step
into the licensee's business too far, too early, and
understand the root cause -- getting to a root cause,
opposed to limiting ourselves to deciding what the
significance is. And that's where we -- we need to go back
and -- maybe we're filing the philosophy here and I think --
DR. APOSTOLAKIS: I don't think this is accurate
what you have there. Training is not information.
MR. ECKENRODE: Well, you get the information out
of training.
DR. APOSTOLAKIS: Yeah.
DR. POWERS: I can see what your problem is; but,
I guess I'm more supportive than maybe your critics are on
this.
MR. ECKENRODE: There's two more you can shoot at
them. We have -- this is where we're getting -- trying to
get into -- we're trying to get into a little bit more of
the significance. And the first one, of course, is that no
information or control capability is better than incorrect
information. You have an incorrect procedure, it's worse
than not having a procedure at all, for instance.
DR. POWERS: I guess I have troubles believing
that this is true, because there are levels of
incorrectness.
MR. ECKENRODE: Yes, of course.
DR. POWERS: There's a minor error and then
there's flat -- completely -- it says right, it should have
been left, okay. When it's that bad, yeah, I agree with
you. When it's a minor incorrectness, then maybe it's not
so bad. And I think no is, therefore, incorrect, has
gradations.
MR. ECKENRODE: Yeah, and the same with the second
one.
DR. APOSTOLAKIS: How does this affect the SDP?
We're talking about the --
MR. ECKENRODE: Well, what we're trying, what
we're hoping to do, and we -- again, we're very preliminary
on this. We haven't gone very far. We're hoping to be able
to identify those kinds of things that -- in other words,
where information is missing, it's less significant, than if
it's incorrect.
MR. TRIMBLE: In one case, you might come out with
no color, and in the other case, a color.
MR. ECKENRODE: Yes.
DR. APOSTOLAKIS: So, the SDP would assign the
weight --
MR. ECKENRODE: Yes.
DR. APOSTOLAKIS: -- and make correct --
DR. WALLIS: So, what you're really implying is a
measure of control capability and its correctness and the
correctness -- it's a measure of this. And you're saying
that on a scale, plus is better than minus, and zero is in
between.
MR. ECKENRODE: Yes.
DR. WALLIS: But, you're implying there's a
measure of these things. So, you have to have that measure.
MR. ECKENRODE: Well, that's what we're trying to
hope to get. The second one is the -- anything less than a
complete failure to perform an action may not be a risk
significant as a complete failure.
DR. POWERS: I wish you would explain to those
people that do severe accidents that there is that
possibility, because they keep analyzing accidents that the
operator seemed to have died or done something, and don't
turn on water unless there's a complete possibility of
saving the core. And I think I'm very much -- I would like
to see it tested.
MR. ECKENRODE: And then, finally, we're hoping to
use the information from -- actually from the second view
graph, the guidance there from Brookhaven changes the risk
important human actions, which is based on Reg Guide 1.174.
They have defined a series of significant risk -- they call
them risk important actions -- human actions and this is all
in the operator area -- well, no, I guess not totally
operator area. It defines certain risk important human
actions and we basically have said if they fail to do these,
they are more significant than others. And we're looking
into this as a possibility to feed into the other part here.
DR. POWERS: Have we gotten that Brookhaven
report?
MR. ECKENRODE: We have the draft. I don't know
where it stands.
DR. POWERS: We haven't seen it.
MR. ROSENTHAL: Would you like it?
MR. ECKENRODE: I have one more --
MR. ROSENTHAL: I'm sorry.
MR. ECKENRODE: I bet we're tolling here.
MR. ROSENTHAL: I'm sorry; I apologize. I
apologize, Dick.
MR. ECKENRODE: That's okay. Finally, we have
been asked to make sure we consider requal -- or a requal
problem, because you actually have a requal inspection
procedure. And this is one we're actually are doing fairly
well at the moment. We decided these are the qualities we
want to look at or the areas we want to look at, both the
written JPM and the simulator, but somewhat different
things. The quality of the exam, itself, would be looked
at, and that's one of the things that's inspected in the
requal inspection. Security problems, we've had these with
the written exams and have some significance. And, then, of
course, the number of failures, one person fails is not as
significant as 20 of them failing. We're using this as one.
And the same with the JPMs.
In the simulator area, actually, the simulator,
itself, the quality of scenarios that are being written for
the simulator tests. And then in the operational test,
itself, whether it's just a single crew that fails or it's
mobile crews. You consider -- obviously, you consider
mobile crews being more significant than a single crew. And
then, as David mentioned earlier, remediation, we're going
to go back on shift with or without remediation. We
consider it to be quite serious.
DR. WALLIS: You use the word "quality" three
times here. I wondered if you know what that means, in
terms of some explanation, or is it just implied that
everybody knows that quality is.
MR. ECKENRODE: The requal exams, of course, are
written by the licensee. We review the exams before they
are given. And in quality there, you can have quality both
from the content and from the psychosometric --
DR. WALLIS: Quality is somebody's judgment.
MR. ECKENRODE: Yes.
MR. BARTON: Eyes of the beholder.
MR. ECKENRODE: Well, it's not quite in the eyes
of the beholder. Some of them are actually incorrect
questions or incorrect, and it's how much we have in that.
So --
MR. TRIMBLE: There certainly is a degree of --
degree of subjectivity. What we were trying to capture is
what -- if the processes for assessing the operators are
broken, then your assurance that their -- is, therefore,
reduced, that they're going to take the right actions on
shift and post accident initiation. So, we thought that's
--
DR. WALLIS: I think in terms of these qualities,
particularly the second one, it might be worthwhile to break
it down, what is it you're trying to measure.
MR. ECKENRODE: Well, I'm sure we will. We just
haven't gotten that far yet.
DR. POWERS: I wonder if you thought about --
well, you call that failure, especially with respect to the
multiple crew failures under the simulators. Suppose I
didn't have any failures, but all my crews kind of get -- I
mean, they weren't really sharp. Is there a gradation in
there that need to be looked at, especially when you talk
about multiple crews. You know, they all get barely passing
versus all of them being very good. That usually doesn't
happen though.
MR. TRIMBLE: When we first started down this
track, we were saying, well, you know, if you find
performance issues in training, then that might have been --
indicate that people that were on shift, there was a risk.
They didn't know how to do things while they were on shift.
Then the comment came out, wait a minute, that's what
training is for. You're supposed to find these things and,
therefore, the discussions began to turn towards -- the
problem really is if they return to shift with the problem
uncorrected, and that's why we got focused in that
direction.
MR. BARTON: The training program is pretty solid.
The licensee offered it in its program. When you get the
scenario you just described, I would start looking at the RC
examiners.
DR. POWERS: Well, I mean that -- I presume that's
an outcome --
MR. BARTON: sure.
DR. POWERS: -- on this. Not holding your feet to
the fire is the right way or something. I mean, I think
that's -- that is a legitimate outcome in the significance
determination process.
MR. ECKENRODE: And those are the things that
we're doing in the assessment process.
DR. POWERS: I am quite excited about your testing
the hypothesis that you've settled on and I sympathize with
your struggling with this significance determination
process, because I sure don't know it.
DR. APOSTOLAKIS: Let me ask two questions before
you go onto the future activities. We were told this
morning that the human performance, in the context of spent
fuel pool liability, is different from other things that we
study now, because it involves long periods of time,
seven-hour shifts, and it's primarily due to organizational
failures, the speaker speculated. Where, in this program,
are you investigating this? In other words, a similar
situation is during the transitions doing low power and
shutdown operations, where our colleagues hear from the
utility sub telling us there are multiple human actions,
sometimes without guidance, and it's public information and
so on. Is there a methodology within the program that
identifies this different areas, so that proper attention
will be paid to them? It seems to me, we have, all of us,
jumped onto the ATHEANA bandwagon without realizing, at the
time, that this is an important class of human actions, but
it's just one class. There are other classes, where perhaps
we should do other things. Is there a mechanism within the
program for identifying those areas or we'll find them as we
go?
MR. ECKENRODE: I don't know what inspection
procedures are out there. There must be some in that area.
I'm not aware of -- you know, I can't identify them. But,
there must be inspection procedures in the area.
DR. APOSTOLAKIS: Or transition. Jack, are you
aware of any during transition periods?
MR. ROSENTHAL: We need to get back to -- about
the time of the -- Millstone was very popular. We did a --
mailed a -- DSEF did a report on spent fuel pool cooling and
there are two classes of events: one was loss of heat
removal and the other was loss of inventory due to actions,
deflate a seal or open -- and we thought that the inventory
ones had received -- perhaps received less attention than
they might have and that the ones with loss of heat removal,
yeah, that was multiple days long. And that report got a
fair amount of reviews. So, we'll have to figure out where
it falls.
The other question, I think, Nathan and I have to
work together and get back to you.
MR. SIU: Yeah, it's a fair question. One of the
things that we're planning to do this summer is to put
together a white paper that talks about what are the
different methods that have been developed, where do they
stand, and some sort of framework. I'm not sure what that
framework would be. Obviously, we have to cover different
modes of operations, one way of looking at the problem.
Yeah, we have to get back to you, in terms of
identifying a -- part of a program that would regularly
examine these areas and make sure that we're doing things
that we need to do.
DR. APOSTOLAKIS: But shouldn't the program,
though, have some sort of an activity that would search for
these different situations and -- as part of something?
Participatory or imagination or something, while you're
trying sort of a systematic approach -- not a systematic
approach -- but identify those different things. And maybe
you ought to put some box there. The inputs to the box can
be somebody's analysis or can be something from the ASP
work. I don't know. But, there ought to be, I think, for
identifying these, because we can't have this. We can't be
not surprised, but be faced with a new situation, where the
guys who did the analysis tell us, you know, we can't use
ATHEANA, because ATHEANA is not for this thing. My God, how
much money have we spent on ATHEANA? And now you're telling
me we can't use ATHEANA? You see, that's the -- anyway,
enough said.

DR. WALLIS: Can I ask a question for you?
MR. ROSENTHAL: Yes, of course.
DR. WALLIS: I'm a little puzzled. I've heard two
very interesting presentations and I was asking myself, do
these complement each other? Does one fit in with the other
and provide something, which is missing in the other? I
don't see how they fit together.
MR. ECKENRODE: We -- in the areas that we work
in, if we need research done, basically, we send a user need
over. And we just did, as a matter of fact, this past
weekend in the oversight area.
DR. WALLIS: When you made your presentation, you
didn't emphasize how it fitted in, in some key way, to --
MR. ECKENRODE: No, I did not.
DR. WALLIS: -- what you were doing.
MR. ECKENRODE: What I saw, the gentleman that was
sent to me said they wanted to hear -- they wanted to hear
about the oversight activities in NRR and that's why --
DR. WALLIS: These are some -- two independent
presentations?
MR. ECKENRODE: They're connected by that chart.
Now, you have to understand, the -- some of the areas of the
emerging issues that you're talking about, those are all
human needs within NRR, request for none.
DR. WALLIS: I know.
DR. POWERS: Let me ask a question about user
need, in connection with areas. Episodically, NRR has to
make the decision on whether to allow manual actions on some
safety system, where it has to be automated and to make that
decision, presumably you have to know how much time is
available. I understand that in your plan, you have an
issue in that area?
MR. ECKENRODE: Yes, they do.
DR. POWERS: Can you tell me what that is? Are we
collecting some data on this? I mean, we've had data, but
it's been proprietary.
MR. ROSENTHAL: You want to get back to 58.8. I'm
going to ask Jay -- we have a study going on and to tell you
the truth, I'm a mere puppet. And Jay Persinsky is my
expert.
MR. BARTON: Jay pulls the strings?
MR. ROSENTHAL: Yes.
MR. PERSENSKY: Not very effectively at all times,
unfortunately. The initiative that's mentioned in the
program, I mentioned briefly when we were talking about 58.8
and B-17 and the closure of B-17. That particular program
is one that is addressed right now more at methodology,
rather than data. The intent is not, as I indicated, at
this point, to endorse 58.8 or the data within it. It's
primarily to come up with a risk informed method for the
plants to, in fact, determine the times that would be used
in the -- for their situation. So, it's very plan specific;
but it's more in that a methodology, as opposed to a --
collecting a lot of data on that.
DR. POWERS: You told me the times had passed us
by, but now we're getting to a realistic point --
MR. PERSENSKY: Correct. And, in fact, Dick
mentioned in the SDP process, the use of a method developed
by BNL for -- based on Reg Guide 1174, for the SDP process.
Well, that method, in fact, was determined and developed for
this operation action time project. So, in fact, there is a
lot of interaction.
The supplemental inspection that Dick talked
about, in fact, we sat around the table for several days
working together, to develop that, as an in-house effort.
So, there is interaction in that sense. Dr. Apostolakis
asked a question about, you know, why these particular
topics. Well, these particular topics have shown up in
various other activities that we've done, either through our
various research topics, our interactions with other
agencies, in terms of these are standard -- very standard
human factors, categories, or whatever term you want to
assign to those topics, as far as what it is that affects
human performance. So, these are all things that come out
of our interaction with the user.
DR. POWERS: Well, in response to your response, I
think it's exciting what you're doing to a risk informed
selection between manual and automatic operations. That
helps a lot in their -- I think when you -- a paragraph to
that affect in the plan would make other people
enthusiastic, as well.
MR. ROSENTHAL: I just want to pick up a couple of
points. ACRS review, yes, the ACRS does influence course of
actions. They are great proponents for us and more risk
informed and I think we're going in the direction that
you've -- that you've indicated in the past. We have not
achieved the level of being as risk informed as we would
like, but we will continue on.
The last thing I want to bring up is actually an
idea that -- of Jay Persinsky's and it's in the plan. Just
like you have thermal hydraulic PIRT meetings, I think we
recognize, with some modesty, that we could use some expert
assistance and we intend to have actually this fiscal year
an experts meeting, to help guide the details of future
human performance work. We would, also, plan to have a
stakeholders meeting; but, in my mind, you know, a public
meeting. Those are -- both would be public. But the
stakeholder meeting and the expert meeting would be
different meetings.
DR. POWERS: With the intention of the expert
meetings to deal with things -- apparently what you've got
or suggest other things that --
MR. ROSENTHAL: Actually, I'm more interested in
where should we go in the future. Ideas are precious
things.
DR. APOSTOLAKIS: Who owns this program?
MR. ROSENTHAL: We.
[Laughter.]
MR. ECKENRODE: The NRC.
DR. APOSTOLAKIS: Yeah, the American taxpayer.
Who is the guy, who would say, hey, I'm not doing anything
here or should, because I would be embarrassed when I have
to defend this? If it's a committee, that doesn't help. Is
it Jack? I think -- years ago, we hold the letter that it
was critical for the human performance program to have a
guiding charge. We've gone there. So, it's you.
MR. ECKENRODE: Yes, it definitely is.
[Laughter.]
DR. APOSTOLAKIS: Okay. I have no more questions.
MR. TRIMBLE: I can't leave Jack hanging out
there.
MR. ROSENTHAL: It's fine; that's fine. The
refitness program has been in NRR one time. Currently, the
DRS -- for the program are taken and we would like a -- we
would appreciate a letter.
DR. WALLIS: I think it's being thrust upon you.
DR. POWERS: Without debating that any further,
thank you, Jack. Thank you, John. We have a lot of work to
do ahead in the time, because -- because Apostolakis is not
going to be with us tomorrow afternoon.
DR. APOSTOLAKIS: No. So, I have to be here when
you discuss the first two items.
DR. POWERS: And that is the intention, is to do
the first two items. What I'd like to do, I think, is to
take a recess for 10 minutes and then come back and have a
first reading and discussion of the joint letter, and then
discuss --
DR. APOSTOLAKIS: I have some preliminary stuff.
DR. POWERS: Okay, discuss the -- we have a
performance issue. And then as time available, we will
discuss the spent fuel pool risk assessment. And Don has a
fourth draft of his safety policy letter, so maybe we can
get a first reading on that, as well. Okay, yes -- and we
no longer need the transcription.
[Whereupon, the recorded portion of the meeting
was recessed, to reconvene at 8:30 a.m., Thursday, April 6,
2000.]

Page Last Reviewed/Updated Tuesday, July 12, 2016