|
|
|
|
Understanding NELAC Acceptance Limits
|
Whether accredited as a National Environmental Laboratory Accreditation Conference
(NELAC) laboratory or not, just about every environmental laboratory in the U.S.
is affected by the proficiency testing (PT) requirements and criteria of the NELAC
program. While some states have specific requirements for a few unique analytes,
they all rely on the NELAC peer-reviewed PT criteria and accredited PT providers
for the majority of analytes they accredit for. Having an understanding of how NELAC
PT evaluation criteria are established is important to the laboratories and accrediting
bodies that rely on the resulting evaluations.
Some analytes have proficiency testing criteria defined in the Code of Federal Regulations
(CFR). As these criteria have been codified in U.S. law, NELAC has no ability to
make modifications to them. These analytes are generally included in the drinking
water program, with some examples being selected metals, asbestos, PCBs, various
pesticides and dioxin. For this group of analytes, the NELAC fields of proficiency
testing (FoPT) tables simply summarize the requirements of the CFR.
For analytes included in the NELAC FoPT tables that are not covered by federal requirements,
The NELAC Institute’s (TNI) PT Board is responsible for establishing acceptance
limits. The PT Board is made up of representatives from state accrediting bodies,
accredited laboratories, the federal government, laboratory consultants and PT providers.
For the purpose of creating or revising FoPT tables, the PT Board will appoint a
subcommittee with expertise in the specific analyte groups being addressed. For
example, the currently active microbiology and whole effluent toxicity FoPT subcommittees
have been formed with people who have experience in these areas and represent the
major stakeholder groups.
The PT Board attempts to review existing FoPTs on a regular basis to ensure that
the current concentration ranges and acceptance limits are producing PT data that
support the NELAC program. When petitioned by states, federal programs or other
entities, the PT Board will consider adding new analytes to the NELAC proficiency
testing program.
The process for creating or expanding PT requirements begins with a charge from
the PT Board. This charge indicates the purpose of the subcommittee – whether to
review current criteria or to consider adding new analytes – gives the subcommittee
a timeline and provides other guidelines by which the subcommittee should function.
The subcommittee will usually then request historical study data from PT providers
for the specific group of analytes being studied. As this is not a required activity,
not all providers submit this data which limits the information available to the
subcommittee from which to conduct their analyses.
If reviewing already existing PT criteria, the subcommittee will look at the means,
standard deviations and failure rates among all of the providers that submitted
data. They will use this data to determine if the current acceptance limits still
provide a good fit to the actual data being reported by laboratories. If it has
been a long time since the acceptance criteria have been established – potentially
allowing for technological advances or other improvements to increase the accuracy
and/or precision of the laboratory community – or if the original acceptance limits
were calculated from unknown or inappropriate data, the subcommittee may find that
they need to revise the acceptance limits. There were many instances of this in
2005 when the PT Board issued the results of the first significant review of WP
and WS acceptance limits originally generated by EPA. Additionally, some acceptance
limits were found to provide an appropriate evaluation of the more recent data.
In these cases, the acceptance limits can remain unchanged.
If a subcommittee is given the task of determining whether to add new analytes to
the PT program, they will follow a similar process. In these cases, even though
the analytes are not already included in the TNI PT program, there is generally
a subset of the PT providers who have been providing PT samples in this area. Whether
to fulfill the needs of a specific state or demand from laboratories, these providers
have been providing these samples and using internally developed performance evaluation
criteria. Again, on a voluntary basis, these providers will be asked to supply this
data to the subcommittee. To supplement these data, the subcommittee will generally
research any other sources of performance data including analytical methods, method
validation studies, international round robins, and others.
The TNI PT program has minimum requirements for the amount of data that must be
available in order for new analytes to be added to the NELAC FoPT tables used for
accreditation purposes. In order to allow for the collection of additional data
for those analytes that don’t meet this threshold, the PT Board has the ability
to add new analytes to experimental FoPT tables. If a laboratory is accredited for
an analyte that is listed on an experimental FoPT table, they must analyze the associated
PTs twice per year, but their performance on these PTs, acceptable or not acceptable,
can not be used by their accrediting body to suspend or revoke their accreditation.
The AB can only take
action if the lab does not analyze and submit data for the
PT. There has been much discussion at recent NELAC meetings and within the PT Board
and PT Committee about the pros and cons related to experimental FoPT tables, so
some modification to the current practice is likely on the horizon.
If an analyte does meet the minimum data requirements and the FoPT subcommittee
determines that appropriate acceptance limits can be developed, they will provide
these in a recommendation to the PT Board. The process for approving any recommendations of the FoPT subcommittee is first, a vote of acceptance from the PT Board, and then
submission to the NELAP Board for their approval. Only upon consent of the NELAP
Board can revisions to existing or the creation of new FoPT tables be finalized.
The acceptance limits used in the NELAC program fall into three categories; regression
equations, fixed limits and study-derived limits. For the Water Supply program,
the acceptance limits roughly evenly split between regression equations and fixed
limits. For Non-Potable Water, the vast majority of limits use regression equations.
In the Solid and Chemical Materials FoPT tables, both regression equations and study-derived
limits are used – with some analytes using both.
Regression equations consist of two straight line equations – one for the study
mean and one for the study standard deviation – which are calculated using the historical
data submitted by PT providers. Various controls are put in place to ensure that
outlier data are removed and that the resulting line produces an acceptable fit
to the data set. These limits attempt to take into account changes in bias and precision
that can occur at the extremes of an analyte’s concentration range. For example,
the expected mean recovery for Non-Potable Water thallium is 94.1% at 60 µg/L and
is 100.6% at 900 µg/L. The acceptance limits at these two concentrations are ±61.6%
and ±17.6%, respectively.
Fixed limits are a single percentage range that is applied to the analyte regardless
of the assigned value. Thallium in the Water Supply program, for example, has acceptance
limits of ±30% over the concentration range of 2 to 10 µg/L. In order for fixed
percentage acceptance limits to be used, historical PT data should indicate that
there is no perceptible variation in bias or precision over the concentration range
of the analyte. Many of the federally mandated acceptance limits are fixed percentages.
Study-derived acceptance limits are generated from each study’s actual mean and
standard deviation. These limits are generally used in situations where a definitive
true value cannot be established, such as for soil, or when the historical data
show that precision is neither consistent nor linear across the needed concentration
range. Many analytes in the solid and chemical materials FoPT table use a combination
of study-derived means and standard deviations for setting acceptance limits. Following
a recent review of historical soil PT data, the subcommittee determined that,
while the use of actual study means was still necessary in many cases, there was enough
consistency in the standard deviations that regression equations can be used for
to calculate this element of acceptance limits for several analytes.
To learn more about The NELAC Institute and potentially be involved in the process
of reviewing and creating fields of proficiency testing tables, visit www.nelac-institute.org.
If you have any questions about ERA’s PT studies, please visit www.eraqc.com or
call us at 1-800-372-0122.
|
|
|
|
|