Designing a Quality Control Strategy

In the modern laboratory three questions must be answered.

By Curtis A. Parvin, PhD, John Yundt-Pacheco and Max Williams

Quality Assurance Series

Editor's note: This is the third in a multi-part quality assurance series.

Many of the ideas about how to design good quality control (QC) strategies to meet the needs of a laboratory were formulated in an era when most lab testing was performed in batches. In this setting, both patient specimens and QC samples were included in each batch of testing. The QC sample results were used to decide whether the patient results in the batch were acceptable. If the QC results were deemed acceptable, it was concluded that the patient results in the batch were also acceptable. If the QC results were unacceptable, it was assumed there was a problem with the batch that was adversely affecting the QC samples and patient specimens.

In the batch testing setting there is a natural association between the set of QC samples and the set of patient specimens that make up the batch (Fig. 1). With batch testing there are two questions that must be answered to define a QC strategy-how many QC samples to include in the batch and what QC rule(s) to apply to the QC sample results to decide whether the batch is acceptable.

Traditional approaches to designing QC strategies focus on finding answers to these two questions that will provide the statistical power needed to detect a "critical" out-of-control error condition in a batch. For instance, Westgard proposes tools such as power function graphs and Op-Spec charts to help find a QC strategy that has a high probability of giving a QC rule rejection when a "critical" out-of-control error condition exists.¹

Today's QC Strategy

In the modern laboratory, the majority of instruments perform discrete testing. With automated discrete analyzers there is no longer a natural association between a set of QC results and a batch of patient specimens. Instead, QC results simply reflect the status of the test system at a point in time when the QC samples are tested. If the QC sample results are unacceptable, it suggests that a problem has occurred sometime earlier. This implies that if the laboratory doesn't do something to correct the problem then most assuredly future patient results will be adversely affected, but it doesn't give the laboratory any information about how many previous patient results were adversely affected (Fig. 2).

Thus, there are three questions that must be answered to define a QC strategy in the modern laboratory:

1. How many QC samples should be tested at a point in time?
2. What QC rules should be applied to the QC sample results?
3. When should QC testing occur?

Finding good answers to the "when" question is one of the hot topics in modern QC strategy design and there are myriad opinions concerning what is minimal and ideal.

Determining 'When'

In our last article in this series, we argued that the primary focus of laboratory QC should be the patient.² In this vein, decisions about when to test QC samples should be made based on the impact those choices will have on the risk of producing unreliable patient results.

In April, Bio-Rad hosted a Convocation of Experts on Laboratory Quality in Salzburg, Austria. The participants were divided into five working groups that addressed different issues in laboratory quality. One of the workgroups considered the issue of when QC testing should be performed. They concluded that QC testing should be performed any time an event occurs that has the potential to adversely affect the testing process (e.g., when reagent lots change, when test system maintenance occurs or when calibrations are performed). If these events are planned and scheduled, QC should be performed prior to the event and again after the event.

Testing QC samples just prior to the event provides the laboratory a level of assurance that the patient results produced since the last QC testing up to the time of the event are acceptable. Testing QC samples immediately after the event gives the laboratory a level of assurance that the test system is in control prior to resuming the testing of patient specimens. In the case of an unplanned event (such as a system failure), there is no opportunity to do QC testing just prior to the unplanned event. In these cases QC testing should still be done immediately after the event to assure that the testing process is operating correctly before continuing with patient testing.

Determining Frequency

How does the lab decide how frequently QC evaluations should be routinely performed? During these intervals, if a test system malfunctions it is not associated with any notable event. Therefore, the laboratory needs to schedule QC evaluations in such a way as to minimize the risk of too many patient results being produced and reported before the laboratory becomes aware of the system malfunction. All too often, lab managers only look only at what the regulations dictate and their staff competencies when scheduling QC evaluations.

For the high-performing laboratory, a place to start is to consider really big out-of-control error conditions. When there is a failure causing a major error, all the patient specimens tested while the failure is present will be unreliable and the error will be detected (because it is so large) at the next scheduled QC evaluation.

In Fig. 3, each vertical line represents a patient specimen being tested. Asterisks denote unreliable patient results produced during the existence of the out-of-control error condition, each diamond represents a routinely scheduled QC evaluation, and a red diamond means the QC results are rejected. In the worst case, the failure occurs right after the last successful QC evaluation-then all the patient specimens between the last successful QC evaluation and the QC evaluation that detects the problem are unreliable. The best case scenario is when the failure occurs just before the QC evaluation that detects the problem; no patient results are compromised. If we consider that a test system failure can begin at any specimen with equal probability, then the expectation is that half the number of patient specimens tested between QC evaluations will be affected in the event of an undetected test system failure. In this case, the number of unreliable patient results produced has little to do with the number of QC samples tested or the QC rule used, but is directly related to the number of patient specimens tested between routinely scheduled QC evaluations.

What happens if results are held until the next QC evaluation? Instead of all the results being compromised, the error would have been detected, corrected and the patient specimens reprocessed before they were reported. Holding patient results until the test method has been checked with a subsequent QC evaluation is one of the best ways to prevent a test system failure that occurs after the last successful QC evaluation. Unfortunately, holding results may not be logistically possible. If results must be released as soon as they are produced, careful thought should be given to the number of unreliable reported patient results that can be tolerated in the event of a test system failure. The number of patient specimens tested between QC evaluations should be selected so that the expected number of unreliable patient results produced during an undetected test system failure is no larger than the tolerable threshold. We have published one possible approach to systematically determining limits on the number of patient specimens that can be tested between QC evaluations to control the expected number of unreliable patient results reported during the existence of an undetected test system failure.³

Dr. Parvin is manager of Advanced Statistical Research; John Yundt-Pacheco is Scientific Fellow; and Max Williams is Global Scientific and Professional Affairs Manager, Bio-Rad.

References

1. Westgard JO. Assuring the right quality right. Madison WI: Westgard QC Inc., 2007.
2. Parvin CA, Yundt-Pacheco J, Williams M. The focus of laboratory quality control: Why QC strategies should be designed around the patient, not the instrument.ADVANCE for Administrators of the Laboratory 2011;20(3):48-9.
3. Parvin CA. Assessing the impact of the frequency of quality control testing on the quality of reported patient results. Clin Chem 2008;54:2049-54.

Copyright 2015 Merion Matters. All rights reserved.