Learning from Laboratory Failures

Monitoring test system failures and QC performance can help identify opportunities for improvement

By Curtis A. Parvin, PhD, John Yundt-Pacheco and Andy Quintenz

In previous articles we have discussed how to design a quality control (QC) strategy,¹ how frequently QC testing should be executed,² how statistical QC and risk management principles can be used to limit the number of unreliable patient results that are reported,³ and steps to consider when recovering from an out-of-control condition.⁴ With these tools, the laboratory should be able to design a risk management strategy that minimizes the number of out-of-control conditions that occur, a QC strategy that minimizes the number of unreliable patient results produced and reported as a result of an out-of-control condition, and a recovery strategy that identifies and corrects unreliable reported patient results in a timely fashion that minimizes patient exposure to hazardous situations. How can the diagnostic process be further improved? By learning when things don't go right.

Failures Will Occur

Despite best intentions, out-of-control conditions will occur and it is the lab's responsibility to deal with them in a manner that minimizes patient harm. Sometimes out-of-control conditions occur because of manufacturing defects in reagent systems, for example, microbial contamination in some small number of reagent packs. Other times mechanical or electronic failures or other environmental factors cause undetected failures in the test system. When failures occur, in addition to detecting and correcting the failures along with any unreliable patient results produced because of the failures, a good QC and risk management program also investigates failures to identify potential opportunities for improving the overall quality of the total test system.

The building blocks for the design of a good QC system to ensure that reliable patient results are produced and reported are

• identify failure modes that create hazards
• evaluate the risk of each hazard
• devise a QC plan to mitigate hazards and recover from failures when they occur
• continually monitor the clinical diagnostic process to evaluate the failures that occur
• assess when it is necessary to re-evaluate hazards, risk or the QC plan.

Each failure provides a new opportunity to strengthen the quality of the clinical diagnostic process and ensure the reliability of future results. This monitoring aspect is referred to as post-implementation monitoring in the CLSI EP23-A guideline.⁵

Data Useful for Improvement

One of the most valuable sources of data for identifying opportunities for improving the quality of the total test system are data generated from failures. Some failure data are fairly easy to obtain - such as the frequency of a specific failure mode or the cumulative frequency of all failure modes for a given testing process. How often are QC rejections generated? How often is calibration required? How often are patient specimens reevaluated? These data should be recorded in a fashion that facilitates analysis. Basic failure rates should be recorded for all analytes, summarized and monitored on a regular basis.

Other critical data on failures are not so easily obtained, especially failures that may persist for an extended period. These types of failure modes have been referred to as persistent out-of-control conditions⁶ and large-scale testing errors.⁷

Questions for the laboratory to address in these cases are:

• How large was the failure (magnitude of the out-of-control condition)?

• When did the failure occur?

• How many patient results were affected by the failure?

An effort should be made to estimate the type, magnitude and duration of a detected out-of-control condition to facilitate a timely and effective recovery from the failure. As a laboratory accumulates data over time regarding the types and magnitudes of out-of-control conditions that have occurred they are in a better position to design QC strategies targeted toward those types of failures.

By collecting and recording the details of each out-of-control condition, the laboratory builds information infrastructure needed to trigger a reassessment of how the laboratory is managing the production of reliable patient results for a clinical process. If the initial risk assessment estimated that a given failure mode was likely to occur once every 2 years and it has occurred 3 times this year, a re-evaluation is in order. The acceptability of the estimated number of unreliable results produced because of an out-of-control ­condition will likely need to be questioned if the out-of-control condition is occurring more frequently than anticipated. The frequency of QC events may need to be increased to detect the out-of-control condition sooner to reduce the impact of the failure, or the QC rule may need to be modified in favor of a QC rule with greater error detection power.

Another benefit that can occur from analyzing the details of failures is the discovery of a hazardous situation that had not yet been identified in the laboratory's risk assessment and addressed in its current QC plan. The newly identified hazard may suggest an additional critical control point in the testing process. Consider this scenario: The QC results from the laboratory's last QC event were on the high side, but still acceptable so patient specimen testing continued. A calibration is performed prior to the next scheduled QC event. QC results are obtained immediately after the calibration. The QC results are acceptable and patient specimen testing continues. Unfortunately, several complaints from treating physicians question the integrity of some of the recent patient results. After investigation, it is determined that a number of the specimens evaluated just prior to the calibration had unacceptably high results. The most likely explanation is that an undetected out-of-control condition existed prior to the calibration that was corrected by the calibration and, therefore, never identified by the lab. To mitigate the likelihood of this hazardous situation occurring in the future, the lab adds a new critical control point. A QC event is scheduled immediately before each calibration. By doing so, the laboratory increases its chance of detecting any out-of-control conditions affecting patient specimens examined since the last accepted QC event before the test system's state is changed by the new calibration.

In the prior scenario, an opportunity to reduce patient risk was accomplished by adding a new critical control point to the QC testing plan. Risk might also be reduced by decreasing the likelihood that an incorrectly reported patient result will lead to patient harm. Assume a hospital has an ED policy to discharge patients presenting with chest pain if they have a negative ECG and a cardiac troponin result below the cutoff. The lab is notified that several patients have re-presented and found to have had MI events. During investigation, the lab discovers that an instrument failure led to false negatives. Due to the patient impact, in addition to other corrective action, the multi-discipline team decides to reduce the likelihood of causing patient harm due to a false negative troponin result by changing discharge requirements to include serial negative troponin results and modifying its QC procedures to assure that any out-of-control condition in troponin testing is detected sooner.

Dr. Parvin is manager of Advanced Statistical Research; John Yundt-Pacheco is Scientific Fellow; and Andy Quintenz is Global Scientific and Professional Affairs Manager, Bio-Rad.

References

1. Parvin CA, Yundt-Pacheco J, Williams M. Designing a quality control strategy: In the modern laboratory three questions must be answered. ADVANCE for Administrators of the Laboratory 2011;20(5):53-4.
2. Parvin CA, Yundt-Pacheco J, Williams M. The frequency of quality control testing. QC testing by time or number of patient specimens and the implications for patient risk are explored. ADVANCE for Administrators of the Laboratory 2011;20(7):66-9.
3. Parvin CA, Yundt-Pacheco J, Quintenz A. Statistical QC & risk management. The combination can improve the overall quality of patient results. ADVANCE for Administrators of the Laboratory 2012;21(8):35-7.
4. Parvin CA, Yundt-Pacheco J, Williams M. Recovering from an out-of-control condition. The laboratory must assess the impact and have a corrective action strategy. ADVANCE for Administrators of the Laboratory 2011;20(11):42-4.
5. CLSI. Laboratory Quality Control Based on Risk Management; Approved Guideline. CLSI document EP23-A. Wayne PA: Clinical and Laboratory Standards Institute; 2011.
6. Parvin CA. Assessing the impact of the frequency of quality control testing on the quality of reported patient results. Clinical Chemistry 2008;54(12):2049-54.
7. Valenstein PN, Alpern GA, Keren DF. Responding to large-scale testing errors. American Journal for Clinical Pathology 2010;133:440-6.

Copyright 2015 Merion Matters. All rights reserved.