Performance of Laboratories in Testing for Rabies Virus -- United States

In response to a recommendation by the Executive Committee of the Association of State and Territorial Public Health Laboratory Directors for an external assessment of the current status of rabies testing, a special laboratory performance evaluation and training exercise was developed at CDC. The objective of the performance evaluation component was to detect problems in performing the fluorescent rabies antibody (FRA) test so that immediate on-site assistance could be directed to laboratories needing consultation or training. This approach to problem identification and resolution has been termed, "Competency Assurance Through Monitoring and Assistance (CATMA)." The exercise was designed to consist of an initial shipment of 10 slides, with duplicate brain impressions on each slide, and a second shipment of 10 slides to only those participants who reported results that were not in agreement with the expected results of the initial shipment. Rabies virus was inactivated by gamma radiation (1-2), so no infectious materials were distributed.

A total of 136 public health laboratories were enrolled in the program: 46 main state health department laboratories, 41 branch or regional state health department laboratories, 38 county health department laboratories, six city health department laboratories, and five state university veterinary laboratories. One hundred twenty-nine (95%) of the 136 laboratories participated in the first shipment.

Performance evaluations were based on the interpretation (positive or negative) reported for each sample by four CDC reference laboratories. Full credit was given for each interpretation agreeing with the reference laboratories' interpretations, and no credit was given for interpretations that were in disagreement. Participants who reported interpretations (in the initial or repeat shipment) for all samples that were in agreement with the reference laboratories' interpretations were awarded a certificate of achievement. In the repeat shipment, participants who reported one or more interpretations that did not agree with the reference laboratories' interpretations were contacted and offered on-site consultation and training services by either CDC or state laboratory personnel. After the consultation and training, participants were given the opportunity to examine another set of slides from CDC and receive certification, if successful.

The initial shipment of slides was made on October 18, 1983, and consisted of two control slides (one strongly positive and one negative) and unknown slides (three strongly positive, two weakly positive, and three negative). A follow-up shipment to 41 participants who reported discrepant results in the first shipment was made on January 17, 1984. The latter shipment consisted of two control slides (one strongly positive and one negative) and unknown slides (one strongly positive, four weakly positive, and three negative). The reference laboratories reported discrepant results with one of the weakly positive slides at the time of shipment; therefore, participants were not held responsible for their interpretations with this sample. In the initial shipment, 41 (32%) of 129 participants reported discrepant results (Table 5). Six (15%) of these 41 participants reported discrepant results in the repeat exercise. In the initial exercise, 34 participants reported one discrepant result; six reported two; and one reported three discrepant results. In the repeat exercise, three participants reported one discrepant result; two reported two, and one reported four discrepant results.

A review of the discrepant results from both shipments, by type of sample, showed that one negative report of 428 reports for the four strongly positive samples represented approximately 2% of the discrepant reports and 0.2% of the reports for strongly positive samples. Thirty-eight negative discrepant reports out of 381 reports for the five weakly positive samples accounted for 63% of the discrepant reports and 10% of the reports for the weakly positive samples. Of the 500 reports for the six negative samples, 21 discrepant positive reports represented 35% of the discrepant reports and 4% of the reports for negative samples. The overall results were as might have been anticipated: the major problem identified was calling weakly positive samples "negative"; the next was calling negative samples "positive." A summary of the number and percentage of laboratories reporting any interpretations in disagreement with the expected results by the type of laboratory is presented in Table 6.

The data in Table 7 demonstrate how improvement in the performance of the FRA test might be measured by using sensitivity, specificity, predictive values, and efficiency as measures. Since no data were readily available on the actual incidence of rabies among animals in the United States, for the purpose of the demonstration, incidence in a suspected group was used. The incidence (9,247 reported rabies cases per 100,000 specimens tested) was determined with data from the CDC Rabies Surveillance Annual Summary for 1980-1982 and data from the CDC Consolidated Annual Report on State and Territorial Public Health Laboratories for fiscal year 1981. Test results from the group of laboratories reporting discrepant results in the initial shipment and those from laboratories participating in the repeat shipment were used in the calculations. The sensitivity (incidence of true positive results obtained by a test for a population known to have the disease or condition) of the FRA test in the initial shipment was 84% (Table 7); this figure increased to 96% in the repeat shipment. Specificity (the ability of a test to give a negative result in the absence of disease) of the test was 94% in the initial shipment and 97% in the repeat shipment. The predictive value of a positive test (the percentage of the positive results that are true positive) was 59% in the initial shipment and increased to 79% in the repeat shipment. The predictive value of a negative test (the percentage of negative results that are true negative) was 98% in the initial shipment and 99% in the repeat shipment. Efficiency (the ability of a test to give a positive result on positives and a negative result on negatives) of the test went from 93% in the initial shipment to 97% in the repeat shipment. The data also demonstrate that a small improvement in specificity of the test was accompanied by a large improvement in the predictive value of a positive test. Reported by Performance Evaluation Br, Div of Technology Evaluation and Assistance; Laboratory Training Br, Div of Laboratory Training and Consultation, Laboratory Program Office, CDC.

Editorial Note

Editorial Note: The organization, planning, and conduct of the exercise represented a cooperative effort between CDC's Laboratory Program Office (LPO) and Center for Infectious Diseases (CID). The performance evaluation component of the exercise was administered by personnel of the Microbiology Section, Performance Evaluation Branch, Division of Technology Evaluation and Assistance, LPO. The consultation and training component was administered by personnel of the Virology Training Section, Laboratory Training Branch, Division of Laboratory Training and Consultation, LPO, through the respective state health departments. The Production Branch, Biological Products Program, CID, furnished the brain impression slides that were distributed to laboratories; other CID personnel served as consultants for the exercise.

The CATMA program offers a useful approach for detecting problems and for targeting assistance to laboratories identified as having chronic problems with the FRA test. The CATMA approach combines the benefits of external quality assessment for a universe of laboratories with the economic benefits of immediate training and on-site consultation targeted only to those laboratories needing these services for maintaining competency in performance of the FRA test.

On-site consultations revealed a variety of technical deficiencies that could be traced to: (1) failure to follow recommended procedures; (2) insufficient quality control of reagents and of the actual test performance; (3) use of fragmented or incomplete protocols; (4) use of inadequate or obsolete microscopes; and (5) tests performed by insufficiently trained staff. In most instances, 2 days' consultation with a CDC staff member using unknown specimens was sufficient to identify the various problems and help the staff begin corrective action. A special remedial continuing education program was then conducted by means of mailed specimens and written consultations; this program enabled all laboratories receiving CDC assistance to achieve an acceptable level of performance on their second set of slides. The performance of the rabies test is essential for patient management, and all training provided was based on CDC-recommended procedures (3-5).

References

1. Gamble WC, Chappell WA, George EH. Inactivation of rabies diagnostic reagents by gamma radiation. J Clin Microbiol 1980;12:676-8.

2. White LA, Chappel WA. Inactivation of rabies virus in reagents used for the fluorescent rabies antibody test. J Clin Microbiol 1982;16:253-6.

Disclaimer   All MMWR HTML documents published before January 1993 are electronic conversions from ASCII text into HTML. This conversion may have resulted in character translation or format errors in the HTML version. Users should not rely on this HTML document, but are referred to the original MMWR paper copy for the official text, figures, and tables. An original paper copy of this issue can be obtained from the Superintendent of Documents, U.S. Government Printing Office (GPO), Washington, DC 20402-9371; telephone: (202) 512-1800. Contact GPO for current prices.

Page converted: 08/05/98

HOME  |  ABOUT MMWR  |  MMWR SEARCH  |  DOWNLOADS  |  RSS |  CONTACT POLICY  |  DISCLAIMER  |  ACCESSIBILITY

Morbidity and Mortality Weekly Report Centers for Disease Control and Prevention 1600 Clifton Rd, MailStop E-90, Atlanta, GA 30333, U.S.A Department of Health and Human Services

This page last reviewed 5/2/01