GRADE: Use of Smallpox Vaccine in Laboratory and Health-Care Personnel at Risk for Occupational Exposure to Orthopoxviruses

Printer friendly version of this page [10 pages]

Background

ACAM2000, a live vaccinia virus vaccine, is the only smallpox vaccine currently licensed and available in the United States (U.S.) for vaccination of persons at risk for orthopoxviral disease. ACAM2000 is produced in Vero cells and derived from a clonal isolate of Dryvax, the New York City Board of Health strain widely used during the smallpox eradication campaign [1-5]. Like Dryvax, ACAM2000 is administered percutaneously using a bifurcated needle, and comes with potential risks of serious adverse events [6-7]. Recommendations of the U.S. Advisory Committee on Immunization Practices (ACIP) regarding smallpox (vaccinia) vaccination, most recently revised in 2003, specify Dryvax as the designated smallpox vaccine in routine non-emergency vaccination programs, which primarily involve laboratory and health-care personnel, as well as select military personnel [8-9]. However, as the license for Dryvax was withdrawn in 2008, and remaining vaccine supplies were subsequently destroyed, the need to develop new ACIP recommendations based on ACAM2000 is paramount [10]. Thus, the ACIP Smallpox Vaccine Work Group has applied the GRADE framework to the available evidence evaluating the administration of ACAM2000 in laboratory and health-care personnel at risk for orthopoxviral disease due to occupational exposure.

 

Policy Question

Should administration of ACAM2000 be recommended routinely for laboratory and health-care personnel at risk for occupational exposure to orthopoxviruses?

Top of Page

PICO of Interest

The population, intervention, comparison, and outcomes (PICO) of interest was defined as follows:

Population (P): Laboratory and health-care personnel at risk for occupational exposure to orthopoxviruses

Intervention (I): Vaccination with ACAM2000

Comparison (C): Vaccination with Dryvax

Outcomes (O): ACIP workgroup members compiled an initial list of relevant outcomes to consider, which included both beneficial and harmful outcomes. A modified Delphi process was then utilized to rate the importance of each outcome. Members of the workgroup used these results to compile a final list of outcomes to be considered (Table 1).

Table 1: Results of survey and identified critical outcomes: ranked in decreasing order of importance

Outcome	Mean Importance Rating(Range)	Standard Error of Importance Rating	Include inEvidence Table? 1, 2
1. Death	8.3 (6-9)	0.29	Yes
2. Postvaccinial encephalitis	7.5 (4-9)	0.35	Yes
3. Eczema vaccinatum	7.4 (4-9)	0.38	Yes
4. Myo/pericarditis resolved with sequelae	7.3 (4-9)	0.41	Yes
5. Progressive vaccinia	7.2 (4-9)	0.36	Yes
6. Cutaneous response	6.1 (2-9)	0.72	Yes
7. Generalized vaccinia	5.8 (1-9)	0.74	Yes
8. Inadvertent inoculation	5.6 (2-9)	0.71	Yes
9. Myo/pericarditis resolve without sequelae	5.5 (1-9)	0.61	Yes
10. Neutralizing antibody response	5.4 (1-9)	0.72	Yes
11. Mild adverse events / injection site reactions	4.2 (1-9)	0.79	Yes

Table 1 Footnotes

¹ Outcomes with importance ratings of 1-3 are generally considered not important and not included in the evidence tables; ratings of 4-6 are considered important but not critical for making a decision; ratings of 7-9 are critical for making a decision and are included in the evidence tables.  Evidence tables are generally limited to 7 outcomes, and therefore a combination of critical and/or critical but not important outcomes may be considered [11].

² The outcomes assessed by the workgroup included both critical and important outcomes. The workgroup felt that all of the rated outcomes should be assessed and considered when making their recommendation. Therefore, in order to keep the number of total outcomes assessed to 7, the workgroup decided to combine outcomes that are normally classified as serious adverse events (SAE) into a single category, which included the following SAE: death, postvaccinial encephalitis, eczema vaccinatum, progressive vaccinia, and generalized vaccinia. Additionally, mild adverse events (MAE) were combined to include the following: those adverse events which were not previously identified as a SAE, inadvertent inoculation, and myo/pericarditis resolved with or without sequelae.

Top of Page

Evidence Retrieval, Assessment and Synthesis: Included Studies

Studies were found which provided a direct analysis of the intervention and comparison. Two reviewers selected studies in two stages: review of titles and abstracts, followed by a review of full-text articles. Studies not directly relevant to the policy question were eliminated and included a variety of records and studies involving: animals, primary molecular investigation, vaccines not of direct interest, reviews, position papers, issue briefs, and meeting notes. Any discrepancies were resolved through discussion between the two reviewers. In all, five studies were included in the analysis [1-2, 12] (Table 2)

Table 2: Characteristics of Included Studies

Author, Year (Study)1	Participants	Intervention	Reported outcomes-of interest
Frey et al., 2009 (Study H-400-002)	18-29 y/o naïve adults	ACAM1000 and ACAM2000	Cutaneous response, neutralizing antibody response, MAE, SAE
Artenstein et al., 2005 (Study H-400-005)	18-29 y/o naïve adults	ACAM2000	Cutaneous response, neutralizing antibody response, MAE, SAE
Acambis, Inc., 2007 (Study H-400-003)	>28 y/o previously vaccinated adults	ACAM2000	Cutaneous response, neutralizing antibody response
Acambis, Inc., 2007 (Study H-400-009)	18-29 y/o naïve adults	ACAM2000	Cutaneous response, neutralizing antibody response, MAE, SAE
Acambis, Inc., 2007  (Study H-400-012)	>31 y/o previously vaccinated adults	ACAM2000	Cutaneous response, neutralizing antibody response, MAE, SAE

Table 2 Footnote

¹ The Centers for Disease Control and Prevention was listed as the sole funding source for all included studies.

Evidence Tables: Summary of Data

1. Outcomes of both benefits and harms were abstracted for each study.

2. Benefit outcomes were assessed and reported in all 5 RCTs and included both cutaneous response (vaccination success) and neutralizing antibody response (based on 50% PRNT).
3. Outcomes considered harms were assessed and reported in 4 out of 5 RCTs and included: SAE, MAE, myo/pericarditis resolved with sequelae, myo/pericarditis resolved without sequelae, and inadvertent inoculation.

Table 3: Smallpox vaccine: Benefits

	Study Population / Treatment Group
	Vaccinia-Naïve Subjects		Previously Vaccinated Subjects
	ACAM2000	Dryvax	ACAM2000	Dryvax
Cutaneous Response (Vaccination Success)
No. of Evaluable Subjects (# studies)	857 (3)	336 (3)	1238 (2)	440 (2)
Number of Vaccination Successes (%)	828 (96.6%)	334 (99.4%)	1041 (84.1%)	433 (98.4%)
Neutralizing Antibody Response
No. of Evaluable Subjects (# studies)	646 (3)	269 (3)	784 (2)	428 (2)
Pooled Geometric Mean Titer (GMT) Ratio1,2, (95% CI of pooled GMT Ratio)	0.677 (0.625, .733)

Table 3 Footnotes

¹Pooled GMT Ratio was generated in Revman. Though GMT’s were reported for all studies included in the analysis, variance data for the evaluable population was only reported in one Phase 2 study (H-400-005); however it was unpublished, as it was not appropriate for the analysis the authors needed. Variance data for Phase 2 and Phase 3 studies was requested from the authors [13]. Data from Phase 3 studies (H-400-009, which looked at the naïve total study population, and H-400-012, which looked at the previously vaccinated total study population) was acquired. The neutralizing antibody titers on Day 30 reported for the total population (both naïve and previously vaccinated individuals) in the Phase 3 studies was reported as both GMT and Log₁₀GMT (± standard deviation).  The standard deviation of the reported GMT (for both naïve and previously vaccinated individuals) was calculated by taking the inverse log of the standard deviation reported for the Log₁₀GMT. The resulting calculated standard deviation for the GMT (for both naive and previously vaccinated individuals), along with the sample size and mean (GMT), were used to calculate the 95% confidence interval (CI) and range. The calculated standard deviation from the evaluable populations within the Phase 3 studies was subsequently applied to the corresponding evaluable populations from Phase 1 and 2 studies lacking variance data.

²Data for each study were converted to a log scale for computing pooled estimates. The pooled results were then converted back into their original metric by taking their anti-log.

Table 4: Smallpox vaccine: Harms

	Study Population / Treatment Group
	Vaccinia-Naïve Subjects		Previously Vaccinated Subjects
	ACAM2000 N1 = 954 n (%) # Studies]	Dryvax N1 = 368 n (%) [# Studies]	ACAM2000 N1 = 1371 n (%) [# Studies]	Dryvax N1 = 448 n (%) [# Studies]
Experienced Serious Adverse Events2	0 (0%) [3]	0 (0%) [3]	0 (0%) [1]	1 (.22%) [1]
Myo/pericarditis Resolved with Sequelae3	1 (.10%) [3]	1 (.27%) [3]	0 (0%) [1]	0 (0%) [1]
Myo/pericarditis Resolved without Sequelae	6 (.63%) [3]	2 (.54%) [3]	0 (0%) [1]	0 (0%) [1]
Inadvertent Inoculation	0 (0%) [3]	0 (0%) [3]	0 (0%) [1]	0 (0%) [1]
Mild Adverse Events	945 (99.05%) [3]	368 (100%) [3]	1325 (96.64%) [1]	443 (98.8%) [1]

Table 4 Footnotes

¹Number indicated represents the total number of subjects enrolled in studies where subjects were administered either ACAM2000 or Dryvax, subsequently monitored for the outcome(s) of interest, and results were reported.

²Though not included in the Acambis 2007 VRBPAC briefing document, one case of generalized vaccinia was reported in Study H-400-012 within the VRBPAC Background Document [1, 7]. This case was discovered in a previously vaccinated subject upon reporting to a scheduled study center visit on Day 10 post vaccination. The subject was admitted to a local hospital for observation, dermatological consult, treatment, and subsequently discharged from the hospital the following day. This event was determined to be study-vaccine related and resolved without sequelae on Day 13.

³One case of myo/pericarditis was categorized by the Acambis report as remaining ongoing [1]. The individual was a subject within the Phase 3 study, and received Dryvax as a naïve vaccine. We have included that subject within the table as having myo/pericarditis resolved with sequelae.

Evidence Tables: Type of Evidence

The evidence type for each outcome was determined based on the studies reviewed. Evidence type was initially ranked according to the following ACIP GRADE criteria [11]:

1 = Randomized controlled trials (RCTs), or overwhelming evidence from observational studies.
2 = RCTs with important limitations, or exceptionally strong evidence from observational studies.
3 = Observational studies or RCTs with notable limitations
4 = Clinical experience and observations, observational studies with important limitations, or RCTs with several major limitations.

The initial evidence type could subsequently be downgraded if any of the GRADE criteria was determined to be serious (-1) or very serious (-2). Evidence could also be upgraded if the strength of association was shown to be strong (+1 when the relative risk is approximately >2 or <0.5) or very strong (+2 when the relative risk is approximately >5 or <0.2) and there was no serious risk of bias [11].

Top of Page

Overall Evidence Type

Table 5. Type of Evidence: The effect of ACAM2000 on identified outcomes

Outcome	Design(# studies)	Risk of bias	Inconsistency	Indirectness	Imprecision4	Other considerations1	Evidence Type
Benefits
Cutaneous Response	RCT (5)	No Serious	No Serious	Serious2	No Serious	None	2
Neutralizing Antibody Response	RCT (5)	No Serious	No Serious	Serious2	No Serious	None	2
Harms
Serious Adverse Events	RCT (4)	No Serious	No Serious	No Serious	Serious4	None	2
Myo/pericarditis Resolved with Sequelae	RCT (4)	No Serious	No Serious	No Serious	No Serious	None	1
Myo/pericarditis Resolved without Sequelae	RCT (4)	No Serious	No Serious	Serious3	No Serious	None	2
Inadvertent Inoculation	RCT (4)	No Serious	No Serious	No Serious	Serious4	None	2
Mild Adverse Events	RCT (4)	No Serious	No Serious	No Serious	No Serious	None	1
Overall evidence type across all critical outcomes5	2

Table 5 Footnotes

¹Strength of association, dose-response, opposing plausible residual confounding or bias, publication bias.

²Cutaneous response and neutralizing antibody response were surrogates for the outcome of primary interest: vaccine efficacy to prevent orthopoxviral disease

³The clinical significance of myo/pericarditis resolved without sequelae is unclear, therefore, myo/pericarditis resolved with sequelae was assessed to be the outcome of primary interest.

⁴ The total number of participants enrolled across all RCTs was <4000. Thus, these studies were not powered to detect serious adverse events (i.e. EV, PV, PVE, death) or inadvertent inoculation. Please See Table 1 in Supplementary Appendix 1 for information regarding sample size needed to detect twice the AE rate [14].

⁵The lowest evidence quality from critical outcomes assessed

Top of Page

Judgments About the Recommendation Category

Factors to consider:

Key Factors	Comments
Evidence type for benefits and harms	Overall evidence type 2 across all critical outcomes Evidence type 1 for myo/pericarditis resolved with sequelae and mild adverse events Evidence type 2 for cutaneous response, neutralizing antibody response, serious adverse events, myo/pericarditis resolved without sequelae and inadvertent inoculation Cutaneous response and neutralizing antibody response were downgraded due to indirectness (assessed to be surrogates for outcome of primary interest) Serious adverse events and inadvertent inoculation were downgraded for imprecision (studies were not powered to detect these events)
Balance between benefits and harms	Vaccination provides benefits through protective antibody response, as well as cutaneous response (take rate) Serious adverse events are uncommon when proper screening methods are applied Very few  documented vaccine failures among vaccinated laboratory personnel
Value	Prevents a potentially serious disease with poor outcomes Inform decisions about smallpox vaccination in laboratory and health-care personnel at risk for occupational exposure to orthopoxviruses
Cost-effectiveness	Not evaluated Likely low risk of disease in persons receiving appropriate dose of smallpox vaccine (ACAM2000) Vaccine provided free of charge upon request

Summary

The overall evidence type was determined to be type 2 across all critical outcomes. It was determined that the benefits of vaccinating with ACAM2000 are likely greater than potential harms. Additionally, there is high value placed on prevention of OPXV infections among laboratory and health-care personnel at risk for occupational exposure to orthopoxviruses through routine recommendation of vaccination with ACAM2000. Though cost-effectiveness was not analyzed in detail, the impact from a social and personal standpoint cannot be overlooked given the potential for disfiguring scars from orthopoxvirus infections. Furthermore, emotional and economic cost associated with treating an unvaccinated individual who is occupationally exposed to an orthopoxvirus could be high.

Top of Page

References

1. Acambis, Inc. (2007). ACAM2000 smallpox vaccine. Vaccines and Related Biological Products Advisory Committee (VRBPAC) Briefing Document. Apr 18 2007.
2. Artenstein AW, Johnson C, Marbury TC, Morrison D, Blum PS, Kemp T, Nichols R, Balser JP, Currie M, & Monath TP. (2005) A novel, cell culture-derived smallpox vaccine in vaccinia naïve adults. Vaccine; 23(25): 3301-3309.
3. Greenberg, R.N., & Kennedy, J.S. (2008). ACAM2000: a newly licensed cell-culture based live vaccinia smallpox vaccine. Expert Opin. Investig. Drugs; 17(4): 555-564.
4. Handley, L., Buller, R.M., Frey, S.E., Bellone, C., & Parker, C. (2009). The new ACAM2000 vaccine and other therapies to control orthopoxvirus outbreaks and bioterror attacks. Expert Review Vaccines; 8(7): 841 – 850.
5. Monath, T.P., Caldwell, J.R., Mundt, W., Fusco, J., Johnson, C.S., Buller, M., Liu, J., Gardner, B.,Downing, G., Blum, P.S., Kemp, T., Nichols, R., & Weltzin, R. (2004). ACAM2000 clonal Vero cell culture vaccinia virus (New York City Board of Health strain): a second-generation smallpox vaccine for biological defense. Int. J. Infectious Diseases; Suppl 2: S31-S44.
6. Acambis, Inc. (2008). ACAM2000 smallpox (vaccinia) vaccine, live. Package insert.
7. Rosenthal, S., Merchlinsky, M., Chowdhury, M. (2007). ACAM2000 (Live Vaccinia Virus Smallpox Vaccine). VRBPAC Background Document. May 17, 2007 VRBPAC Meeting.
8. Centers for Disease Control and Prevention (CDC). (2001). Vaccinia (Smallpox) Vaccine: Recommendations of the Advisory Committee on Immunization Practices (ACIP), 2001. MMWR; 50(RR-10): 1-25.
9. Centers for Disease Control and Prevention (CDC). (2003a). Recommendations for using smallpox vaccine in a pre-event vaccination program. Supplemental recommendations of the Advisory Committee on Immunization Practices (ACIP) and the Healthcare Infection Control Practices Advisory Committee (HICPAC). MMWR; 52 (RR-7): 1-16.
10. Centers for Disease Control and Prevention (CDC). (2008). Notice to Readers: newly licensed smallpox vaccine to replace old smallpox vaccine. MMWR; 57(8): 207-208.
11. Ahmed, F. (2013). U.S. Advisory Committee on Immunization Practices (ACIP) Handbook for Developing Evidence-based Recommendations. Version 1.2. Sep 10, 2013.
12. Frey, S.E., Newman, F.K., Kennedy, J.S., Ennis, F., Abate, G., Hoft, D.F., & Monath, T.P. (2009) Comparison of   the safety and immunogenicity of ACAM1000, ACAM2000, and Dryvax in healthy vaccinia-naïve adults. Vaccine; 27(10): 1637 - 44.
13. Chowdhury, M.K. (2006). ACAM2000 STN 125158 Memorandum: Statistical Review and Evaluation. August 16, 2006. Obtained by request from Dr. Michael Merchlinsky.
14. Lane, J.M., Neff, J.M., Millar, J.D. (1970). Complications of smallpox vaccination, 1968: results of ten statewide surveys. Journal of Infectious Diseases; 122(4): 303-309.

Top of Page

Supplementary Appendix 1: Reported Rates of Serious Adverse Events

Table 1.  Rates of Serious Adverse Events

	Rates of SAE in vaccinated population (# cases / million vaccinations)1		% Chance You Would NOT see SAE in ACAM2000 RCTs		Sample Size Needed to Detect Twice the AE Rate (Power 0.8)
	Naïve(n= 1207)	Previously Vaccinated(n=1670)	Naïve	Previously Vaccinated	Naïve	Previously Vaccinated
Eczema vaccinatum	38.5	3	95.50%	99.50%	611,565	7,848,844
Progressive vaccinia	1.5	3	99.80%	99.50%	15,697,723	7,848,844
Postvaccinial encephalitis	12.3	2	98.50%	99.60%	1,914,325	11,773,284
Inadvertent Inoculation	529.2	42.1	52.80%	95.00%	44,459	559,267
Death	1.5	NA	99.80%	NA	15,697,723	NA

¹Rates of SAEs from Lane JM, Ruben FL, Neff JM, Millar JD. (1970). Complications of smallpox vaccination, 1968: results of ten statewide surveys.  Journal of Infectious Diseases. 122(4): 303-309.

Top of Page