OUP user menu

Evaluation of Four Qualitative Third-Generation HIV Antibody Assays and the Fourth-Generation Abbott HIV Ag/Ab Combo Test

Frank H. Wians Jr. PhD, MT(ASCP), DABCC, FACB, Holly A. Moore AS, MT(AMT), Donna Briscoe BS, MT(ASCP), Kathy M. Anderson BA, MT(ASCP), Pamela S. Hicks BS, MLT(ASCP), MT(HEW), Debbie L. Smith AD, MLT(ASCP), Tammy A. Clark BS, MT(ASCP)SH, Maggie M. Preston BS, MT(HHS), MLT (ASCP), Beckie Gammons BS, MT(ASCP), Cynthia S. Ray BS, MT(ASCP), Christine Bond BS, MT(ASCP), Joshua T. West BS, MLS(ASCP)
DOI: http://dx.doi.org/10.1309/LM38NCGQ0GWJIMZV 523-535 First published online: 1 September 2011

Abstract

Objective: Compare the agreement between four third-generation rapid or automated human immunodeficiency virus (HIV) tests vs the fourth-generation Abbott ARCHITECT HIV Ag/Ab Combo assay (Combo Test) (Abbott Laboratories, Abbott Park, IL) in 7 clinical laboratories located throughout the Dallas-Fort Worth, TX, metroplex area.

Methods: We tested a total of 220 specimens from the general population of patients being screened for HIV antibody status, including Western blot (WB) and HIV-1 RNA viral load by NAAT in patients with initially positive HIV screening tests. We performed a method precision study of the Combo Test and evaluated the agreement between the 4 comparative methods (CMs) against the Combo Test method (TM) using a 2 × 2 contingency table to determine values for positive percent agreement (PPA), negative percent agreement (NPA), and overall percent agreement (OPA) according to the guideline (EP12-A2) for the evaluation of qualitative assays published by the Clinical Laboratory Standards Institute (CLSI).

Results: The Combo Test yielded consistent (reproducible) results when the HIV antibody (or p24 antigen) concentration was 20% away or more from C50. In addition, values for PPA, NPA, and OPA for CM vs TM results were all 100%, or not significantly different from 100%.

Conclusion: The high detection rate of the Combo Test, when coupled with its high diagnostic sensitivity and specificity, rapid TAT (<30 minutes) of test results using a fully automated instrument (ARCHITECT), and excellent agreement with HIV-1/2 antibody results by several commercially available HIV-1/2 screening tests and with WB and NAAT results, indicate that the Abbott HIV-1/2 Ab/Ag Test is a useful screening test in the identification of HIV-infected individuals.

Keywords:
  • human immunodeficiency virus
  • fourth-generation assay
  • comparative method
  • test method
  • agreement

The genetic variability of the human immunodeficiency virus (HIV) is illustrated by the number of groups, genetic subtypes, and the geographic distribution of these groups and subtypes throughout the world (Figure 1). Moreover, HIV-1 group M accounts for the majority of HIV infections in the United States, while HIV infection with a closely related virus, HIV-2, is rare in the United States but prevalent in West Central and West Africa (Figure 1). In addition, the prevalence of HIV-1 group O infection was thought to be low in the United States—the first case of this infection was reported in the United States in 1996.1 There are marked differences in the global distribution of HIV infection (Figure 2).2

The 2010 Global Report of the Joint United Nations Programme on HIV/acquired immunodeficiency syndrome (AIDS) (UNAIDS) indicated that in the 10-year period between 1999 and 2009, there was a 27% increase in the number of people living with HIV, with Sub-Saharan Africa still bearing the largest burden of HIV-infected individuals (Figure 2).2 In addition, during this same time period, the number of new HIV infections and risk of HIV transmission to infants declined by ∼20% with interventions (eg, use of highly active antiretroviral therapy) available in the United States and other developed countries.2,3 The factors contributing to these encouraging findings include the cost-effective availability of antiretroviral drug therapy, more widespread use of condoms, and the introduction of significant educational programs and materials, especially in high-risk populations, related to the prevention of HIV infection and AIDS. Despite these successes, the Centers for Disease Control (CDC) has estimated that as many as 25% of newly infected individuals in the United States are unaware of their infection; these individuals account for more than 50% of new HIV infections; and, they remain concentrated among men who have sex with men and African Americans.2,4 Fortunately, the recent availability of fourth-generation HIV assays, capable of identifying individuals with HIV infection earlier than prior generation assays (Figure 3), is likely to improve this statistic if the individuals who are unaware of their HIV status are screened using a fourth-generation HIV assay. Several studies have demonstrated that fourth-generation HIV assays can reduce the “window (or seroconversion) period” (ie, period during which a pathogen can be detected in a patient specimen by nucleic acid testing but not by serology, as the patient’s immune system has not yet responded to the pathogen by producing measurable levels of antibodies) such that newly infected individuals can be identified, treated, and counseled.58 The reduction in the window period afforded by fourth-generation HIV assays is due to the ability of these assays to detect not only HIV-1/2 antibodies but also the 24 kilodalton HIV group-specific (capsid or core) antigen (gag), p24, which can be detected as early as, and in some cases, even sooner than, 17 days post-HIV infection and up to 5–20 days sooner than third-generation HIV antibody enzyme immunoassays (EIAs) (Figure 3). This is not surprising when one considers that HIV antigen presentation must, obligatorily, precede antibody production to these antigens at levels detectable by HIV-1/2 antibody assays.

Currently, there are no fourth-generation HIV rapid tests available and, because of the only recent availability of fully automated fourth-generation HIV assays, the most popular HIV screening assays are third-generation EIAs,911 especially qualitative rapid tests such as the OraSure OraQuick ADVANCE (OraSure Technologies, Bethlehem, PA) and Clearview STAT-PAK HIV-1/2 (Inverness Medical, Princeton, NJ) assays. Among all laboratories participating in the most recent College of American Pathologists (CAP) external proficiency survey, AHIV-C, the percentage of all labs (n=∼825) reporting results for the HIV-1/2 antibody methods shown in Table 1 decreased in the order: 49.8% (OraQuick ADVANCE) >43.4% (STAT-PAK) >2.5% (Other) >1.9% (Abbott, but not the Combo Test) >1.5% (Ortho) >0.9% (Multispot). Thus, these 2 rapid tests accounted for greater than 90% of the HIV tests used by all laboratories reporting HIV results for this CAP Survey. The performance characteristics, including diagnostic sensitivity and specificity, of the OraQuick and STAT-PAK HIV-1/2 assays, have been reported previously.12

Figure 1

Phylogeny of HIV groups and subtypes, and their geographic distribution. Color maps (top right): North America, Europe, South America, Japan, Australia, and Asia; (bottom center): West-Central (Congo Republic, Gabon, Cameroon, Central African Republic, Equatorial Guinea; Congo [Democratic Republic of Congo]; and, Angola]), and West Africa (ie, all countries shown except those considered West-Central Africa). CRFs, circulating recombinant forms.

Figure 2

Global distribution of HIV disease (percentage of population) in 2009.2 Below the thick black dotted line on the continent of Africa is “sub-Saharan” Africa.

When evaluating qualitative assays, most laboratorians use a similar strategy to that used when evaluating quantitative assays—select, without obvious bias, a minimum of 20–30 patient samples for testing by the test method (TM) being evaluated and hope that there are no discordances (ie, 100% agreement) between results for the current or comparative method (CM) and the TM so the issue of deciding how many discordances can occur without invalidating the TM as an acceptable replacement for their CM is moot. It is important to note, however, that the discordances provide valuable information, especially if patient-specific information and/or the results of additional testing (eg, WB, quantitative HIV RNA level, and/or NAAT) are combined to determine where truth lies (ie, was it the CM or the TM that provided the correct result?). Using this strategy, typically, the assumption is made that the precision, diagnostic sensitivity, and diagnostic specificity of the TM have been validated previously by other studies as consistent with the information for these analytical and clinical performance characteristics indicated in the manufacturer’s product insert.

Fortunately, the Clinical Laboratory Standards Institute (CLSI) has provided an evaluation protocol, EP12-A2, for evaluating qualitative test performance that provides statistical validity to the results of such an evaluation compared to the aforementioned strategy.13 The CLSI protocol EP12-A2 recognizes 2 possible approaches when evaluating qualitative methods: 1 in which the comparator is diagnostic accuracy criteria and the other in which the comparator is not diagnostic accuracy criteria. There are marked differences in the information on TM performance provided by these 2 different approaches.

Using the approach involving diagnostic accuracy criteria, applied to the evaluation of a qualitative HIV TM, samples from patients with known HIV disease, based on clinical findings and on results from a test (eg, HIV-1 RNA viral load) different from the TM being evaluated and with high diagnostic accuracy, are tested by the TM, while in the absence of diagnostic accuracy criteria, the performance of the qualitative TM is evaluated by testing these samples by both the CM and the TM. In both cases, a 2 × 2 contingency table (Figure 4) of the test results by the CM and TM is constructed; however, when diagnostic accuracy (ie, diagnostic sensitivity and specificity) criteria are used, this table allows the user to determine estimates for the following parameters of TM performance:

  • Diagnostic sensitivity

  • Diagnostic specificity

  • Predictive value of a positive test result (PPV)

  • Predictive value of a negative test result (NPV)

  • Detection rate ([number of individuals with known disease positive by a CM/total number of individuals tested] × 100)

Moreover, a statistical test (eg, McNemar test) can be used to conclude whether or not there is a statistically significant difference between the (sensitivity, specificity) pairs of the 2 tests. An excellent example of the evaluation of qualitative HIV methods using diagnostic accuracy criteria is provided by the publication of Pandori and colleagues.14

Figure 3

The effect of assay improvements with each generation of HIV assays on the window period for identifying individuals with acute or recent infection (adapted from a figure available from Abbott Diagnostics compiled from information provided in References 58).

View this table:
Table 1

All Participant Summary Data for Anti-HIV-1/2 From CAP Survey AHIV-C 2010

AHIV-11AHIV-12AHIV-13*AHIV-14AHIV-15
ManufacturerPosNegPosNegPosNegPosNegPosNeg
Abbott**160115160016115
Bio-Rad Multispot7007430707
Clearview STAT-PAK357103585230603581357
OraSure OQ ADVANCE410104103723914092408
Ortho120012120012012
Other2100211110021021
Total8232182346735818234820
Referee labsn.p.n.p.n.p.n.p.135n.p.n.p.n.p.n.p.
  • * Due to lack of consensus among referee labs, participant labs were not graded on this challenge.

  • ** Not the Combo Test, but, rather, the OraQuick assay licensed previously by OraSure Technologies for distribution by Abbott. This license is no longer in force.

  • CAP, College of American Pathologists; AHIV, anti-human immunodeficiency virus; Pos, positive; Neg, negative; n.p., not provided (by CAP).

In the absence of diagnostic accuracy criteria, the same 2 × 2 contingency table can be used to obtain estimates for the following parameters of agreement between the CM and TM results:

  • Positive percent agreement (PPA)

  • Negative percent agreement (NPA)

  • Overall percent agreement (OPA)

The disadvantages of the evaluation protocol not using diagnostic accuracy criteria compared to the protocol that uses diagnostic accuracy criteria are:

  • agreement is not a measure of “correctness”

  • agreement measures are not universally applicable to all populations because of possible differences in the prevalence of HIV infection/disease between 2 different populations

  • the usual measures of test performance (ie, diagnostic sensitivity, specificity, PPV, and NPV) cannot be determined

Figure 4

2 × 2 Contingency Table for BUMC Data.

Despite these disadvantages, the CLSI protocol, EP12-A2, indicates: “ … reporting how often the candidate method and comparative method (CM) agree could be useful.”13

Because of the potential advantages of the decreased window period of fourth-generation over earlier generation HIV screening assays,15 we undertook an evaluation of the Abbott ARCHITECT (Image 1) fourth-generation HIV Ag/Ab Combo (or Combo Test) assay against currently used HIV screening assays in an unselected (ie, no information was available on the HIV status of the individuals tested or the results of any HIV testing, eg, Western blot [WB], HIV-1 viral load, and/or nucleic acid amplification testing [NAAT] performed prior to the evaluation) population of individuals at 7 sites participating in this evaluation. The Combo Test was the first fourth-generation HIV diagnostic assay approved by the U.S. Food & Drug Administration (June 18, 2010).

View this table:
Table 2

Comparison of Qualitative HIV Assay Characteristics

Characteristic1OraQuick ADVANCEHIV 1/O/2 EnhancedHIV 1/2 STAT-PAKAnti-HIV 1+2HIV Ag/Ab Combo Test
ManufacturerOrasureSiemensChembioOrthoAbbott
Assay generation3rd3rd3rd3rd4th
HIV Ab subtype detected1 and 21 (inc Gp O) and 21 and 21 and 21 (Gps O and M) and 2
HIV-1 Ag detectedn.d.n.d.n.d.n.d.p24
Device/instrumentSU-POCTADVIA CentaurSU-POCTVITROSARCHITECT
Sample typeOF, P2, WB3S, P4S, P5, WB3S, P4(S, P)6
Sample volume, mL0.0050.05n.s.0.080.15
Immunoassay typeLFIC-PAGABMCILFIC-CGDIBACMIA
Solid phaseNitrocellulose membranePMPsNitrocellulose membranePlastic wellPMPs
Capture Ag’s/Ab’s/SubstanceSynthetic HIVenv Ag’sSB-([HIV 1 and 2 rAg’s] + Gp O peptide Ag)CG-(ABP+HIV-1&2 Ag’s)rAg’s7HIV-1&2 Ag’s + HIV-1 p24 mouse Mcl Ab
Signal molecule/antigenProtein-A goldAE-([HIV 1 and 2 rAg’s] + Gp O peptide Ag)CGD particlesLuminol derivativeAE-conjugates8
Number of calibratorsn.a.2n.a.11
Number of controls34325
Reproducibility, %CV range9
Between run0.02.5–13.02.1–6.91.5–3.0
Between testing site0.02.2–8.61.6–4.7
Read time or TTFR, min20–4045–6015–204828
Diagnostic sensitivity, %99.6 (98.9–99.8)100.0 (99.7–100.0)99.7 (98.9–100.0)100100.0 (98.2–100.0)
Diagnostic specificity, %99.9 (99.6–99.9)99.9 (99.8–100.0)99.9 (99.6–100.0)99.6 (99.1–99.8)100.0 (99.2–100.0)
  • 1 All data from manufacturer’s product insert.

  • 2 EDTA plasma.

  • 3 EDTA, sodium heparin, sodium citrate, or ACD Solution A whole blood (WB) from venipuncture or fingerstick.

  • 4 EDTA, sodium heparin, sodium citrate, or ACD Solution A plasma.

  • 5 EDTA, sodium heparin, or sodium citrate plasma; WB from venipuncture or fingerstick.

  • 6 Serum in glass or plastic tube with or without serum separator; tripotassium (K3)- or disodium (Na2)-EDTA plasma in glass; Na2-EDTA plasma with or without gel separator, sodium heparin, or lithium heparin with gel separator in plastic tube.

  • 7 HIV-1 Env 13, HIV-2 Env AL, HIV-1 ENV 10, and HIV-1 p24 antigens.

  • 8 5 Conjugate types: 1) AE-(HIV-1 Ag); 2) AE-(HIV-2 Ag); 3) AE-(HIV-1 Group M Ag); 4) AE-(HIV-1 Group O Ag); 5) AE-(HIV-1 p24 Ab).

  • Ab, antibody; SU-POCT, single use-point-of-care test; OF, oral fluid; P, plasma; WB, whole blood; n.s., not stated; LFIC, lateral flow immunochromatography; PAG, protein-A gold; CGD, colloidal gold dye; HIVenv Ag’s, antigens representing HIV envelope epitopes; Ab’s, antibodies; n.d., not detected; n.a., not applicable; inc, including; Gp(s), group(s); ABMCI, antigen bridging microparticle chemiluminescent immunoassay; PMPs, paramagnetic particles; SB, streptavidin-biotinylated; rAg’s, recombinant antigens; CG-ABP+HIV-1/2 Ag’s, colloidal gold dye particles-(antigen binding protein+HIV-1&2 antigens) conjugate; IBA, immunometric bridging assay; CMIA, chemiluminescent microparticle immunoassay; Mcl, monoclonal; AE, acridinium ester; TTFR, time to first result.

  • 9 Variability of assay performance for all samples/replicates per sample/testing sites/reagent lots/days/techs: 5/27/3/3/3/9 (OraQuick); 12/5/3/3/6/3 (HIV 1/O/2 Enhanced); 16/5/3/3/3/3 (HIV1/2 STAT-PAK); 4/2/3/3/20/1 (Anti-HIV 1+2); 13/4/3/3/5/3 (HIV Ag/Ab Combo).

View this table:
Table 3

Summary of Testing Sites and Number of Patients’ Specimens Tested by Comparative Method and Test Method, Western Blot, and Quantitative or Qualitative NAAT for HIV RNA

No. of Patients’ Specimens Tested
Testing SiteHIV Assay (CM)Type of Specimen TestedCM and TMWBNAAT
AMHOraQuick ADVANCEK3-EDTA-P211
BASCentaur HIV 1/O/2 EnhancedSerum5252
BUMCOraQuick ADVANCEK3-EDTA-P51111
THDCentaur HIV 1/O/2 EnhancedK3-EDTA-P37119
THFWVitros Anti-HIV 1+2Serum13
THPOraQuick ADVANCESerum251
THSWOraQuick ADVANCEK3-EDTA-P21
Total2202714

We determined the PPA, NPA, and OPA for each CM vs the Combo TM.

Materials and Methods

Sites

The laboratories that participated in this study are located in 1 of the following 7 sites: 1) Texas Health Arlington Memorial Hospital (AMH), Arlington; 2) Baylor All Saints (BAS), Fort Worth; 3) Baylor University Medical Center (BUMC), Dallas; 4) Texas Health Presbyterian Hospital (THD), Dallas; 5) Texas Health Harris Methodist Hospital (THFW), Fort Worth; 6) Texas Health Presbyterian Hospital (THP), Plano; and, 7) Texas Health Harris Methodist Hospital Southwest (THSW), Fort Worth, TX.

HIV Assays and Type/Number of Specimens Tested

The characteristics of the qualitative HIV assays used in this study are summarized in Table 2. The HIV assay used at each participating site, and the type and number of patients’ specimens tested using these assays are shown in Table 3. These assays included: OraQuick ADVANCE Rapid HIV-1/2 Antibody Test (OraSure Technologies), HIV 1/O/2 Enhanced (Siemens, Tarrytown, NY), Anti-HIV 1+2 (Ortho-Clinical Diagnostics, High Wycombe, U.K.), and HIV Ag/Ab Combo (Abbott, Wiesbaden, Germany).

All patients’ specimens were selected with no obvious bias and obtained by fingerstick or venipuncture. Specimens obtained by venipuncture were collected using 1 of the collection tube types indicated in the manufacturer’s product insert for each HIV assay. All specimens were tested according to the manufacturer’s instructions using the assays shown in Table 3. At each testing site, all samples were assayed by both the CM specific to each site and the candidate or TM (ie, the Abbott HIV Ag/Ab Combo Test) within 2 hours of specimen collection, or after thawing (THD), in a single run by a single operator. According to the manufacturer’s product insert for the ADVIA Centaur EHIV assay, when 10 specimens were subject to 6 freeze/thaw cycles, no clinically significant differences were observed. Test results were reported as reactive (R) or nonreactive (NR) based on the information provided in the manufacturer’s product insert.

A total of 220 test results were generated across 7 testing sites and all assays. In addition, 27 specimens across all sites were assayed by WB, and 14 specimens were assayed for HIV-1 ribonucleic acid (RNA) by quantitative or qualitative NAAT (Table 3).

WB Testing

Western blot confirmatory testing of positive immunoassay screening tests was performed either in-house or by an external, CAP-accredited referral lab (Med Fusion, Lewisville, TX) using the Bio-Rad (Redmond, WA) Genetic Systems (GS) HIV-1 WB assay. Interpretation of patients’ serum or (ethylenediaminetetraacetic acid [EDTA]/heparin/sodium citrate/CPDA-1/ACD)-plasma was based on the pattern of bands observed on the immunoblot (Table 4).

Quantitative and Qualitative HIV-1 RNA Testing by NAAT

Quantitative or qualitative HIV-1 RNA testing was performed using EDTA-plasma obtained from whole blood within 24 hours of collection by centrifugation at 800–1600 × g for 20 minutes at room temperature. Quantitative HIV-1 RNA testing was performed at Med Fusion, Lewisville, TX, using the AmpliPrep/COBAS TaqMan HIV-1 Test (Roche Diagnostics, Indianapolis, IN). According to the manufacturer’s product insert, the linear range of this test is: 20 cp/mL to 1.0 × 107 cp/mL. Qualitative HIV-1RNA testing was performed using the Aptima HIV-1 RNA Qualitative Assay (Gen-Probe, San Diego, CA). According to the manufacturer’s product insert, the analytical sensitivity of this assay is 30 cp/mL (ie, 98.5% of samples containing 30 cp/mL of HIV-1 RNA yield positive results).

View this table:
Table 4

Method Precision

Using patients’ serum samples and the protocol described in EP12-A2,13 we performed a method precision study on the TM by preparing 3 samples: 1 with concentration at C50; 1 with concentration 20% below C50 (C−20); and, 1 with concentration 20% above C50 (C+20). The C50 sample contains an analyte concentration near the assay cutoff value such that this sample yields 50% positive results and 50% negative results on testing of many replicates of this sample. Each sample was assayed by the TM 30 times in the same run. Method precision was considered acceptable if the C−20 to C+20 concentration range bounded (contained) the C5–C95 interval. The C5–C95 interval corresponds to the concentrations around the cutoff value such that observed results at concentrations outside this interval are consistently negative (ie, concentrations <C5) or consistently positive (ie, concentrations >C95). The true percentage of positive results lying within an estimated C5–C95 confidence interval varies with the number of samples (n) tested and approaches 95% as n increases.13

Statistical Analysis of Agreement Between CMs and TM

All data were analyzed according to the guidelines for the evaluation of qualitative test performance provided in EP12-A2.13 For each comparison of CM results vs TM results, a 2 × 2 contingency table was constructed and values for PPA, NPA, OPA, and the lower (LL) and upper (UL) limits of the 95% score confidence interval were calculated as shown in Figure 4. When results by the CM are based on diagnostic accuracy criteria, PPA and NPA are the diagnostic sensitivity and specificity, respectively, of the TM.13 The use of “score confidence limits,” in lieu of “exact confidence limits,” is recommended for reasons beyond the scope of this article.13 In addition, the McNemar test was performed using MedCalc software (MedCalc Software, Mariakerke, Belgium) to determine if there was a significant difference between paired proportions at a P value of 0.05.

Results

TM Precision

Signal-to-CutOff (S/CO) values (mean ± SD) for the 30 replicates of the C−20, C50, and C+20 samples using the TM were: 0.79 ± 0.04, 0.99 ± 0.04, and 1.19 ± 0.03, respectively. The percentage of positive and negative (%pos, %neg) results among the 30 replicates by the TM for samples C−20, C50, and C+20 were: (0.0, 100.0), (40.0, 60.0), and (100.0, 0.0), respectively. Therefore, the largest confidence that can be achieved where the lower end of the one-sided confidence interval for both the C−20 and C+20 samples is at or above 95% is: 0.8870 × 0.8870 × 100% = 78.68%, where 0.8870 is obtained from a table comparing confidence (probability scale) as a function of N (number of replicate measurements) and the fraction (as a %) of positive (or negative) results (see Appendix, p39 in Reference 13). A confidence value of 79% indicates that the Combo Test yields consistent (reproducible) results when the HIV antibody (or p24 antigen) concentration is 20% away or more from C50.

Image 1

Abbott Architect™ i2000SR instrument (operated by Holly Moore, MT(ASCP), BUMC).

Agreement Between CMs and TM

The PPA, NPA, and OPA for OraQuick results vs results by the Combo Test HIV assay were all 100%, or not significantly different from 100%, at 3 of 4 sites (AMH, THSW, THP) using the OraQuick HIV assay, while at 1 site (BUMC), PPA and OPA were less than 100% (Figure 5). However, the difference between paired proportions was not statistically significant (Figure 4). The PPA for the results of the OraQuick vs results by the Combo Test HIV assays at BUMC was the lowest (58.3%) for all HIV assays at all sites because there were 5 samples (Nos. 7 to 11, Table 5) that were initially R by OraQuick, weakly or NR on repeat testing, NR by the Combo Test HIV assay, and WB negative (ie, no bands detected). In addition, Combo Test results for all 16 discordant (ie, CM and TM results disagreed) samples across all sites were consistent with WB and/or NAAT results (Table 5).

Among the 2 sites (BAS and THD) using the Centaur HIV assay, values for the 3 measures of agreement (PPA, NPA, and OPA) between this assay vs the Combo Test HIV assay were: BAS: 84.2%, 93.9%, and 90.4%; THD: 80.0%, 100.0%, and 89.2% (Figure 5).

The Vitros HIV assay at site THFW provided 100% agreement for PPA, NPA, and OPA vs results by the Combo Test HIV assay (Figure 5).

In Figure 5, a table is included indicating the number of potential false positive (FP) and false negative (FN) results that cause values for PPA, NPA, and OPA shown in the figure to vary from 100%. The data in the table are referred to as potential FP or FN results because a diagnostic accuracy criterion was not used to definitively identify the individuals in this study as with or without HIV infection prior to testing with the comparative and TMs.

Figure 5

Percentage agreement between individual site-specific HIV assays vs the test method (TM; Abbott HIV Ag/Ab Combo Test) (error bars represent the lower and upper limits [UL] of the 95% “score” confidence interval). Abbreviations are same as those shown in Tables 2 and 3. FP, false positive; FN, false negative; PPA, positive percent agreement; NPA, negative percent agreement; OPA, overall percent agreement.

Discussion

Differences between values for PPA, NPA, or OPA for comparative and TMs between sites (eg, OraQuick ADVANCE HIV-1/2 vs Combo Test Results at BUMC compared to the 3 other sites [AMH, THP, THSW]; Figure 5) are most likely due to sample size and differences in the prevalence of HIV disease in the unselected (ie, not based on known HIV status by a “gold standard” method [eg, HIV-1 RNA viral load]) population of patients being screened for HIV antibodies at each individual site. Among the 7 sites in this study, BUMC is the largest (1025 licensed beds; 39,000 admissions/y; 67,000 ED visits/y; and, 241,000 outpatient visits/y), with an active epidemiology and infection control department, and it is likely that this site screens patients with a higher prevalence of HIV disease and a greater heterogeneity of HIV antibody and antigen levels in their blood than the patients screened at the other 6 sites. Estimates for diagnoses of HIV infection (n=1246) in 2008 for the Dallas (the location of BUMC) and Fort Worth metroplex area, and adjacent cities (the location of the other 6 sites participating in this study), were 974 (78.2%) and 272 (21.8%), respectively.16 This heterogeneity resulted in 5/51 samples tested at BUMC demonstrating initially R OraQuick results that were all Combo Test NR and WB negative (Table 5). Fortunately, these patients would have been correctly classified as “screen negative” because all initially R specimens are repeat tested, and if the initial and repeat results disagree, the specimens are sent for WB testing. The Combo Test would have correctly classified these 5 patients as “screen negative” without the need for repeat screening or confirmatory WB testing (Table 5).

View this table:
Table 5

Results for Samples With Discordant Comparative and Test Method Results on Which WB and/or HIV-1 RNA Testing Was Performed

No.Site IDComp MethodHIV Result by Comp MethodRepeat Result by CMAvg S/CO Ratio by Combo TestInterpretation of Combo TestWB ResultsWB Band PatternQuantitative (cp/mL) or Qualitative (Pos/Neg) HIV-1 RNA by NAAT
1AMHOraQuickNR4.64R1.7 × 106
2BASCentaurR0.09NRNegNo bands present
3BASCentaurR0.27NRNegNo bands present
4BASCentaurR0.08NRNegNo bands present
5BASCentaurNR6.04RNegNo bands presentPos
6BASCentaurNR9.65RNegNo bands presentPos
7BUMCOraQuickRNR0.18NRNegNo bands present
8BUMCOraQuickRWR0.12NRNegNo bands present
9BUMCOraQuickRWR0.17NRNegNo bands present
10BUMCOraQuickRNR0.16NRNegNo bands present
11BUMCOraQuickRWR0.15NRNegNo bands present
12THDCentaurR0.22NRNegNo bands present
13THDCentaurR0.20NRNegNo bands present
14THDCentaurR0.16NRNegNo bands present
15THDCentaurR0.30NRNegNo bands present
16THPOraQuickNR12.96RPos
  • WB, Western Blot (Pos, Neg, or Ind); cp/mL, copies/milliliter; NAAT, nucleic acid amplification test; Comp, comparative; S/CO, signal-to-cutoff ratio; AMH, Texas Health Arlington Memorial Hospital; BAS, Baylor All-Saints; BUMC, Baylor University Medical Center ; THD, Texas Health Presbyterian Hospital; THP, Texas Health Presbyterian Hospital ; R, reactive; NR, nonreactive; WR, weakly reactive; Ind, indeterminate; Pos, positive; Neg, negative.

The Vitros HIV antibody test provided 100% agreement with Combo Test results on the relatively small number of samples tested at site THFW (n=13), while OPA for the Centaur method was 89.2% (95% Score CI: 75.3% to 95.7%) at site THD (n=37) and 90.4% (95% Score CI: 79.4% to 95.8%) at site BAS (Figure 5). The Centaur method would be expected to yield similar results as the Combo Test because among the HIV screening tests we evaluated, the principle of these methods was most similar between the Centaur and Architect methods, including the ability to detect Group O HIV-1 antibodies (Table 2 and Figure 1).

The weakness of the WB confirmatory test in correctly classifying patients, especially those with acute or recent HIV infection, as HIV positive or negative, was illustrated in the study by Pandori and colleagues.14 In this study, among 5 patients with acute or recent HIV infection, based on positive HIV-1 RNA findings (range of quantitative HIV-1 RNA levels: 335 cp/mL to ≥1 × 107 cp/mL), 4 of these patients had NR OraQuick Advance results, indeterminate WB results, and positive HIV-1 viral loads of 30,734 cp/mL to ≥1 × 107 cp/mL. In addition, the remaining patient had a R OraQuick result, an indeterminate WB result, and a positive HIV-1 RNA viral load (4,571,787 cp/mL). The Combo Test was R on samples from all 5 of these patients (Table 6).

The ability of the Combo Test to yield R results in samples that are NR by a point-of-care (POC) rapid test but contain high levels of HIV-1 RNA highlights the enhanced clinical utility of the Combo Test in correctly classifying HIV status on the basis of this single test (Table 6; samples 1 to 4). The data in Table 6 underscore the heterogeneity of the time course of antibody production in individuals with significant HIV viremia that can cause third-generation HIV assays, which detect only HIV antibodies, to be NR and fourth-generation assays, which detect both antibodies and HIV p24 antigen, to be R. It is likely that this advantage of fourth-generation HIV screening assays, when coupled with the reduced window period provided by these assays,58,14,15 underlies the central role of these assays in the proposed CDC HIV screening algorithm (Figure 6B).

View this table:
Table 6

Summary of Quantitative HIV-1 Results*

No.OraQuick-FSAvg S/CO Ratio by Combo TestCombo Test ResultWB ResultHIV-1 RNA Viral Load, cp/mL
1NR2.05RInd30,734
2NR35.93RInd≥ 500,000
3NR13.89RInd413,186
4NR132.66RInd≥ 10,000,000
5R48.35RInd4,571,787
  • * Data from Pandori et al.14 OraQuick-FS, OraQuick ADVANCE HIV 1/2 Rapid Test-Fingerstick whole blood sample tested; Avg, average; S/CO, signal-to-cutoff ratio; WB, Western blot; cp, copies; R, reactive; NR, nonreactive; Ind, indeterminate.

Figure 6

HIV screening algorithms: A) algorithm recommended by U.S. Public Health Service in 1989; B) proposed algorithm by CDC APHL HIV Diagnostics Conference, 2010. A1, test-1; A2, test-2; A3, test 3; HIV, human immunodeficiency virus; IA, immunoassay; WB, Western blot; +, reactive or positive; (−), nonreactive or negative; NAAT, nucleic acid amplification test.

View this table:
Table 7

Two-Test Comparisons of OraQuick, STAT-PAK, and Combo Test HIV Results*

2-Test ComparisonPPA (%)NPA (%)OPA (%)
OraQuick vs Combo Test100.0 (74.1–100.0)13.2 (6.5–24.8)28.1 (18.6–40.1)
STAT-PAK vs Combo Test100.0 (74.1–100.0)13.2 (6.5–24.8)28.1 (18.6–40.1)
OraQuick vs STAT PAK100.0 (74.1–100.0)100.0 (93.2–100.0)100.0 (94.3–100.0)
  • * Data adapted from Pandori and colleagues14 (values in parentheses are the 95% score confidence interval).

  • PPA, positive percent agreement; NPA, negative percent agreement; OPA, overall percent agreement.

The difficulty in achieving 100% agreement for HIV-1/2 antibody results for currently available POC and non-POC TMs is illustrated further by the all-participant summary results from a recent CAP Survey (AHIV-C, 2010). Among laboratories participating in this Survey that reported results for the Clearview STAT-PAK (n=358 labs) and OraQuick ADVANCE (n=411 labs) rapid TMs on survey challenge AHIV-13, 306/358 labs (85.5%) reported negative results by the STAT-PAK method, while 372/411 labs (90.5%) reported positive results by the OraQuick method (Table 1). However, referee results (n=18), by an unspecified assay/method on survey sample AHIV-13, yielded 13/18 (72.2%) positive results and 5/18 (27.8%) negative results.

As expected, values for PPA, NPA, and OPA are different for any 2-test comparison between HIV antibody results obtained on samples from a selected vs an unselected population of individuals. This point is illustrated using data from the study by Pandori and colleagues14 on selected samples (n=64) from patients with acute or recent HIV infection, as indicated by positive (>20 cp/mL) quantitative HIV-1 RNA results (range of values: 335 cp/mL to ≥1 × 107 cp/mL). Using these data, we determined values for PPA, NPA, and OPA for each 2-test comparison: OraQuick vs Combo Test; STAT-PAK vs Combo Test; and, OraQuick vs STAT-PAK (Table 7; Note: Pandori and collagues14 did not evaluate the other HIV antibody methods used in our study). As shown in Table 7, values for PPA, NPA, and OPA were all 100% for the OraQuick vs STAT-PAK method; however, when results for both of these methods were compared to those of the Combo Test, PPA was 100% for both tests, but NPA and OPA were considerably smaller: 13.2% and 28.1% for both tests, respectively (Table 7). In addition, in the study by Pandori and colleagues,14 the detection rate (ie, [no. of positive results by TM/total No. of known patients with acute or recent HIV infection tested] × 100) for the OraQuick, STAT-PAK, and Combo TMs was: 17.2%, 17.2%, and 89.1%, respectively.

When using a laboratory test to screen an unselected population for a disease (eg, HIV) for which good to excellent treatment options are available that have an optimum impact on disease course and long-term outcome, the earlier the patient with disease is detected, the test should have high diagnostic sensitivity (ie, few FNs). However, when FPs are also undesirable because of the emotional effect on the patient, there is a premium on high diagnostic specificity (ie, few FPs) as well.

View this table:
Table 8

Potential Causes of False-Negative and False-Positive Results in HIV Assays*

False Positive
False NegativeAny of the following conditions, infections, findings, or lifestyle choices
Pre-analytical or analytical causes
  Hemodilution
  Mislabeling of samples or wells
  Variability in test kits, including the inability to detect HIV-1 subtype O individuals using an HIV kit that does not detect subtype O antibodies
  Masking of HIV antigenic determinants or insufficient antigenic determinants for detection
  By the HIV assay
  Pipetting error
Renal failure
Alpha-interferon therapy in hemodialysis patients
Flu or flu vaccination
HSV I or II
URI
Recent viral infection or vaccination against HBV or with tetanus toxoid
Pregnancy in multiparous women
Malaria
RA
Anti-lymphocyte, -collagen, -smooth muscle, -nuclear, -mitochondrial, -parietal cell, -HAV IgM, -HBc IgM, or -polystyrene antibodies
Heat-treated, lipemic, hemolyzed, or icteric specimens
Normal human ribonucleoproteins
Visceral leishmaniasis
Fungal infections, including TB and Mycobacterium avium
Assay inhibition due to:
  Powder from powdered gloves
  Storing samples in serum separator tubes
Blood transfusions
Myeloma
Hemophilia
Biological, pathologic, or pharmacologic determinants
  Window (preseroconversion) period status
  Delayed antibody synthesis in infants
  Diminished immune response in patients
  • With HIV 1-related immune dysfunction

  • On immunosuppressive therapy

  • With concurrent EBV or CMV infection

Stevens-Johnson syndrome
Organ transplantation
ALD or AH
Autoimmune disease
Receptive anal sex
Congenital or drug-induced hypogammaglobulinemia
  Formation of antigen-antibody complexes
  • * Adapted from Reference 25.

  • HIV, human immunodeficiency virus; EBV, Epstein-Barr virus; CMV, cytomegalovirus; HSV, herpes simples virus; URI, upper respiratory tract infection; HBV, hepatitis B virus; RA, rheumatoid arthritis; ALD, alcoholic liver disease; AH, alcoholic hepatitis; HAV, hepatitis A virus; HBc, hepatitis B core; TB, tuberculosis.

The study by Pandori and colleagues14 demonstrated that the Combo Test had the highest detection rate for acute and recent HIV disease among all CMs tested, including the OraQuick and STAT-PAK tests; other studies1721 have shown high diagnostic sensitivity and specificity for the Combo test; and, the turnaround time (TAT) of test results by all 3 of these tests is approximately the same (∼20–28 minutes; Table 2). It is important to note that immunoassays reported to have 100% diagnostic sensitivity and specificity or 100% agreement between a CM and a TM are not “perfect” (ie, there are never any false-positive or false-negative results) because all immunoassays, regardless of the generation of the assay, are prone to false-positive or false-negative results for a variety of reasons summarized in Table 8.

There has been at least 1 report indicating that fourth-generation HIV assays, albeit not the Combo Test, may still be negative in patients with HIV seroconversion illness (ie, evolving WB reactivity and a positive, often markedly elevated, viral load in a patient with symptoms of HIV infection).22 Clearly, such reports validate the difficulty of any fourth-generation HIV assay in reducing the window period to the time of HIV RNA detectability. To resolve this problem, Stekler and colleagues23 have suggested that, “NAAT be integrated into HIV testing programs that serve populations that undergo frequent testing and that have higher rates of HIV acquisition, particularly if rapid HIV antibody testing is employed.” It remains to be proven if this same statement applies to fourth-generation HIV assays, despite the suggestion in some studies, including the study by Pandori and colleagues,14 based on negative or indeterminate WB results on samples with positive fourth-generation HIV assay and NAAT results, that this may be the case.

Recently, a new algorithm has been proposed to replace the algorithm that has changed little since it was recommended by the U.S. Public Health Service in 1989 (Figure 6A).24 This new algorithm takes advantage of the higher diagnostic accuracy (ie, sensitivity and specificity) of a fourth-generation HIV screening assay and NAAT confirmatory testing (Figure 6B). It should be noted, however, that the WB is not being abandoned completely, as it is well recognized that WB testing will continue to be useful in:

  • Confirming long-standing HIV infection

  • Evaluating diagnostic dilemmas

  • Testing vaccine recipients for antibody response

  • Monitoring patients with end-stage AIDS

The high detection rate of the Combo Test for individuals with acute or recent HIV infection, when coupled with its high diagnostic sensitivity and specificity, rapid TAT (<30 minutes) of test results using a fully automated instrument (ARCHITECT), excellent agreement with HIV-1/2 antibody results by several commercially available HIV-1/2 screening tests (Figure 5), including 1 (OraQuick ADVANCE) of the 2 most popular tests among laboratories participating in the CAP AHIV Survey (Table 1), and with WB and/or NAAT results among discordant samples (Table 5) indicate that the Abbott HIV-1/2 Ab/Ag Test is a useful screening test in the identification of HIV-infected individuals.

Acknowledgments

The authors thank Abbott Diagnostics for providing HIV Ag/Ab Combo Test kits for evaluation.

Abbreviations

HIV
human immunodeficiency virus
CM
comparative method
TM
test method
PPA
positive percent agreement
NPA
negative percent agreement
OPA
overall percent agreement
CLSI
Clinical Laboratory Standards Institute
AIDS
acquired immunodeficiency syndrome
UNAIDS
United Nations Programme on HIV/acquired immunodeficiency syndrome
CDC
Centers for Disease Control
gag
group-specific antigen
EIAs
enzyme immunoassays
CAP
College of American Pathologists
WB
Western blot
NAAT
nucleic acid amplification test
AMH
Arlington Memorial Hospital
BAS
Baylor All-Saints
BUMC
Baylor University Medical Center
THD
Texas Health Presbyterian Hospital, Dallas
THFW
Texas Health Harris Methodist Hospital
THP
Texas Health Presbyterian Hospital
THSW
Texas Health Harris Methodist Hospital Southwest
R
reactive
NR
nonreactive
EDTA
ethylenediaminetetraacetic acid
LL
lower limits
UP
upper limits
S/CO
signal-to-cutoff
%pos
percentage of positive
%neg
percentage of negative
FP
false positive
FN
false negative
POC
point-of-care
TAT
turnaround time

References

View Abstract