| |
Mainstreaming Rapid Oral HIV testing
|
| |
| |
EDITORIAL
AIDS: Volume 20(12) 1 August 2006 p 1667-1668
Rietmeijer, Cornelis A; Thrun, Mark W
From the Denver Public Health Department, Departments of Medicine and Preventive Medicine and Biometrics, University of Colorado at Denver and Health Sciences Center, Denver, Colorado, USA.
Last December, two newspaper articles, one in the San Francisco Chronicle [1] and the other in the New York Times [2], reported on clusters of false-positive results of oral fluid rapid HIV testing in a number of settings. This led to fears among public health officials involved in HIV prevention efforts that the rapid tests for HIV, particularly those tests involving oral fluid specimens, were not performing as well in the real world as expected from studies in more controlled settings. Given the increasing popularity of rapid testing in the USA and elsewhere, the papers by Wesolowski et al. [3] and Delaney et al. [4] in this issue of AIDS are reassuring. In an evaluation of a combined total of over 38 000 oral fluid tests performed in a wide variety of clinical and non-clinical sites, rapid testing using oral fluid specimens, in general, performed well within the boundaries specified by the manufacturer of the test. While slightly more false positives were found with oral fluid compared to whole blood testing, the specificity in both studies fell within the 99.6-99.9% confidence interval listed by the manufacturer. There were a number of exceptions of sites where the test appeared to perform notably worse. However, in-depth investigation of these sites suggested that the false-positive test results may have been associated with remediable operant errors rather than poor performance of the test itself, although intrinsic problems with the use of oral fluid specimens due to cross-reacting antibodies may have also played a role. (Note from Jules Levin: Read both studies below & I think you'll see that number of false positives were relatively small but they did occur and a significant number of them were not due to operator errors. The study suggests that confirmatory testing & proper operator training should address this).
Why are these findings relevant? Enhancing HIV testing is increasingly seen as critical both in providing care to HIV-infected patients at an earlier stage and as an important tool in preventing the further spread of HIV. In the USA, a refocusing of HIV prevention efforts was heralded about 3 years ago by the Centers for Disease Control and Prevention (CDC) in their Advancing HIV Prevention initiative [5]. Among its four key recommendations, two relate to enhancing HIV testing: making testing routine in medical settings and to develop innovative ways to increase testing in high-risk, non-medical settings. In both settings, the rapid HIV test plays an important role in increasing test uptake. It is simple to use, specimens are easy to obtain, processing of the test can be done outside the laboratory in the field, and a quick turn-around time allows communication of results within 20 min, thus virtually guaranteeing that persons with positive tests obtain results immediately.
The encouraging findings from Wesolowski et al. [3] and Delaney et al. [4] will no doubt lead to a further mainstreaming of HIV testing. Rapid testing allows for HIV testing on a routine basis in settings where high-risk individuals are frequently encountered, such as gay bars, bathhouses, and needle-exchange venues, as well as clinical sites where the challenge of scheduling follow-up visits has limited the broader implementation of standard HIV enzyme immunoassay (EIA). In these latter settings, including sexually transmitted infection (STI) clinics and emergency departments, HIV testing is increasingly offered as part of a standard package of services, unless the client/patient specifically declines to be tested for HIV; so-called 'opt-out' testing [6].
From an HIV medical care perspective, early detection of HIV is critical in effectively managing its sequelae. As was noticed recently in these pages, survival is significantly shorter in patients who initiate treatment when they have progressed to lower CD4 cell counts despite virological suppression [7]. While in the early years of the HIV/AIDS epidemic HIV testing was often actively discouraged because no treatment options were available, therapeutic interventions can now be life saving or at least can increase the quantity and quality of life, so that one can indeed argue that withholding or discouraging HIV testing is unethical.
There is also growing consensus that the early diagnosis of HIV infection has prevention benefits. Those who are aware of their infections are significantly reducing high-risk behaviours, thus limiting the number of secondary infections compared to individuals who are unaware of their HIV serostatus [8]. Moreover, the cost-effectiveness of HIV counselling and testing as a means to prevent HIV transmission has been documented extensively [9].
While HIV testing is cost-effective and likely even cost-saving from a societal perspective, it should be noted that many HIV prevention programmes are cost-effective or cost-saving simply because HIV disease is so costly (US$200 000 for lifetime medical care costs alone). Thus even the prevention of one or a few cases may render a typical prevention programme cost-effective. As is true of many public health interventions, these savings do not often benefit the agency that incurred the costs of the prevention activities, but rather society at large, which has been reluctant to funnel those savings back to the prevention programmes. In fact, a paltry 4% of all federal HIV-related expenditures in the USA are used for prevention, and this proportion has been steadily declining over the years [10]. With the limited resources available, HIV prevention programmes thus have to be increasingly frugal in spending their funds. While not the intent of CDC's prevention initiative, a shift to HIV testing may thus come at the expense of other HIV prevention programmes. This may be problematic. A recently published modelling study supported by the RAND Corporation suggests that HIV counselling and testing in general may be less cost-effective than other prevention programmes, including community-level interventions targeting high-risk populations, needle-exchange programmes, and counselling and testing of partners of persons with HIV [11]. As with other modelling studies, the underlying assumptions in this study are debatable. Also, a strategy involving rapid testing without counselling was not considered.
Nonetheless, there is no doubt that mainstreaming of new HIV testing modalities could lead to a watershed moment in the prevention and diagnosis of HIV akin to the advent of highly active antiretroviral therapy (HAART) in the treatment of HIV. It would be regrettable, however, if the emphasis on testing would eclipse prevention programmes that have proven their worth in the numerous studies spawned by the HIV epidemic over the past 25 years.
Performance of an oral fluid rapid HIV-1/2 test: experience from four CDC studies
AIDS: Volume 20(12) 1 August 2006 p 1655-1660
Delaney, Kevin Pa; Branson, Bernard Ma; Uniyal, Apurvab; Kerndt, Peter Rc; Keenan, Patrick Ad; Jafa, Krishnaa,e; Gardner, Ann Df; Jamieson, Denise Jg; Bulterys, March
From the aDivision of HIV/AIDS Prevention - Surveillance and Epidemiology, National Center for HIV, STD and TB Prevention, Centers for Disease Control and Prevention, Atlanta, Georgia, USA
bHealth Research Association, Inc., Los Angeles, California, USA
cSTD Control Program, Los Angeles County Department of Public Health, Los Angeles, California, USA
dDepartment of Family Medicine and Community Health, University of Minnesota School of Medicine, Minneapolis, Minnesota, USA
eEpidemiology Program Office, Office of Workforce and Career Development, Centers for Disease Control and Prevention, Atlanta, Georgia, USA
fArizona Department of Health Services, Division of Public Health Services, Office of HIV/AIDS, Phoenix, Arizona, USA
gDivision of Reproductive Health, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, Atlanta, Georgia, USA
hGlobal Aids Program, National Center for HIV, STD and TB Prevention, Centers for Disease Control and Prevention, Lusaka, Zambia.
Abstract
Objective: To evaluate the performance of a rapid HIV antibody test used with whole blood and oral fluid in settings where the test is likely to be used.
Design: In four separate studies, we compared the accuracy of the rapid test performed on whole blood and oral fluid specimens with the results of conventional HIV tests.
Methods: Oral fluid and whole blood from persons of unknown HIV status recruited from clinics, labor and delivery units, and outreach venues were tested with the OraQuick Advance rapid HIV-1/2 antibody test. Sensitivity and specificity were compared with results of the enzyme immunoassay (EIA) and Western blot algorithm used by the study sites.
Results: OraQuick sensitivity was 99.7% with whole blood and 99.1% with oral fluid from 327 persons who were HIV antibody positive by the conventional algorithm. OraQuick specificity was 99.9% with whole blood and 99.6% with oral fluid from 12 010 HIV-negative persons; EIA specificity was 99.7%. A cluster of 16 false-positive oral fluid tests occurred in one study, in which specificity was lower (99.0%) than in the other three studies (99.6-99.8%).
Conclusions: In diverse settings in four studies, the OraQuick test showed high sensitivity and specificity for HIV antibody in whole blood and oral fluid specimens. Slightly more false-positive and false-negative results occurred with oral fluid than with whole blood, but performance with both specimen types was similar to, or better than, that of conventional EIAs.
Introduction
In March 2004, the US Food and Drug Administration (FDA) approved the OraQuick rapid HIV-1 antibody test for detection of antibodies to HIV-1 in oral fluid [1]. (At the time of the studies reported here the device was called OraQuick HIV-1. On 22 June 2004 the FDA approved an additional indication for detection of HIV-2 in oral fluid, whole blood and plasma specimens and change of the name to OraQuick Advance. Website: http://www.fda.gov/cber/pma/p01004716.htm )
Waived under the Clinical Laboratory Improvement Amendments of 1988 (CLIA) [2], this test is intended as a point-of-care screening test for HIV antibodies when used with oral fluid, fingerstick and venous whole blood specimens. The test can be performed with 5 ml of whole blood or on an oral fluid specimen collected by swabbing the flat pad of the test device once around the outer surface of the upper and lower gums [3]. Results are read in 20-40 min. The manufacturer indicates a specificity of 99.8% [95% confidence interval (CI): 99.6-99.9] with oral fluid and 100% (95% CI: 99.7-100) with whole blood [3].
Previously, only one system for HIV testing of oral fluid had received FDA approval for use in the USA [4]. Conventional oral fluid testing follows the algorithm for testing blood [5]: specimens are screened by enzyme immunoassay (EIA); those that are repeatedly reactive are confirmed by Western blot. Results for EIA-reactive, Western blot-negative specimens are reported as HIV negative. The process of laboratory EIA/Western blot testing may take several days to weeks.
Oral fluid collection for HIV testing offers significant advantages in outreach settings: it is non-invasive, can be conducted almost anywhere, eliminates costs of phlebotomy training and equipment, and reduces biohazardous risks and waste. In 2000, at testing sites in the USA funded by the Centers for Disease Control and Prevention (CDC), nearly 30% of the 1.53 million tests were performed on oral fluid, most often in field outreach or other non-clinical settings [6]. Persons tested at these sites were more likely to test positive than persons at serum testing sites. However, approximately half of those tested failed to return for their test results [6]. Thus, an accurate oral fluid rapid HIV test could help persons at higher risk to both access testing and learn their test results on the same day.
Since 2000, CDC has sponsored four studies in which OraQuick whole blood and oral fluid testing was performed on the same individual in settings where the test is likely to be used. In this paper, we compare the accuracy of OraQuick testing with results of the conventional EIA and Western blot algorithm in these four studies.
Methods
Study settings and participants
One of these four studies (A), conducted between July 2003 and December 2004, included pregnant women undergoing HIV screening at 18 hospitals in six US cities as part of the Mother-Infant Rapid Intervention At Delivery (MIRIAD) Study [7,8]. The other three studies enrolled high-risk persons at (B) 41 community outreach sites in Minneapolis, Minnesota (MN) between July 2002 and August 2004, and three HIV testing sites and two sexually transmitted disease (STD) clinics in (C) Los Angeles, California (LA) between April 2000 and January 2005, and (D) Phoenix, Arizona between April 2001 and February 2003. In each of the four studies, all participants provided oral fluid and fingerstick or anticoagulated whole blood specimens for testing with OraQuick (on both oral fluid and whole blood.) At hospital sites, OraQuick was either performed in the laboratory by trained technicians or in labor and delivery by trained nurses, midwives and/or physicians [7]. At community outreach sites, OraQuick was performed by trained HIV counselors, most of whom did not have previous laboratory training. In Los Angeles and Phoenix, OraQuick was performed by trained technicians in onsite laboratories.
Gold standard comparison tests
The specimen type and EIAs and Western blots used as gold standard comparison tests varied by study. The LA, Phoenix and MN studies used the Vironostika HIV-1 Microelisa (bioMerieux Inc., Durham, North Carolina, USA), the EIA most commonly used in state public health labs in the USA [9]. Serum specimens repeatedly reactive by EIA and those from persons with reactive rapid tests were tested with Genetic Systems HIV-1 Western blot (Bio-Rad Laboratories, Inc., Hercules, California, USA) (LA and MN) or the Cambridge Biotech Western blot (Calypte Biomedical, Rockville, Maryland, USA) (Phoenix). In MIRIAD, FDA-approved EIA and Western blot tests were performed on blood specimens according to protocols already in place at participating hospitals; none used the Vironostika Microelisa [7]. In MN, the specimen collected for comparison testing was determined by the result of the fingerstick whole blood OraQuick test. For participants with negative fingerstick OraQuick results, oral fluid specimens were collected for testing with the Vironostika Microelisa approved for use on oral fluid, and if repeatedly reactive, by the Orasure Western blot (Orasure Technologies, Bethlehem, Pennsylvania, USA). For participants with reactive fingerstick OraQuick results, serum specimens were collected and tested with comparison tests as described above.
Quality assurance and quality control measurements
Staff in all four studies were trained to perform the test according to the instructions in the manufacturer's package insert [3]. Initial quality assurance guidance for these studies included the following: both a positive and negative external control was run each day testing was performed, and staff were asked to record the test lot number, daily temperatures in device storage and testing environments, and the times when the test was started and read. The MN study continued to run positive and negative external quality control tests daily for the duration of the study. The other three studies ran external quality controls in accordance with the OraQuick package insert [3] and CDC quality assurance guidelines [10] when these were released in 2003.
Analyses
Only participants with valid results for both rapid and conventional tests were included in the analysis. Participants with indeterminate Western blot results were excluded from calculations of rapid test performance because true infection status could not be determined in this group. Sensitivity, specificity, and positive and negative predictive values were calculated based on the results of the EIAs and Western blots from each site using standard formulas. Exact 95% confidence intervals for these proportions were calculated [11]. For calculation of EIA specificity, persons with a non-reactive EIA, a reactive screening EIA that was non-reactive on repeat testing, or a negative Western blot result after a repeatedly reactive EIA were classified as uninfected. Differences in performance of tests on the same individuals were evaluated using McNemar's test. Differences in performance on the same specimen type across studies were evaluated using Fisher's exact test. All analyses were conducted using SAS system version 8.2 for Windows (SAS Institute Inc., Cary, North Carolina, USA).
Human subjects
All four studies were conducted under protocols approved and monitored by institutional review boards at CDC and at each participating site.
Results
Sensitivity
Of 12 343 participants, 6 with indeterminate Western blots whose infection status was unresolved were excluded from further analysis. EIA and Western blot confirmed 327 (2.7%) of the remaining 12 337 as HIV positive (Table 1). Of these, 326 tested positive by OraQuick with whole blood (sensitivity 99.7%) and 324 with oral fluid (sensitivity 99.1%, P = 0.63). Two of the three persons with false-negative oral fluid tests had reactive whole blood OraQuick tests. No Western blot positive person had a reactive oral fluid and non-reactive whole blood OraQuick test. Sensitivity of OraQuick on each specimen type varied slightly but not significantly across the four studies (Table 1).
Specificity
Overall
Of the 12 010 persons who tested negative by the reference algorithm, 11 975 were EIA negative and 35 were initially reactive by EIA but negative by the EIA/Western blot algorithm (EIA specificity 99.7%). For all studies combined, OraQuick specificity was 99.9% with whole blood and 99.6% with oral fluid (P < 0.0001) (Table 1). Nine (0.07%) participants were OraQuick false positive with both blood and oral fluid; 3 (0.02%) were false positive with only whole blood, and 45 (0.37%) were false positive with only oral fluid.
Minnesota study
Although specificity was slightly lower with oral fluid than with whole blood in all four studies, in the Minnesota study, oral fluid specificity (99.0%) was significantly lower (P < 0.05) than in any of the other three studies. In the first 2 years of the study, among 2017 HIV-negative persons, oral fluid test performance in the Minnesota study was similar to that observed in the other studies (specificity 99.7%, 95% CI: 99.3-99.8).
The decrease in observed specificity was attributed to 16 false-positive results, of which only 1 was also OraQuick false positive with whole blood, occurring in 388 HIV-negative persons tested (specificity 95.9%, 95% CI: 93.4-97.6) between April 2004 and the end of the study in August 2004.
Outreach workers performing the test recalled that, during this period, oral fluid tests with very faintly reactive results occurred which appeared qualitatively different from the usual weakly reactive OraQuick result, including some test lines that they described as gray or without color. However, the intensity and color of the test lines was not recorded during the study. Daily temperatures were recorded in both test and external control storage logs from 15 April 2004 through 31 August 2004, and in individual test logs completed at testing sites from 7 June 2004 through 31 August 2004. All temperatures were within the manufacturer's specifications for OraQuick at that time (2-27 C for storage and 15-27 C for testing.) All test devices run as part of external quality control gave the expected results. The false positives occurred with devices from six different test lots. The manufacturer reported that all implicated test lots were manufactured and shipped according to standard procedures and that all components met quality control specifications.
Positive and negative predictive value
OraQuick specificity with whole blood was 99.9% in all four prospective studies, but the observed positive predictive value of a reactive whole blood OraQuick test ranged from 81.8% among Minnesota participants (HIV prevalence 0.3%) to 98.6% among Los Angeles participants (HIV prevalence 5.1%) (Table 1). In all studies, the positive predictive value of OraQuick with oral fluid was lower than that observed with OraQuick screening of whole blood. Although estimates of the positive predictive value of oral fluid OraQuick and conventional EIA test results varied (Table 1) these differences were not significant (data not shown). The predictive value of a negative rapid test was >99.9% with whole blood and oral fluid in all four studies.
Discussion
In four separate studies, the OraQuick test demonstrated high sensitivity and specificity for HIV antibody with both whole blood and oral fluid specimens. Our findings are consistent with the clinical trial data reported by the manufacturer to the FDA [3] and other evaluations [12,13]. We also found that OraQuick sensitivity and specificity were lower with oral fluid than with whole blood. The negative predictive value was high in all four studies, and thus, counselors and clients can have confidence that a negative OraQuick test result, in the absence of a recent exposure to HIV, is conclusive.
The lower specificity with oral fluid is certain to have practical implications, especially in populations with low HIV prevalence. The 95% confidence intervals for specificity for the combined studies suggest that false-positive OraQuick tests can be expected to occur at a rate 2-6 times higher with oral fluid than with whole blood. In low-prevalence settings, this will reduce the positive predictive value considerably. Although we found that the positive predictive values for OraQuick with both oral fluid and whole blood were comparable to, or sometimes better than, that of conventional EIAs, unlike the EIA, clients receive the rapid test result at the point of care, before confirmation. Thus, counselors and clients must be aware of the limitations of reactive rapid HIV screening tests and the need for confirmation in accordance with current guidelines [14].
The consistent performance of OraQuick in real-world settings represented by these four studies is reassuring. Similar accuracy was achieved in different populations (pregnant women, high-risk persons with both high and low HIV prevalence) and by both laboratory technicians and persons with little or no prior experience with laboratory testing (hospital labor and delivery staff and trained HIV counselors).
However, we observed a cluster of false-positive oral fluid OraQuick tests in the Minnesota study. Although the causes for this and other reported clusters are currently unknown [15,16], possible operator errors such as overcollection of samples, which we did not assess in any of the four studies reported here, or differences in the interpretation of very faint or gray lines, which were interpreted as preliminary positive in Minnesota and not reported in any of the other three studies, may have played a part in these clusters [16]. Furthermore, in the Minnesota study, information on medical conditions, e.g. Epstein-Barr virus, hepatitis A or B infection, rheumatoid factor or multiparity, which may be associated with false-positive results [3], was not collected. Efforts to recontact clients to obtain this information after the study had ended were unsuccessful. Importantly, all clients with false-positive preliminary rapid HIV test results will be correctly classified as uninfected if CDC guidelines for confirmation are followed. Rapid test providers must also implement appropriate quality-assurance procedures to monitor operators and test performance, and to ensure that operators follow the manufacturer's instructions consistently [3,10]. Higher than expected numbers of false-positive OraQuick test results should be reported to the manufacturer, which is obligated to investigate and report such complaints to the FDA [15].
In these four studies, most serum and oral fluid specimens were screened with the Vironostika HIV-1 Microelisa, which uses a whole viral lysate substrate, and confirmed with Western blot. Seroconversion studies [3,17] and post-marketing surveillance [14] suggest that this EIA and the Western blot are less sensitive to early infection than some newer EIAs and rapid tests. Thus, it is possible that some clients classified as negative by the EIA used as the gold standard were actually infected with HIV. If such clients were missed by both the Vironostika and OraQuick tests, sensitivity estimates reported here would be artificially inflated. Conversely, misclassifying reactive OraQuick tests of truly infected persons as false positive would have biased the reported specificity downwards. Theoretically the latter form of misclassification could partially explain the cluster observed in the Minnesota study. However, none of the 327 Western blot positive persons identified in these studies had a reactive oral fluid and non-reactive whole blood OraQuick test, as was observed in 15 of the 16 false-positive oral fluid tests in the reported cluster. The sensitivity of the whole blood test suggests it is unlikely that false-negative whole blood OraQuick and Vironostika tests could explain the decrease in specificity observed in Minnesota, given the prevalence of HIV infection observed in that study. The problems of imperfect gold standards exist in all evaluations of antibody tests. In practice, discrepancies can only be resolved through follow-up testing. A limitation of these four studies is the lack of follow-up testing for persons with discordant (false-positive or false-negative) OraQuick results relative to these imperfect gold standards; for such clients their gold standard result was considered definitive.
A recently published randomized controlled trial which assessed client HIV testing preferences in two outreach settings [18] found that, compared with conventional tests, more clients accepted testing with oral fluid tests and rapid tests, and more clients tested with either of these alternatives received their test results. Because oral fluid OraQuick tests produce results almost immediately, most clients learn their HIV test result. With conventional oral fluid collection, nearly half of those who test HIV-positive never receive their results [6]. Thus, using the OraQuick test with oral fluid in outreach settings where obtaining blood specimens is not feasible will help to identify more HIV-positive persons, even though its sensitivity is slightly lower with oral fluid than with whole blood [19]. In settings where blood is routinely available, the difference in specificity should be considered when deciding which specimen to use for OraQuick testing.
With adequate quality assurance, the CLIA-waived OraQuick Advance test produced accurate results with both oral fluid and whole blood. Such CLIA-waived, versatile tests offer the ability to provide HIV testing that is acceptable and useful in outreach settings for persons at high risk for acquiring HIV infection who may not learn their HIV status any other way. The convenience, safety and acceptability of non-invasive oral fluid collection, combined with nearly immediate rapid test results, suggest there will be a continuing demand for oral fluid rapid tests, even if they are slightly less sensitive and specific than whole blood or serum tests. Use of the OraQuick test, currently the only rapid test approved for oral fluid, will help identify more HIV-infected persons and link them to vital care and treatment services.
Post-marketing surveillance of OraQuick whole blood and oral fluid rapid HIV testing
AIDS: Volume 20(12) 1 August 2006 p 1661-1666
Wesolowski, Laura Ga; MacKellar, Duncan Aa; Facente, Shelley Nb; Dowling, Terib; Ethridge, Steven Fa; Zhu, Julia Ha; Sullivan, Patrick Sa; for the Post-marketing Surveillance Team
From the aDivision of HIV/AIDS Prevention, National Center for HIV, STD and TB Prevention, Centers for Disease Control and Prevention, Atlanta, Georgia, USA
bSan Francisco Department of Public Health, AIDS Office, HIV Prevention Section, San Francisco, California, USA.
Abstract
Objective: Post-marketing surveillance was conducted to monitor the performance of the OraQuick Advance rapid HIV-1/2 antibody test (OraQuick) on whole blood and oral fluid.
Design: Surveillance of routinely collected data on clients tested with OraQuick in 368 testing sites affiliated with 17 state and city health departments between 11 August 2004 and 30 June 2005.
Methods: For whole blood and oral fluid, we report the median (range) health department OraQuick specificity and positive predictive value (PPV), and the number of clients with discordant results (e.g. who had a reactive rapid test not confirmed positive by Western blot or indirect immunofluorescence). At one site with lower than expected oral-fluid specificity, we evaluated whether device expiration, manufacturing lot, operator practices, or device-storage or testing-area temperatures were associated with false-positive tests.
Results: During the surveillance period, 135 724 whole blood and 26 066 oral fluid rapid tests were conducted. The median health department whole blood OraQuick specificity was 99.98% (range: 99.73-100%) and PPV was 99.24% (range: 66.67-100%); the median oral fluid specificity was 99.89% (range: 99.44-100%) and PPV was 90.00% (range: 50.00-100%). A total of 124 discordant results were reported from 68 (0.05%) whole blood and 56 (0.22%) oral fluid rapid tests. The oral fluid specificity at the site with excess oral fluid false-positive tests was 98.7% (95% confidence interval: 98.18-99.11%). The increase in false-positive tests at that site was not associated with any specific device characteristic, operator procedure or temperature condition.
Conclusion: The specificity of OraQuick performed on whole blood and oral fluid during post-marketing surveillance was compatible with the manufacturer's claim within the package insert. However, one site experienced lower than expected oral fluid specificity. Sites that observe that the specificity of OraQuick is lower than the range indicated in the package insert should notify the manufacturer and evaluate quality assurance procedures.
Introduction
Approximately 24-27% of the 1 039 000-1 185 000 persons in the United States living with HIV do not know their serostatus [1]. The Advancing HIV Prevention initiative, introduced by the Centers for Disease Control and Prevention (CDC) in 2003, aims to reduce the number of HIV-infected persons who are unaware of their status [2]. As part of this initiative, CDC purchased and distributed OraQuick Advance rapid HIV-1/2 antibody tests (OraQuick) to health departments and community-based organizations, and conducted post-marketing surveillance to monitor the product's performance. OraQuick was the first rapid HIV test waived under the Clinical Laboratory Improvement Amendments of 1988 for use at point of care [3]. It can be used by trained operators in non-clinical settings on whole blood and oral fluid. The manufacturer's specificity claim for whole blood is 100% (95% confidence interval (CI): 99.7-100%) and 99.8% for oral fluid (95% CI: 99.6-99.9%) [4]. In this paper, we describe the whole blood and oral fluid specificity and positive predictive value of OraQuick at health departments that conducted surveillance, and the outcomes of an investigation at one site that reported excess oral fluid false-positive tests.
Methods
Design
Health departments in three cities (Chicago, New York City and San Francisco) and 14 states (Arizona, Delaware, Florida, Indiana, Louisiana, Massachusetts, Michigan, Montana, Nebraska, New Jersey, New York, North Carolina, Utah and Wisconsin) selected rapid test sites in their jurisdictions to participate in post-marketing surveillance. Whole blood and oral fluid OraQuick tests conducted at 368 sites between 11 August 2004 and 30 June 2005 were included in surveillance. Specimens from clients with reactive rapid tests were submitted to local laboratories for Western blot (WB) or indirect immunofluorescence (IFA) confirmation. Clients with discordant results (e.g. those who had a reactive rapid test not confirmed positive by WB or IFA) were asked to return for follow-up testing 4 weeks after their initial test [5]. Demographic data and HIV test results were entered into local health department data systems. Electronic data were submitted to CDC in accordance with variable maps designed to create one standard dataset.
Definitions
Non-reactive OraQuick tests were assumed to be true negatives. A true positive was defined as a reactive OraQuick test followed by a positive WB or IFA. Initially discordant results were reclassified as: (1) true positive if the follow-up WB or IFA was positive or the client had a detectable viral load; (2) false positive if the last follow-up confirmatory tests was negative or if no record of follow-up testing was obtained; and (3) indeterminate if the last WB or IFA was indeterminate. Specificity was defined as the number of negative tests divided by the sum of negative and false-positive tests. Positive predictive value (PPV) was defined as the number of true-positive tests divided by the sum of true-positive plus false-positive tests.
Analyses
For each health department, we report the number of participating sites and the following outcomes by specimen type: the number of negative and true-positive tests, percent seropositivity, number of false-positive tests, specificity and PPV. Exact confidence intervals were calculated for specificity using the Clopper-Pearson method [6]. The median (range) seropositivity, specificity and PPV for the 17 health departments were also calculated. Specificity was compared with the manufacturer's claim for whole blood and oral fluid [4]. Statistical comparisons were made using the chi-square test. All analyses were conducted using SAS (Statistical Application Software v9.1, SAS Institute Inc., Cary, North Carolina, USA).
Investigation of excess false-positive oral fluid tests
One site (site X) in San Francisco that conducted oral fluid tests from March 2005 to November 2005 reported an excess of false-positive tests starting in June 2005. Staff from the San Francisco Department of Public Health, California State Office of AIDS, CDC, and OraSure Technologies conducted an investigation to assess whether device expiration, manufacturing lot, operator practices, device-storage or testing-area temperatures might have been associated with these false-positive tests. As part of our investigation of excess false-positive tests, we report the seropositivity, specificity and PPV for site X by month. We also compare the specificity of oral fluid OraQuick at site X with 11 other rapid test sites in San Francisco from March to November 2005.
Human subjects
Surveillance activities and investigations were conducted as part of post-marketing surveillance determined at CDC to be non-research.
Results
Surveillance population characteristics
A total of 162 317 OraQuick tests were conducted at the 368 sites. Of these, 29.5% were conducted at counseling and testing sites, 29.0% at sexually transmitted disease (STD) clinics, and 17.7% at correctional facilities; 63.8% were conducted on males, 50.0% on clients 29 years of age, and 41.7% on clients who were black, 39.8% white, 17.4% Hispanic, 2.1% Asian, 1.3% American Indian/Alaskan Natives, and 0.3% Native Hawaiian or other Pacific Islander. Of the OraQuick tests, 135 724 (83.6%) were conducted on whole blood, 26 066 (16.1%) on oral fluid, and 527 (0.3%) on an unrecorded specimen type.
Discordant results
A total of 124 initially discordant results were reported; 68 (0.05%) of 135 724 whole blood rapid tests, and 56 (0.22%) of 26 066 oral fluid rapid tests. Of the 68 discordant whole blood results, 13 (19.1%) were classified as true positive, 15 (22.1%) as indeterminate and 40 (58.8%) as false positive. Of the 56 discordant oral fluid results, 4 (7.1%) were classified as true positive, 5 (8.9%) as indeterminate and 47 (83.9%) as false positive. Of discordant results that were classified as false positive, a similar proportion of false-positive whole blood versus false-positive oral fluid results were based on 1 follow-up tests (52.5 versus 38.3%, P = 0.18).
Seropositivity, specificity, positive predictive value
The median HIV seropositivity was 0.83% (range among 17 health departments: 0.08-2.60%) among tests conducted using whole blood and 1.00% (range: 0-4.02%) among tests conducted using oral fluid. The median specificity was 99.98% (range: 99.73-100%) for whole blood and 99.89% (range: 99.44-100%) for oral fluid. The median PPV was 99.24% (range: 66.67-100%) for whole blood and 90.00% (range: 50.00-100%) for oral fluid. Point estimates for whole blood specificity at all health departments exceeded 99.7%, the lower bound of the 95% confidence interval of the manufacturer's specificity claim [4]. Point estimates for oral fluid specificity at all health departments except one (Arizona) exceeded 99.6% (Table 1). At this health department, the upper bound of the 95% CI for oral-fluid specificity overlaps the lower bound of the 95% CI of the manufacturer's claim (Table 1).
Investigation of excess oral fluid false-positive tests
Specificity and positive predictive value TOP
At site X in San Francisco, of 2585 oral fluid OraQuick tests conducted, one was invalid and 33 (1.3%) were classified as false positive (11 of the 33 did not have follow-up testing). The oral fluid specificity at this site fell below the manufacturer's claim and was significantly lower than 11 other sites in San Francisco (Table 2). The number of false-positive oral fluid tests increased from July through October (Fig. 1). Because of staff concerns, the site discontinued use of oral fluid OraQuick on 16 November 2005.
Device characteristics
The 33 false-positive OraQuick tests at site X were from unexpired devices from four lots: 17 (51.5%) from one lot, 13 (39.4%) from a second, 2 (6.1%) from a third and 1 (3.0%) from a fourth. Each of these lots was used at other sites in San Francisco. Of the 33 devices with false-positive results, 29 (87.9%) had test lines (lines that denote a preliminary positive result) that were qualitatively described as 'gray', 'extremely faint' or 'shadowy'.
Operator practices
Seven primary operators at site X performed and interpreted the 33 false-positive OraQuick tests within 20-40 min. One operator read 14 (42.4%) devices, one read eight (24.2%), one read seven (21.2%) and four read one each (12.0%). In all 33 cases, the test line was read by more than one operator before the preliminary positive result was given to the client. Operator practices were observed to be in accordance with the package insert, with the exception of oral fluid specimen collection (some operators recommended that clients swab the upper and lower gum lines more than once with the OraQuick device). Operators were retrained on oral fluid specimen collection practices in October. Thirteen (39.4%) of the 33 false-positive results occurred after this retraining.
External controls and temperatures
From March through November 2005, staff performed 163 (6.3% of tests conducted) OraQuick external control runs at site X. Of these, two invalid results were observed; the remaining results were concordant with the negative or positive control. All temperatures recorded in storage and test logs during this period were within the manufacturer's specifications.
Discussion
From August 2004 through June 2005, surveillance of the OraQuick Advance rapid HIV-1/2 antibody test on over 150 000 clients in 17 health departments demonstrated that the device performed in accordance with manufacturer's specificity claim for use with both whole blood and oral fluid. Our findings support those from independent studies that found high OraQuick specificity for whole blood and oral fluid [7-9]. However, in accordance with the manufacturer's claim, we found that OraQuick specificity on oral fluid is slightly lower than that on whole blood [4]. Based on our observed median specificities, 11 false-positive results can be expected for every 10 000 oral fluid OraQuick tests performed on non-infected persons, and 2 false-positive results can be expected for every 10 000 whole blood OraQuick tests on non-infected persons.
Given the low HIV seropositivity observed in some health department sites, we were not surprised to find some low positive predictive values. The lower PPV for oral fluid than whole blood tests at some health departments resulted primarily from fewer HIV-infected persons who tested with oral fluid rather than from lower specificity of the oral fluid test. These findings underscore the importance that all persons who have a reactive rapid HIV test need to be informed that the positive result is preliminary and that confirmatory testing is necessary [10]. When the confirmatory test following a preliminary positive rapid test is not positive, current guidelines for follow-up testing should be heeded [5,11].
Post-marketing surveillance was subject to several limitations. First, health departments and rapid test sites were not randomly selected and some sites may have been selected to participate because of their prior experience with rapid testing or higher quality test performance. If true, our specificity estimates might be overestimates. It is unlikely, however, that OraQuick performance at test sites included in surveillance was very different than at sites that were not included. Approximately half of the 162 317 rapid tests were conducted by five health departments that included all of their rapid test sites in post-marketing surveillance. Also, the specificity at these five health departments was comparable with all other health departments. Second, false-positive designations were based on the Western blot and IFA, which are older assays and which might be less sensitive in identifying persons with recent infection compared with newer generation rapid tests [12]. However, additional diagnostic tests performed on the 87 clients with false-positive results indicate that HIV was not present. Remnant sera from clients with false-positive results were tested at CDC from initial confirmatory specimens (n = 55) and from second confirmatory specimens (n = 22) and all tests were non-reactive by third-generation enzyme immunoassay. Additionally, all 32 nucleic acid amplification tests conducted on specimens from the 87 clients with false-positive results had undetectable viral loads. These results were concordant with Western blot and IFA results on these clients.
We observed lower oral-fluid OraQuick specificity at site X compared with other sites in San Francisco. We did not identify a clear cause for the lower specificity at this one site. Manufacturing error seems an unlikely cause since the lots used at this site were used at other sites in San Francisco where an increase in whole blood false-positive tests did not occur. Although it is possible that the observed lower specificity at site X was a random event due to chance alone, the increase in false positives is probably attributed to one or more unmeasured site-specific operator or host factors. These factors may be sufficiently rare that they are difficult to identify.
One or more operators may have routinely overcollected oral fluid specimens by instructing clients to swab around the gums more than once, rather than once as recommended [4]. Although unknown, it is plausible that overcollection of oral fluid might result in higher concentrations of antibodies that either cross-react or bind non-specifically with antigen in the test strip. Also, increased saliva might affect dilution or flow through the test device. However, over one-third of the false-positive tests at site X occurred after operators were retrained on oral fluid collection, and operators generally performed the tests correctly when observed. The reported increase in false-positive tests at site X might also be attributed to differences in interpreting test results. In other rapid test sites, gray lines may be unrecognized by staff or classified as non-reactive or invalid. The manufacturer should consider modifying the package insert to clarify how gray lines should be interpreted [4].
During post-marketing surveillance, whole blood and oral fluid OraQuick performed in the specificity range documented in the package insert. However, in one site, OraQuick specificity fell below this range during a specific time period. Operators who observe preliminary positive test results on whole blood or oral fluid specimens should continue to follow existing counseling and confirmatory testing guidelines [13]. If test providers observe an increase in false-positive OraQuick rapid HIV tests, they should notify the manufacturer, and assess adherence to package insert instructions and CDC quality assurance guidelines [4,11].
|
|
| |
| |
|
|
|
