| |
Increased Reporting of Detectable Plasma HIV-1 RNA Levels at the Critical Threshold of 50 Copies per Milliliter With the Taqman Assay in Comparison to the Amplicor Assay
|
| |
| |
JAIDS Journal of Acquired Immune Deficiency Syndromes:
May 2009 - Volume 51 - Issue 1 - pp 3-6
doi: 10.1097/QAI.0b013e31819e721b
Rapid Communication
Lima, Viviane PhD; Harrigan, Richard PhD; Montaner, Julio S G MD
From the British Columbia Centre for Excellence in HIV/AIDS, Vancouver, Canada.
Abstract
Objective: To investigate the application of the new COBAS Ampliprep Taqman HIV-1 assay in comparison with the COBAS HIV-1 Ampliprep AMPLICOR MONITOR ultrasensitive assay version 1.5, with a particular focus on the most clinically relevant region near the lower limit of quantification.
Methods: Scatterplots and the Bland-Altman plot were used to inspect the degree of agreement between the 2 assays when tested on samples from the British Columbia Centre for Excellence in HIV/AIDS monitoring and evaluation system. Consistency of clinically applicable values at low HIV-1 RNA copy number was assessed from samples from individuals with previously undetectable Amplicor results.
Results: Despite general agreement of these assays over a wide dynamic range, the Taqman assay resulted in a nearly 2-fold increase (from 3.6% to 6.9%) in the number of patients experiencing a plasma HIV-1 RNA level >50 copies per milliliter after being suppressed to levels <50 copies per milliliter consistently during the previous year (P < 0.01). In addition, rare discrepancies between the 2 assays were observed to be as high as 0.7 log10 copies per milliliter. The kappa statistic was 0.19, indicating only slight agreement at the critical threshold of 50 HIV RNA copies per milliliter, with 43% of undetectable samples in the Amplicor assay testing detectable in the Taqman assay (median 70 copies/mL; interquartile range 60-83 copies/mL).
Conclusions: The increased frequency of detectable plasma HIV-1 RNA levels at the threshold of 50 copies per milliliter with the new Taqman assay has important implications for highly active antiretroviral therapy monitoring. Until there is clinical evidence that patients with detectable HIV by the Taqman assay (but undetectable HIV with Amplicor) have differential outcomes, it is unclear whether the Taqman assay is appropriate for routine management of HIV-1 therapy, and caution is required in the interpretation of low-level viremia by the Taqman assay until a clinical validation of a new cutoff is performed.
INTRODUCTION
According to international guidelines, the stated goal of highly active antiretroviral therapy (HAART) is to suppress plasma HIV-1 RNA to levels below 50 copies per milliliter, given that this cutoff has been shown to delay ongoing virus evolution and T-cell activation.1 This has been shown to enable immune reconstitution, prevent disease progression, and prolong survival.1,2 The data used to derive these guidelines have been mostly based on results from clinical trials and cohort studies using the Roche Amplicor quantitative polymerase chain reaction assay.
The Roche COBAS HIV-1 Ampliprep Amplicor Monitor (from here on referred to as the Amplicor assay) ultrasensitive assay version 1.5 (Roche Diagnostics Systems, Inc, Laval, Quebec, Canada) has been used since April 1, 1999, to measure plasma HIV-1 RNA levels in the St. Paul's Hospital virology laboratory. This facility is the single designated laboratory measuring plasma HIV-1 RNA levels in the province of British Columbia, Canada. On February 1, 2008, a new COBAS Ampliprep Taqman HIV-1 assay (referred to here as the Taqman assay) (Roche Diagnostics Systems) was introduced as an alternative. The Taqman assay is technically simpler and has a wider dynamic range, which is more convenient for the quantification of very high viral load samples.
After this change took place, physicians reported a sudden and surprising increase in the frequency of unexplained low detectable HIV-1 RNA results among patients previously consistently suppressed (<50 copies/mL) on the Amplicor assay. We therefore undertook the present study to investigate the validity of the Taqman assay compared to the Amplicor assay, with a particular focus on the clinically relevant area near the lower limit of quantification.
METHODS
Data from study participants were extracted from the British Columbia Centre for Excellence in HIV/AIDS (the Center) monitoring and evaluation system. The center is responsible for the distribution of antiretroviral agents, at no cost, to all eligible HIV-infected individuals in the province, through its HIV/AIDS Drug Treatment program, according to specific guidelines generated by the Therapeutic Guidelines Committee.3 In all analyses, results obtained using the Amplicor assay were considered as the de facto gold standard. In British Columbia, 95% of the HIV subtype has consistently been clade B subtype, and this has not changed over time. There are a small number of clades A and C subtypes, and only a smattering of other subtypes observed, and the make up of the non-B clades has also not changed over time.
Clinical validity was evaluated in 2 steps. To be eligible in the first analysis, participants were required to be on HAART and to have at least 3 plasma HIV-1 RNA level measurements <50 copies per milliliter between November 1, 2006, and January 31, 2008, measured using the Amplicor assay. Note that the 3 measurements do not necessarily need to be consecutive. For these individuals, the median time of follow-up was 13 months [interquartile range (IQR): 12-14 months], and they had a median of 5 viral load measurements (IQR: 4-7 measurements). Furthermore, these individuals were required to have at least 1 measurement of plasma HIV-1 RNA levels using the Taqman assay between February 1, 2008, and April 30, 2008. A parallel analysis was performed in a second cohort of patients with no change in viral load monitoring. This analysis consisted of patients on HAART who had at least 3 measurements of plasma HIV-1 RNA levels <50 copies per milliliter between November 1, 2005, and January 31, 2007, measured via the Amplicor assay. As before, these 3 measurements do not necessarily need to be consecutive. For these individuals, the median time of follow-up was 13 months (IQR: 12-14 months), and they had a median of 5 viral load measurements (IQR: 3-7 measurements). These patients were also required to have at least 1 measurement of plasma HIV-1 RNA level using the Amplicor test between February 1, 2007, and April 30, 2007. The frequency of newly detectable plasma HIV-1 RNA levels at ≥50 copies per milliliter was compared between these 2 consecutive cohorts. The first cohort, therefore, represents patients switching from the Amplicor to the Taqman assay and the second cohort representing the control group monitored with the Amplicor assay throughout.
The second set of analyses included patients identified in the first analysis who were fully suppressed between November 1, 2006, and January 31, 2008, and who subsequently had at least 1 detectable (≥50 copies/mL) plasma HIV-1 RNA level using the Taqman assay during February 1, 2008, and April 30, 2008. The samples tested using the Taqman assay were then retested using the Amplicor assay using previously stored aliquots. The total storage time was less than 3 months in monitored and alarmed -20°C freezers. These results were used to characterize the sensitivity, specificity, positive and negative predictive values, percent agreement, and the kappa statistic.4 Values <40 and <50 copies per milliliter were coded as 39 and 49 copies per milliliter, respectively. We then took the log10 of 39 and the log10 of 49 and used these values in the scatterplot. Scatterplots and the Bland-Altman plot5 were used to visually inspect the degree of agreement between the 2 assays. The text was revised to reflect this point. Although the Taqman assay has a nominally lower limit of quantification (40 copies versus 50 copies per milliliter for Amplicor), clinical guidelines define a target of 50 copies per milliliter, so this was used as the relevant cutoff. In any case, very few samples fell between 40 and 50, and using different cutoffs made no difference to the conclusions presented here.
All tests were 2 sided, and P values less than 0.05 were considered as indication of statistical significance. All analyses were performed using SAS software version 9.1.3 Service Pack 3 (SAS, Cary, NC). The center's drug treatment program including mandated analyses deriving from them such as those presented herein have been vetted and approved by the University of British Columbia Research Ethics Board at its St. Paul's Hospital site. Individuals in the center's drug treatment program do not provide written informed consent for the purposes of the analyses presented herein. These administrative analyses occur within the context of a universal health care system where individuals receive medical care, laboratory monitoring, and antiretrovirals free of charge. The ethical approval includes publication in scientific journals and presentations at scientific meetings.
RESULTS
In an initial validation study of 978 samples (predominantly subtype B), an excellent correlation (88% in the log10 scale using the Spearman correlation coefficient) between the Amplicor and Taqman assays was observed over the entire range of quantification of these assays (C. Sherlock, personal communication). After the routine introduction of the Taqman assay, a sudden and anomalous increase in the number of individuals with newly detectable HIV RNA levels led to further investigation. In the first analysis, a total of 1806 individuals were fully suppressed between November 1, 2006, and January 31, 2008, using the Amplicor assay. Of those, 124 individuals (6.9%) were not suppressed below 50 HIV-1 RNA copies per milliliter based on the Taqman assay results between February 1, 2008, and April 30, 2008. By comparison, in the previous year, a total of 1774 individuals had fully suppressed HIV-1 RNA between November 1, 2005, and January 31, 2007, using the Amplicor assay. Of those, 63 (3.6%) were not suppressed based on the same assay between February 1, 2008, and April 30, 2008. These results suggest that the introduction of the Taqman assay may have produced a nearly 2-fold increase (from 3.6% to 6.9%) in the number of patients apparently experiencing a plasma HIV-1 RNA level >50 copies per milliliter after being consistently suppressed to levels <50 copies per milliliter during the previous year.
In the second analysis, 215 viral load samples from 124 individuals previously tested with the Taqman assay were retested with the Amplicor assay. We reassayed 159 (74%) of the 215 eligible HIV-1 RNA plasma samples because of a shortage of residual sample. Figure 1A shows the scatterplot of the results of both assays. The values are poorly correlated in the range 1.6-2.0 log10 copies per milliliter, the area where critical clinical decisions are made regarding the management of patient's disease progression. Figure 1B shows the Bland-Altman plot describing the degree of agreement between both assays. The mean values (SD) were 1.86 (0.57) and 1.82 (0.56) log10 copies per milliliter for the Taqman and Amplicor assays, respectively. The maximal discrepancy between the 2 assays was 0.72 log10 copies per milliliter, and a total of 18 samples (11%) showed a discrepancy of more than 0.30 log10 copies per milliliter.
Table 1 shows the cross tabulation of samples with plasma HIV-1 RNA levels <50 versus ≥50 copies per milliliter in both Taqman and Amplicor assays. A total of 59 viral load samples (37%) were suppressed in both assays. The Taqman reported considerably more samples as having viral loads >50 HIV RNA copies than the Amplicor (P value <0.01). Comparing the results of the Taqman assay to the Amplicor assay showed a 0.46 sensitivity, 0.88 specificity, 0.94 positive predictive value, 0.29 negative predictive value, and 45% percent overall agreement. The kappa statistic was 0.19 (95% confidence interval 0.10 to 0.29) indicating only slight agreement for samples near the limit of detection of the assays.
DISCUSSION
Overall, these 2 HIV RNA assays perform comparably over their range of quantification. However, our results showed an increased frequency of detectable plasma HIV-1 RNA levels near the clinically critical threshold of 50 copies per milliliter with the new HIV-1 RNA COBAS Taqman Assay in comparison to the HIV-1 RNA COBAS Ampliprep Amplicor Assay. There was poor agreement near this lower limit of quantification, and a surprisingly high proportion of individuals who were fully and consistently suppressed on HAART with HIV-1 RNA levels <50 copies per milliliter using the Amplicor assay experienced unexpected HIV-1 RNA levels >50 copies per milliliter when the Taqman assay was introduced. Comparing the results of the Taqman assay with the Amplicor assay, as the current gold standard, we obtained a 0.46 sensitivity, 0.88 specificity, 0.94 positive predictive value, 0.29 negative predictive value, and a 45% percent agreement. The kappa statistic of 0.19 (95% confidence interval 0.10 to 0.29) indicates only slight agreement. Some disagreement is to be expected as the assays approach their lower limit of quantification, but the data indicate an important systematic difference in reporting of low-level viremia depending on the assay used.
Other comparisons of these assays have been presented: Damond et al6 observed that among 140 viral load samples measured by these 2 same assays, 34 (24%) showed discrepancies of at least 0.5 log10 copies per milliliter. The mean values obtained with the TaqMan and the Amplicor assays were 2.9 (SD 0.54) and 4.2 (SD 0.82) log10 copies per milliliter, respectively, corresponding to a difference of 1.3 log10 copies per milliliter (P value < 0.01). In comparison to our study, the frequency of discordant results from this study by Damond et al6 was lower (25%) than that from the study presented here (43%). In contrast, articles by Oliver et al7 and Manavi8 have reported on differences between these 2 assays, with the Taqman assay having a higher detectability than the Amplicor assay. In Oliver et al,7 3.7% (n = 187) samples had TaqMan results ≥0.5 log10 copies per milliliter higher than the results using Amplicor, and 3.1% (n = 187) samples gave results ≥0.5 log10 copies per milliliter lower than the Amplicor assay. Additionally, only 6.4% (n = 187) of samples that were detectable using the TaqMan assay were undetectable based on the Amplicor assay. Manavi8 reported on only 14% (n = 113) of samples being detectable using the TaqMan assay being undetectable based on the Amplicor assay. Also, Gueudin et al9 reported on 26% of their study samples showing the same degree of discrepancy between assays. Our findings, however, represent the first formal assessment of the new HIV-1 RNA COBAS Taqman Assay against the comparison to the HIV-1 RNA COBAS Amplicor Assay in a real clinical setting. In this context, it is important to emphasize that we focused our interest on the critical threshold of 50 copies per milliliter because this is widely regarded as the target of HAART according to international guidelines.1 These guidelines were developed using the Amplicor assay viral load threshold. Indeed, this value is the only viral load threshold indicated for therapy monitoring in the treatment guidelines, suggesting that development and assessment of viral load assays should be focused at least as much on the consistency of detection at the 50 copies per milliliter limit as on the precision and accuracy across the rest of the range of these assays.
The increased frequency of detectable plasma HIV-1 RNA levels at the critical threshold of 50 copies per milliliter with the new HIV-1 RNA COBAS Ampliprep Taqman Assay in comparison to the HIV-1 RNA COBAS Ampliprep Amplicor Assay described here can have large practical implications for HAART monitoring, despite a relatively small absolute magnitude of difference between the assays. We are not at all convinced that the Taqman reflects true viremia. Rebeiro et al10 have clearly shown that the Taqman is more susceptible to artifacts from primary plasma preparation tubes (PPT). Although the PPT tube artifact does not affect our results (given that we do not use PPT tubes), other artifacts may be causing spurious signals in the Taqman assay. The real issue is the clinical (and practical) significance of these signals of low-level viremia. Because the treatment guidelines and clinical trials are based on endpoints of low level viremia as measured by the standard Amplicor, this is the quandary. The increased frequency of detectable plasma HIV-1 RNA levels at the threshold of 50 copies per milliliter with the new Taqman assay invariably leads to additional viral load testing, repeated medical visits, and significant stress for the patient. It is currently impossible to distinguish whether the Taqman or the Amplicor is more likely to report real results.
In summary, our data demonstrate that the increased frequency of detectable plasma HIV-1 RNA levels at the threshold of 50 copies per milliliter with the new Taqman assay has important implications for HAART monitoring. Until there is clinical evidence that patients with detectable HIV by the Taqman assay (but undetectable HIV with Amplicor) have differential outcomes, it is unclear whether the Taqman assay is appropriate for routine management of HIV-1 therapy. Special caution should therefore be exercised in the interpretation of low-level viremia by the Taqman assay until a prospective clinical validation of the assay is completed.
|
|
| |
| |
|
|
|
