Performance evaluation of HIV infection diagnostic tests

1. Instituto Adolfo Lutz, São Paulo, São Paulo, Brazil
2. Universidade Federal de Minas Gerais (UFMG), Belo Horizonte, Minas Gerais, Brazil
3. Fundação Arnaldo Vieira de Carvalho, Faculdade de Ciências Médicas da Santa Casa de São Paulo, São Paulo, São Paulo, Brazil

DOI: 10.5935/1676-2444.20200024

Corresponding author

Márcia Jorge Castejon
ORCID 0000-0002-3525-740X
e-mail: marcia.castejon@ial.sp.gov.br

First Submission on 7/29/2019

INTRODUCTION

The number of new infections caused by the human immunodeficiency virus (HIV) has been increasing worldwide. In the last seven years, there was reduction of just 18% of new infections, from 2.2 million in 2010 to 1.8 million in 2017. Although it is almost half of the number of new infections compared with the peak recorded in 1996 (3.4 million), the decline is not fast enough to reach the target of fewer than 500 thousand people by 2020⁽¹⁾.

In 2017, in Brazil, 42,420 new HIV cases and 37,791 cases of the acquired immunodeficiency syndrome (Aids) were diagnosed, with a detection rate of 18.3/100 thousand inhabitants, constituting a 15.7% decrease. This reduction has been more accentuated since the recommendation of “treatment for all” implemented in December 2013. In that same year, among men, the Southeast region presented a predominance of the exposure category in homosexuals (46.3% of the cases) and bisexuals (9.5%), while in the other regions it happened in heterosexuals⁽²⁾. In recent years, studies have indicated an important increase in HIV infection among men who have sex with men (MSM)^{(3, 4)}.

In the whole world, HIV infection is a public health issue, in which MSM population has been disproportionately affected⁽⁵⁾. Owing to the high incidence in this group, and the highest estimated proportion of infected cases⁽²⁾, it is important to assess the ability of tests to identify HIV cases in the initial phase of antibody production.

Researches on HIV prevention among the population of transsexual women have been scarce⁽⁶⁾. A literature review estimated that transsexual women are 49 times more likely to be infected by HIV than the general population – a possibility higher than in any other population⁽⁶⁾. HIV prevalence seems to be higher among transvestites and transsexual women than among any other population in Brazil – and might be increasing^{(6, 7)}, although surveillance data combine transvestites, transsexual women and MSM into the same category. According to a study carried out by Baral et al. (2013)⁽⁶⁾ with transgender women from several countries, the encountered prevalence in Brazil was 33.7% [95% confidence interval (CI) 28.7-39.4].

Especially for the MSM population, data from the literature suggest that early HIV diagnosis is the main tool for adequate care of infected people⁽⁸⁾. In this regard, HIV testing has been a key prevention strategy; also, early identification of the infection by means of frequent testing and the timely start of antiretroviral therapy (ART) offer enormous benefits both to subjects and community⁽⁵⁾. Accurate and opportune detection of HIV primary infection is fundamental for the future health of the infected individual and to prevent virus transmission⁽⁹⁾.

In Brazil, serological assays for diagnosis of HIV infection must be carried out according to recommendations by the Ministry of Health, which advocates the employment of test flowcharts, defined according to the aim of testing: diagnosis, surveillance, or blood donor selection⁽¹⁰⁾.

Many countries resort to diagnosis by means of rapid tests (RT) for HIV screening, due to difficulties for the conduction of automated testing. Although RT quality is promising to increase screening safety in blood banks, uncertainty around performance of some point-of-care RTs has stimulated debate on their application. Because of this, it is necessary to have knowledge about the usage conditions and the reasons that can lead to deficient RT performance⁽¹¹⁾.

Studies performed in key populations reveal the preference for rapid testing when compared with conventional serology. Thus, RTs are fundamental for improved access to diagnosis, being an important epidemiological tool. In contrast, some researchers highlight the low sensitivity of some RTs in the presence of recent infections leading to false-negative results. Identification of subjects in the acute phase of the disease, with high viral load, can provide an opportunity for preventive and therapeutical interventions⁽¹²⁾.

It is fundamental to assess the methods of antibody detection used in algorithms of HIV testing concerning their analytical performance. Anti-HIV antibodies appear shortly after HIV infection and generally increase in the first 6-12 months of infection⁽¹³⁾. Different factors can influence the quality of laboratory exam results, among them: systematic and random errors intrinsic to assays; biological factors of the host and agent, such as viral diversity and ART; lack of training and knowledge updating^{(10, 13)}.

Evidence suggests that early treatment with ART, even with an elevated level of CD4+ T lymphocytes, has positive effects on the health and well-being of people affected by HIV. The impact of ART upon the reduction of HIV transmission, besides improving patients’ quality of life, prevents new infections by the virus along with other prevention options currently available⁽¹⁴⁾. ART can have an impact on RT performance, supposedly by reducing the production of anti-HIV antibodies, as very low levels of viremia observed with successful therapy may be insufficient to provide maintenance of a specific anti-HIV-1 immune response⁽¹⁵⁾. Thus, individuals on ART who do not disclose their HIV-positive status may obtain false-negative results in HIV tests due to viral suppression and the low concentration of anti-HIV antibodies in the blood^{(13, 16, 17)}.

With this in mind, false-negative results in the RTs used for HIV diagnosis can damage data from a population survey. Therefore, this work was aimed at assessing the performance of point-of-care RTs used for HIV diagnosis compared with conventional assays. Blood samples came from the incidence study in a population of transvestites and transsexual women in the city of São Paulo, São Paulo, Brazil.

METHODS

For the development of this study, the research project “Estudo de Coorte Transnacional – avaliação de desempenho de métodos utilizados no diagnóstico da infecção pelo HIV” was recorded in the Technical and Scientific Council of Instituto Adolfo Lutz (CTC-IAL 26-I/2016), after approval of the ethical aspects by the Ethics Research Committee of Centro de Referência e Treinamento DST/Aids (CRT-DST/Aids) (Report no. 1.937.513).

The analyzed serum specimens came from the study “The transnational cohort: global HIV epidemiology and prevention research for transwomen”, which aims at estimating HIV incidence, social and biomedical events of gender transition, risk behaviors related to gender affirmation and the multiple stress forms in all behavioral ecological levels of influence associated with HIV acquisition. The establishment of a transnational cohort was based on recruitment through the Respondent Driven Sampling (RDS) strategy. The population was formed of HIV-negative transvestites and transsexual women residents of the city of São Paulo, who accepted to participate in this study by signing the informed consent form.

In the period from April 2017 to March 2019, 1,467 specimens from 836 volunteers (transvestites and transsexual women) who accepted to participate in the cohort were evaluated by RTs.

Serological screening for HIV, by means of flowchart 1 advocated by the Ministry of Health⁽¹⁰⁾, was conducted with an initial immunochromatographic assay for detection of anti-HIV 1+2 antibodies – RT1 [ABON (Abon Biopharm, Hangzhou, China)]; the samples with reactive results underwent a second immunochromatographic assay other than the first one – RT2 [HIV Test Bioeasy (Standard Diagnostic, Inc., Yongin-Si, Korea)]. The used RTs were acquired by the Brazilian Ministry of Health and conducted at CRT-DST/Aids, São Paulo.

Among the 1,467 blood samples examined at RT1, 1,385 (94.4%) were collected by venous puncture and referred to the Immunology Center at IAL (CIM-IAL) for the conduction of conventional laboratory tests. Several reasons precluded the conduction of conventional laboratorial tests with the other 82 (5.7%) samples, including: blood collection by digital puncture, volunteers’ refusal to consent storage of blood, and sample loss during transportation.

Among the 1,385 samples, 1,145 (82.7%) obtained a non-reactive result at RT1, and 240 (17.3%) samples were reactive at RT1 and RT2. It is worth emphasizing that besides the sample tubes used for serum tests (RTs and laboratory conventional tests), a blood tube (plasma) for molecular tests was drawn and stored (-70°C).

Conventional serological tests for HIV diagnosis were carried out with reagents available at CIM-IAL, and samples were analyzed according to flowchart 6, made available by the Ministry of Health⁽¹⁰⁾. Regardless of the result obtained at RT, in sample analysis, the chemiluminescent immunoassay (CLIA) initial test was used [Advia Centaur HIV Ag/Ab Combo (CHIV); Siemens Healthcare Diagnostics, Inc., NY, USA] and, when reactive, the rapid immunoblot (RIB) confirmatory test [Imunoblot Rápido DPP HIV1/2 (IBR DPP HIV); Bio-Manguinhos, Fiocruz, Rio de Janeiro, Brazil]. For samples with non-reactive or indeterminate result at RIB, the plasma sample underwent the molecular test. Initially, plasma was processed (Abbott HIV RT-PCR) and the ribonucleic acid (RNA) of the sample was extracted in the automated instrument Abbott Real Time M2000sp and analyzed by means of the real time polymerase chain reaction (RT-PCR) (Abbott Real Time M2000rt), according to recommendations by the manufacturer; the result was expressed in copies/ml.

Samples that obtained reactive results at the CLIA and RIB assays were considered reactive for HIV, as well as those presenting viral load (VL) ≥ 5,000 copies/ml.

Agreement levels between RT1 and CLIA results and flowcharts 1 and 6 were interpreted by means of the Kappa (K) index, as proposed by Altman (1999)^{(18, 19)} and adapted from Landis and Loch (1977). K value < 0.2 represents poor agreement; 0.21-0.4, fair; 0.41-0.6, moderate; 0.61-0.8, good; 0.81-1, very good agreement.

RESULTS

From April 2017 to March 2019, the HIV/Aids laboratory of CIM-IAL received the 1,385 serum samples from the volunteers (transvestites and transsexual women) and the results obtained in the RT (flowchart 1). Specimen evaluation was performed in the conventional serological assays, according to recommendations of flowchart 6 by the Ministry of Health. Of this total, 795 (57.4%) samples came from the first collection; 292, from the second; 199, from the third; and 99, from the fourth, because of the cohort follow-up to measure HIV incidence.

At the HIV/Aids laboratory of CIM-IAL, regardless of the results obtained in the RTs, all samples were tested in the CLIA and, when reactive, analyzed in the RIB, according to the recommendations advocated at flowchart 6 of the technical manual by the Ministry of Health.

Table 1 illustrates the results of 1,385 samples obtained from RTs and conventional laboratory assays for HIV.

The 240 (17.3%) samples with reactive results in both RTs (RT1 and RT2) demonstrated agreement of 100% by conventional laboratory tests; 239 results were concluded by serological assays; and one by HIV-1 VL above 5,000 copies/ml. According to flowchart 1, the 1,145 (82.7%) samples with non-reactive result at TR1 did not undergo TR2.

As shown by data described in Table 1, in the sample (it refers to the volunteer’s second sample) with indeterminate RIB result (gp41 band reactivity), VL test was carried out (flowchart 6), which presented viremia of 2,757,243 copies/ml. For diagnostic confirmation, a new sample (third sample) was drawn and results were all reactive, including at RIB, with reactivity in all HIV-1 specific bands (gp160, g120, gp41, p24) detectable in this assay. It is worth highlighting that the first sample of this volunteer presented non-reactive result both at TR and CLIA.

All samples (n = 240) with reactive results at RTs (RT1 and RT2) were also reactive at CLIA. Concerning the 1,145 samples non-reactive to HIV at RT1, four (0.3%) were reactive at CLIA and, when tested at RIB, three (75%) confirmed positivity and one (25%) tested non-reactive (absence of bands).

In the sample with non-reactive results at RT and RIB and reactive result at CLIA, the index value found was close to the cut-off (2.52 index/reference value ≥ 1). At the VL test, that sample did not produce signal of RNA presence (non-detected RNA), being interpreted as indeterminate for HIV.

Pursuant to criteria established for interpretation of K index, the degree of agreement for RT1/CLIA was considered very good (K = 0.99). Similarly, the degree of agreement for HIV diagnosis by flowchart 1 (two RTs)/flowchart 6 (CLIA and RIB/VL) was also considered very good (K = 0.99). In both, the proportion of agreement was around 100%.

Table 2 illustrates sample reactivity in the different assays and collection periods performed approximately each six months (first, second, third, and fourth).

In Table 2, the inclusion of samples for diagnostic confirmation stands out. Even with result defined as reactive, a new blood sample was drawn for assay conduction. The different number of specimens for confirmation between RT and CLIA/RIB concerns a sample whose result was reactive in conventional assays since the first sample.

DISCUSSION

The new developed technologies, the diversity of available tests in the market, and the different settings in which HIV diagnosis is made reinforce the necessity of frequent evaluations of the used assays, especially in what regards cases of recent infection. The main deficiencies of tests for HIV early detection were corrected, largely with the advent of fourth generation assays (Ag/Ab)⁽²⁰⁾. In retrospective studies involving samples with serological profile of negative HIV-1 Western blot and reactive nucleic acid test (NAT), when evaluated in immunoassays of fourth and third generation, the encountered reactivity was 62%-83%, and 20%-37%, respectively^{(9, 21)}. In this study, in a sample that initially presented reactive result at CLIA and indeterminate result at RIB (presence of HIV-specific band that did not meet the positivity criterion established by the Ministry of Health) – with availability for the conduction of VL test (2,757,243 copies/ml) – it was possible to establish the definitive laboratory diagnosis (sample reactive to HIV). It is a recent infection and in those cases, the use of an initial fourth-generation test (high sensitivity) followed by the VL complementary test is no doubt the most adequate way to confirm it⁽²²⁾.

In the serum evaluation with a new sample (third sample) of that person, it was observed that the reactivity profile was present in all bands of the used RIB (gp160, gp120, gp41, p24); this demonstrates that the appearance of a marker in circulation depends on the infection stage^{(23, 24)}. This case presents one of the situations that can be more frequent with the improved identification and incorporation of people living with HIV, having as consequence the increased proportion of recent cases among the total of infected subjects. The adoption of proper methods and flows enables the early diagnosis of the infection, with an impact on virus transmission and the onset of new cases⁽¹⁰⁾.

Several factors can impair the results of the research, especially when it comes to samples from a study of HIV incidence. Among them, participants may not reveal their serological status and the use of ART and, based on a result non-reactive to HIV, besides being included in the cohort study, they can believe their initial results were incorrect or even that they are cured.

As reported by Fogel et al. (2017)⁽¹³⁾, it is important to recognize that the increased availability and early use of ART can impact the results of population surveys in which incidence and prevalence of HIV are key outcomes. The characteristics of antibody formation can be altered in patients with acute HIV infection who receive ART (virus suppression). In this setting, patients can have an incomplete evolution of antibody responses and will rarely show complete or partial seroconversion^{(9, 13, 25)}. HIV diagnosis is susceptible to flaws, among them the limitation inherent in the test itself, such as its sensitivity. According to recommendations by the Ministry of Health⁽¹⁰⁾, during the use of ART, RT is not indicated. Therefore, it is important to evaluate performance in a survey on HIV incidence at a population of transvestites and transsexual women, because the presence of false-negative results at RTs can impair the quality of data in this study.

As observed by Delaney et al. (2011)⁽¹⁶⁾, when in the presence of negative results at RTs, individuals with recent potential of exposure and those with high risk of contracting HIV infection must be advised to go through a new evaluation. In our study, three false-negative results were detected at RT, but the samples, when analyzed again at CIM-IAL by the same used RT, although of different lots, showed results reactive to HIV. As a matter of fact, the quality of results can be influenced by several factors, such as errors during the analytical and post-analytical phases (problems intrinsic to the employed kit lot, errors in result transcription, change of sample tubes, among others)⁽²⁶⁾. The rapid evolution of processes has required better professional training and the adoption of efficient management tools to ensure result quality. Hence, it is necessary to expand knowledge, monitor and train the team of professionals involved in the conduction of tests in order to qualify and quantify flaws in the (pre-analytical, analytical, and post-analytical) processes to help in the implementation of corrective and preventive measures and target efficacy⁽²⁷⁾.

One of the samples defined as undetermined [CLIA (2.52/1) reactive (confirmed in duplicate), RT and non-reactive RIB and non-detected VL], is probably a false-reactive result at the screening laboratory test. False-positive results can appear with the use of any test or method, regardless of the used flowchart, either due to the limitation of the method itself or to the characteristics inherent in the individual. Flowcharts have been employed to reduce the chances of false-reactive results, by the use of additional tests to confirm the presence of HIV infection. Another possibility to obtain those results is when the subject has early initiated ART and has not revealed the previous serological status. Non-attendance by the bearer at CRT-DST/Aids biannual follow-up precluded the collection of a new specimen to elucidate laboratory diagnosis.

CONCLUSION

This study demonstrates a high agreement rate among the results obtained from RTs and conventional laboratory assays. In cases of recent infection, it is clearly important to use the laboratory testing algorithm that includes an initial screen with the fourth-generation assay followed by the VL test to determine HIV status. Although a small percentage of false non-reactive results at RT have been found, as it is a single method, this evaluation reinforces the need to adopt effective management tools to ensure the quality of HIV infection diagnosis.

ACKNOWLEDGEMENTS

The authors thank André R. Campos and Elizete S. Goya from CIM-IAL, and Norberto Camilo Campos by the technical laboratory support in the conduction of this work. They also thank the members of the study project “The transnational cohort: global HIV epidemiology and prevention research for transwomen”, especially José Luis G. Gonzales Junior, Paula M. S. D. Silveira and Gustavo S. R. Saggese; and the professionals of CRT-DST/Aids, São Paulo, for the conduction of HIV rapid tests.

REFERENCES

1. Programa Conjunto das Nações Unidas sobre HIV/Aids. Estatísticas globais sobre HIV 2018. UNAIDS Brasil. Available at: https://unaids.org.br/2018/07/unaids-alerta-o-progresso-esta-lento-e-nosso-tempo-esta-acabando-para-alcancarmos-as-metas-de-hiv-ate-2020.

2. Brasil. Ministério da Saúde. Secretaria de Vigilância em Saúde. Boletim Epidemiológico HIV/Aids 2018. Brasília 2018. Available at: http://www.aids.gov.br/pt-br/pub/2018/boletim-epidemiologico-hivaids-2018.

3. Veras MASM, Calazans GJ, Ribeiro MCSA, et al. High HIV prevalence among men who have sex with men in a time-location sampling survey, São Paulo, Brazil. AIDS Behav. 2015; 19(9): 1589-98. PubMed PMID: 25384906.

4. Kerr L, Kendall C, Guimarães MDC, et al. HIV prevalence among men who have sex with men in Brazil: results of the 2^nd national survey using respondent-driven sampling. Medicine. 2018; 97: S9-S15. PubMed PMID: 29794604.

5. Conway DP, Guy R, Davies SC, et al. Rapid HIV testing is highly acceptable and preferred among high-risk gay and bisexual men after implementation in Sydney Sexual Health Clinics. PLoS One. 2015; 10(4): e0123814. doi:10.1371/journal. pone.0123814. PubMed PMID: 25898140.

6. Baral SD, Poteat T, Strömdahl S, Wirtz AL, Guadamuz TE, Beyrer C. Worldwide burden of HIV in transgender women: a systematic review and meta-analysis. Lancet Infect Dis. 2013; 13(3): 214-22. PubMed PMID: 23260128.

7. Martins TA, Kerr LR, Macena RH, et al. Travestis, an unexplored population at risk of HIV in a large metropolis of northeast Brazil: a respondent-driven sampling survey. AIDS Care. 2013; 25(5): 606-12. PubMed PMID: 23082818.

8. van Griensven F, Guadamuz TE, de Lind van Wijngaarden JW, Phanuphak N, Sunil Suhas Solomon SS, Lo YR. Challenges and emerging opportunities for the HIV prevention, treatment and care cascade in men who have sex with men in Asia Pacific. Sex Transm Infect. 2017; 93(5): 356-62. PubMed PMID: 28729520.

9. Wood BR, Spach DH. National HIV curriculum. Acute and recent HIV infection. Washington 2018. Available at: https://www.hiv.uw.edu/go/screening-diagnosis/acute-recent-early-hiv/core-concept/all.

10. Brasil. Ministério da Saúde. Portaria SVS/MS nº 29, de 17 de dezembro de 2013. Aprova o manual técnico para o diagnóstico da infecção pelo HIV em adultos e crianças e dá outras providências. Available at: http://www.aids.gov.br.

11. Pruett CR, Vermeulen M, Zacharias P, Ingram C, Tagny CT, Bloch EM. The use of rapid diagnostic tests for transfusion infectious screening in Africa: a literature review. Transfus Med Rev. 2015; 29(1): 35-44. PubMed PMID: 25447555.

12. Adams S, Luo W, Wesolowski L, et al. Performance evaluation of the point-of-care INSTI™ HIV-1/2 antibody test in early and established HIV infections. J Clin Virol. 2017; 91(1): 90-4. PubMed PMID: 28372890.

13. Fogel JM, Piwowar-Manning E, Debevec B, et al. Impact of early antiretroviral therapy on the performance of HIV rapid tests and HIV incidence assays. J Acquir Immune Defic Syndr. 2017; 75(4): 426-30. PubMed PMID: 28471839.

14. World Health Organization. Guideline on when to start antiretroviral therapy and on pre-exposure prophylaxis for HIV. Genebra 2015. Available at: https://apps.who.int/iris/bitstream/handle/10665/186275/9789241509565_eng.pdf;jsessionid=4C3788E14544055234EAC4E8D9B48338?sequence=1.

15. Bongertz V, Ouverney EP, Fernandez SC, et al. Anti-human immunodeficiency virus type 1 humoral immune response and highly active antiretroviral treatment. Mem Inst Oswaldo Cruz. 2007; 102(7): 817-25. PubMed PMID: 18060317.

16. Delaney KP, Branson BM, Uniyal A, et al. Evaluation of the performance characteristics of 6 rapid HIV antibody tests. Clin Infect Dis. 2011; 52(2): 257-63. PubMed PMID: 21288853.

17. Piwowar-Manning E, Fogel JM, Laeyendecker O, et al. Failure to identify HIV-infected individuals in a clinical trial using a single HIV rapid test for screening. HIV Clin Trials. 2014; 15(2): 62-8. PubMed PMID: 24710920.

18. Cohen JA. Coefficient of agreement for nominal scales. Educ Psychol Meas. 1960; 20(1): 37-46. Available at: http://journals.sagepub.com/doi/abs/10.1177/001316446002000104.

19. Altman DG. Practical statistics for medical research. Inter-rather agreement. 8 ed. London: Chapman & Hall; 1999. p. 403-9.

20. Alexander TS. Human immunodeficiency virus diagnostic testing: 30 years of evolution. Clin Vaccine Immunol. 2016; 23(4): 249-53. PubMed PMID: 26936099.

21. Centers for Disease Control and Prevention. Laboratory testing for the diagnosis of HIV infection: updated recommendations. Atlanta 2014. Available at: http://dx.doi.org/10.15620/cdc.23447.

22. Fogel JM, Piwowar-Manning E, Donohue K, et al. Determination of HIV status in African adults with discordant HIV rapid tests. J Acquir Immune Defic Syndr. 2015; 69(4): 430-8. PubMed PMID: 25835607.

23. Fiebig EW, Wright DJ, Rawal BD, et al. Dynamics of HIV viremia and antibody seroconversion in plasma donors: implications for diagnosis and staging of primary HIV infection. AIDS. 2003; 17(13): 1871-9. PubMed PMID: 12960819.

24. Taylor D, Durigon M, Davis H, Archibald C, Konrad B, Coombs D. Probability of a false-negative HIV antibody test result during the window period: a tool for pre- and post-test counselling. Int J STD AIDS. 2015; 26(4): 215-24. PubMed PMID: 25033879.

25. Metadilogkul O, Jirathitikal V, Bourinbaiar AS. Serodeconversion of HIV antibody-positive AIDS patients following treatment with V-1 immunitor. J Biomed Biotechnol. 2008; 2009: 934579. PubMed PMID: 18989372.

26. Castejon MJ, Yamashiro R, Oliveira CAF, et al. Performance of rapid tests compared to conventional tests used for HIV diagnosis. J Bras Patol Med Lab. 2018; 54(6): 364-71.

27. Vieira KF, Shitara ES, Mendes ME, Sumita NM. A utilidade dos indicadores da qualidade no gerenciamento de laboratórios clínicos. J Bras Patol Med Lab. 2011; 47(3): 201-10. Available at: http://www.scielo.br/pdf/jbpml/v47n3/v47n3a02.pdf.