Abstract
Background/Aim: The present study compared the accuracy of visually analyzed (VA) and automatically analyzed (AA) ColonView (CV) quick test; a new-generation fecal immunochemical test (FIT) for hemoglobin (Hb) and hemoglobin/haptoglobin (Hb/Hp) (Biohit Oyj, Helsinki, Finland) in subjects participating in colorectal neoplasia (CRN) detection in Brazil. A traditional guaiac-based fecal occult blood test (gFOBT) test (HemoccultSENSA) was used as a reference. Patients and Methods: A cohort of 509 colonoscopy-referral patients were asked to collect three consecutive fecal samples, to be analyzed by both CV and SENSA. Results: In ROC analysis for the AA reading, the optimal cut-off value for CV Hb was ≥8.0912 and that for CV Hb/Hp was ≥1.8983. With these cut-offs, the sensitivity (Se), specificity (Sp), and efficiency of CV AA in detecting colorectal adenoma (CRA) were: 64.2%/78.6%, 53.4%/35.3%, and 58.6%/56.5%, for Hb and Hb/Hp, respectively. In the HSROC analysis, the AUC values for i) VA and ii) AA modes were as follows: i) AUC=0.551 (95%CI=0.500-0.602), ii) AUC=0.606 (95%CI=0.550-0.662). The difference between these AUC values was statistically significant (p=0.0160). Conclusion: The present study confirms the previous results on the applicability of the ColonView quick test in CRN screening. Of the two optional reading modes, the AA reading showed significantly better diagnostic accuracy as compared to the VA reading (or SENSA), in detecting the CRA endpoint in colonoscopy-referral patients.
- Fecal occult blood
- fecal immunochemical test
- FIT
- colorectal adenoma screening
- sensitivity
- specificity
- false negative
- false positive
- ROC
- HSROC
Although, several randomized trials have shown a decrease in colorectal cancer (CRC)-related mortality in patients who undergo screening with guaiac-based fecal occult blood testing (gFOBT) (1-6), the clinical trials have demonstrated that the diagnostic performance of FITs is superior to standard gFOBTs in detecting colorectal neoplasia (CRN) (7-12). Recent studies have also confirmed that using the FITs results in higher participation rates in CRC screening, because of a simplified sampling protocol, not interfered by the dietary restrictions necessary for the gFOBTs (7, 9, 11-13). Other important issues to be considered while selecting the test for CRC screening include i) the ideal cut-off value for hemoglobin (Hb) detection, ii) the relative value of qualitative and quantitative tests, as well as iii) the number of samples needed to reach an optimal sensitivity and specificity. According to the European Guidelines for quality assurance in CRN screening (7), another advantage of the FITs in population screening is that these tests can be automated and the end user can adjust the cut-off at which a positive result is reported (14). Although, an increasing number of FITs have been developed and are available on the market, relatively few of those have been tested for diagnostic accuracy in CRC screening. Current data imply, however, that new FITs have superior test characteristics as compared with gFOBTs (7, 15, 16).
Vasilyev et al. (11) and Guimaraes et al. (12) have previously compared the diagnostic accuracy of HemoccultSENSA (SENSA) and CV in a head-to-head comparison study among CRN screening patients. The present study is an extension of these analyses by applying hierarchical summary receiver operating characteristic (HSROC) and area under curve (AUC) analyses to test different cut-offs and to compare the diagnostic accuracy of the two reading modes of CV: Visual analysis (VA) and automatic analysis (AA), using colorectal adenoma (CRA) as the endpoint in patients referred to colonoscopy for detection of CRN.
Patients and Methods
The Barretos CRN screening cohort included 5,090 subjects, but 4,316 patients were ineligible for the study, 193 patients disagreed to participate, in 37 patients colonoscopy was missing and 38 patients had other reason for exclusion (Figure 1). Three fecal samples were requested and all subjects underwent diagnostic colonoscopy with biopsy confirmation. The study protocol and inclusion/exclusion criteria of study patients were detailed in a previous report by Guimaraes et al. (12).
Sample collection, processing and interpretation of results. A new-generation FIT, ColonView® quick test (subsequently CV) (Biohit Oyj) does not necessitate any preparatory steps of the patient or compliance with any restrictions in the daily diet or medication. The guaiac-based traditional FOBT (Hemoccult SENSA, Beckman Coulter Inc., Passadena, CA, USA) was used as the reference test in this study. The sample collection protocol of both tests was described in more detail recently (12).
For the CV, two optional reading modes are available: visual analysis (VA) and automatic analysis (AA). The latter is performed by using opTrilyzer Lateral flow reader (Chembio Diagnostics GmbH, Berlin, Germany), as described before (11, 12). In fully compliant patients, three stool samples were tested by CV and the result was interpreted positive if any of the three samples tested positive for either Hb or Hb/Hp complex. The analytical sensitivity for CV Hb is 15 ng/ml, and for CV Hb/Hp complex, 4 ng/ml (17).
Normal colonoscopy was used as the gold standard indicating a negative result regarding the study endpoints, as described before (12).
Statistical analysis. STATA/SE version 17.0 (StataCorp, College Station, TX, USA) was used for analysis. The statistical tests presented were two-sided, and p-values <0.05 was considered statistically significant. Using 2×2 tables, sensitivity (Se) and specificity (Sp) with 95% confidence intervals (95%CI) for each test was determined. Conventional ROC analysis was used to graph for Se and Sp as well as to find the optimal cut-off values for both Hb and Hb/Hp of the CV test. Meta-analytical technique (metaprop) was used to create separate forest plots for Se and Sp, with each set of data included (i.e., test components Hb, Hb/Hp, cut-offs). We also calculated the summary estimates of Se and Sp, positive (LR+) and negative likelihood ratio (LR-) as well as diagnostic odds ratio, using a random effects bivariate model and fitted the summary hierarchical receiving operating characteristic (HSROC) curves for the CRA as the endpoint. Roccomp test was used to compare the statistical significance between the AUC (area under the curve) values of the AA and VA reading modes.
Results
Patient data of the study. The whole screening cohort at Barretos Cancer Hospital (BCH) included 5,090 patients, but due to various reasons, 4,584 of the subjects had to be excluded (Figure 1). There were 506 eligible patients for the study, of whom 127 patients were excluded due to missing stool samples. The final cohort included 379 colonoscopy-referral patients evaluated by the VA and AA reading modes of the CV tests and SENSA.
VA screening tests for colorectal adenoma endpoint. The Se, Sp and efficiency of the SENSA test detecting CRA were as follows: 21.1%, 83.0%, 52.8% (Table I). The Se, Sp and efficiency of the visually analyzed CV Hb and CV Hb/Hp tests detecting CRA were as follows: 46.9%/43.6%, 64.0%/60.6%, 55.7%/52.3% (Table I). The PV+ of CV Hb VA test (Table I) was slightly higher than that of test CV Hb/Hp VA test (Table II); 55.0% versus 51.6%, indicating the proportion of the patients with a positive test result who have the disease. When SENSA + CV Hb + Hb/Hp VA (Table I) were used as a combined test panel for the CRA endpoint, the panel had 52.7% Se, 54.3% Sp and 53.5% efficiency (Table I).
ROC analysis and optimal cut-off values of the CV. The ROC analysis showed the optimal cut-off value of ≥8.0912 for CV Hb AA (Table II) and ≥1.8983 for CV Hb/Hp AA (Table II). Using these cut-offs in the AA mode, the Se, Sp and efficiency of the Hb (Table II) and Hb/Hp (Table II) tests detecting CRA were as follows: 64.2%/78.6%, 53.4%/35.3% and 58.6%/56.5%. The PV+ of CV Hb AA test (Table II) was slightly higher than that of test CV Hb/Hp AA test (Table II); 56.4% versus 53.8%, indicating the proportion of the patients with a positive test result who have the disease. When CV Hb + Hb/Hp AA (Table II) was used as a combined test panel for the CRA endpoint, the panel had 83.5% Se, 30.7% Sp and 56.4% efficiency (Table II). Figure 2 shows the AUC values in ROC analysis of the CV Hb AA and CV Hb/Hp AA tests for the CRA endpoint: 0.605 and 0.590, respectively.
Diagnostic accuracy of the CV test in the VA mode. The overall Se of the CV in the VA mode for detecting CRA was 43% (95%CI=31-55%) (Figure 3). The three most sensitive CV test VA panels (CV Hb VA, CV Hb + Hb/Hp VA, and CV + SENSA panel) showed 47-53% Se (Figure 3). The overall Sp of the VA mode was 64% (95%CI=54-75%) (Figure 4). The three most specific VA test modes showed Sp values of 61-83% (Figure 4).
Diagnostic accuracy of the CV test in the AA mode. The overall Se of the AA reading mode for detecting CRA was 69% (95%CI=57-79%) (Figure 5). The two most sensitive AA tests (CV Hb/Hp AA at cut-off ≥1.8983, and CV Hb + Hb/Hp AA test panel) showed 79-84% Se (Figure 5). The overall Sp of the AA reading mode for the CRA endpoint was 46% (95%CI=35-57%) (Figure 6). The three most specific AA tests (CV Hb AA at cut-off ≥9.8796, CV Hb/Hp AA at cut-off ≥8.1592 and CV Hb + Hb AA at cut-off ≥8.0912) in CRA diagnosis showed an Sp range of 53-57% (Figure 6).
HSROC and AUC values. HSROC curves were used to visualize the pooled overall accuracy of VA (Figure 7) and AA (Figure 8) reading modes in CRA detection. In HSROC analysis, the AUC values for i) VA, and ii) AA modes were as follows: i) AUC= 0.551 (95%CI=0.500-0.602) (Figure 7), and ii) AUC=0.606 (95%CI=0.550-0.662) (Figure 8). The difference between these AUC values (Roccomp analysis) was statistically significant (p=0.0160).
Discussion
FITs have been developed to solve the analytical problems associated with the traditional gFOBTs. Our aim was to test the performance of visually analyzed (VA) and automatically analyzed (AA) new-generation FIT for Hb and Hb/Hp (ColonView quick test) in subjects participating in CRN detection in Brazil. A traditional gFOBT test (SENSA) was used as the reference test.
For reading of the CV test results, two optional modes are available: VA and AA (18). For the automatic reading, the Quick Test Reader (QTR) is needed. QTR is a mobile device for quantitative evaluation of lateral flow assays. The device is initially tested and configured by the original device manufacturer (opTricon GmbH), and subsequently validated and configured by Biohit (Biohit Oyj) for the use in CV AA reading (18). In this calibration, VA and AA are interlinked in that the weakest band visualised as positive (by several observers) was equivalent to the reader AA value of 20 ng/ml (18).
In quantitative tests, the optimal cut-off levels for Hb detection depend on the desired tradeoff between sensitivity and false-positive (FP) rate. This can be neatly done by using conventional non-parametric ROC analysis (19, 20), selecting the coordinate points in the ROC curve as indicators of these cut-off values, separately for Hb and the Hb/Hp complex. The cut-off values have become increasingly important with the introduction of quantitative iFOBTs in which it is possible to adjust the cut-off limit to obtain an acceptable compromise between clinical sensitivity and specificity.
The ideal balance between Se and Sp in FOBT depends on the screening cohort, and FP rate is associated with an increased number of unnecessary colonoscopies (21). Therefore, an optimal FOBT for CRN screening will provide an opportunity for quantitative optimization of the cut-off values. While the choice of FOBT for CRN screening mainly focuses on diagnostic accuracy, other associated factors also require consideration including test costs and compliance. A recent study on longitudinal adherence of the patients with CRN screening in a primary care cohort showed that less than 50 percent of the patients completed gFOBT (SENSA) screening during the 2-year follow-up period (22). Therefore, a significant advantage of the FITs is an improved screening compliance as compared to gFOBTs. In addition, the use of gFOBTs include complicated dietary restrictions, leading to poor test compliance among the test subjects (15, 16). As to the CV test, and particularly its AA reading mode, the test offers a number of other advantages in CRN screening, such as integration into the laboratory information system and the opportunity for quality control (11, 12, 15, 16).
The present study is limited by its design that does not enable us to predict whether enhanced Se of the CV test for detection of CRA will result in a decreased mortality rate in CRC. While our study sample has an effect on the pre-test probability of CRA, our CRA detection rate of eligible patients in CRC screening cohort 185/506 (36.6%) was slightly higher than that of earlier CRA screening cohorts (23-25).
The FIT tests were recently subjected to a meta-analysis by Lee et al. (26) and Meklin et al. (16). Interestingly, in these meta-analyses the authors used only the CRC endpoint while pooling the FIT results. This decision was based on their observation that only a subset of all studies reported adequate data on CRA and that there was a wide variability in the definitions of adenomas (16, 26). Thus, all studies of CRA were excluded from these two meta-analyses. However, the conclusion was that the CV quick test is superior to the traditional gFOBTs in screening (15, 16).
There are three previous screening cohorts available where the CRA endpoint was used (23-25). Allison et al. (23) assessed 139 patients with advanced neoplasms (14 CRC, 128 CRA) and found 29.5% Se for FIT and 22.8% Se for panel of SENSA and FIT. However, they did not use HSROC and AUC analysis (27-31) to confirm the diagnostic performance of the tests in these patients. Hundt et al. (24) collected stool samples before colonoscopy of the patients with CRA and tested the diagnostic performance of six different FITs. They reported a pooled Se of 25% (95% CI=18-34%) for the CRA endpoint. Similar to Allison et al. (23), the authors did not measure the diagnostic accuracy of the FITs using the HSROC analysis (24). In addition, this study deviated from the real-life conditions in that the stool samples were not directly stored in a buffer vial, but frozen before testing instead.
The only previous study where both Hb and Hb/Hp complex are included is the report of Haug et al. (25), who used automated ELISA (RIDASCREEN) to study a cohort of 130 patients with advanced adenoma. Being one of few tests with both Hb and Hb/Hp complex, RIDASCREEN has characteristics that are very similar to those of the CV test, except that the former is a quantitative ELISA test and the latter is a quick test. Not unexpectedly, the AUC values for Hb and Hb/Hp reported by Haug et al. (25): 0.68 (95%CI=0.65-0.71) and 0.64 (95%CI=0.61-0.67), respectively, are only slightly higher than those obtained in the present study with the AA reading mode of the CV (Figure 8).
HSROC analysis has become a convenient approach to evaluate the diagnostic accuracy of various diagnostic tests with different components that can be treated like study IDs in the HSROC analysis (27-31). Figure 7 shows the HSROC curve for the VA reading mode in CRA diagnosis, with a reasonably modest AUC value of 0.551 (95%CI=0.500-0.602). The diagnostic accuracy of the AA reading mode is significantly higher than that of the VA mode, with AUC=0.606 (95%CI=0.550-0.662) (p=0.0160) (Figure 8). This makes the AA reading mode a preferred method of interpreting the results of the CV test, particularly with the CRA endpoint, where the diagnostic accuracy of the CV test is markedly inferior as compared with the CRC endpoint (32).
Conclusion
The CV quick test interpreted using the automatic reading mode (AA) test showed significantly higher diagnostic accuracy for the CRA endpoint than did the VA reading mode, or the obsolete guaiac-based HemoccultSENSA test. As pointed out, caution should be followed while comparing the results of different quantitative FIT tests, because different commercial products have different analytical characteristics, and direct comparisons might be misleading. The limitation of this study is the colonoscopy-referral setting, because these patients represent a population with markedly higher CRA prevalence as compared with a native screening setting of asymptomatic subjects. These results clearly implicate that CRA screening by the CV quick test in its AA mode has the major benefit of avoiding unnecessary endoscopy or radiological procedures while keeping the false positive results at minimum.
Acknowledgements
The study was funded by the Päivikki and Sakari Sohlberg Foundation, Helsinki, Finland.
Footnotes
↵This article is freely accessible online.
* These Authors contributed equally to this study.
Authors’ Contributions
All Authors contributed to the collection and analysis of data, drafting and revising the manuscript, and read and approved the final article.
Conflicts of Interest
Tapani Tiusanen, PhD, is an employee of Biohit Company, Helsinki, Finland. The other Authors report no conflicts of interest or financial ties in relation to this study.
- Received September 24, 2021.
- Revision received October 13, 2021.
- Accepted October 14, 2021.
- Copyright © 2021 International Institute of Anticancer Research (Dr. George J. Delinasios), All rights reserved.