Abstract
Background/Aim: Several algorithms have been developed to assess the risk of predicting BRCA mutation and breast cancer (BC) risk. The aim of this study was to evaluate the accuracy of these prediction algorithms in the Israeli population. Patients and Methods: Risk for developing breast cancer and the probability for carrying BRCA1/2 mutations using BOADICEA, BRCAPRO, IBIS, MYRIAD and PENN2 models were computed for individuals counseled and genotyped at the Oncogenetics unit in 2000 and 2005. The predicted mutation carriers and BC risks were compared with actual carrier rates by genotyping and BC diagnoses derived from the Israeli National Cancer Registry database. Results: Overall, 65/648 (10%) study participants were BRCA1/2 mutation carriers. Of 373 cancer-free participants at counseling, 25 had breast cancer by 2016. BOADICEA and BRCAPRO performed best for predicting BRCA mutation (AUC=0.741, 0.738, respectively). No model was clinically useful in predicting breast cancer risk. Conclusion: BOADICEA and BRCAPRO outperformed the other tested algorithms in BRCA mutation prediction in Israeli women, but none was valuable in breast cancer risk prediction.
Germline mutations in either the BRCA1 (MIM#113705) or the BRCA2 (MIM#600185) genes confer a substantially increased risk for developing breast and ovarian cancer, that are up to X6 and X25 that of the general, average risk population, respectively (1). Thus, identifying asymptomatic BRCA1/2 mutation carriers is of paramount importance as it enables tailoring an early surveillance scheme (for breast cancer) from an early age (25-30 years) and offer mutation carriers the possibility of risk reducing surgeries (2). The eligibility for insurance covered genetic testing for being a BRCA1/2 mutation carrier varies across populations, but in general is recommended for anyone with a predicted carrier risk of 10% or higher (3). These recommendations apply primarily to outbred, genetically heterogeneous populations, where the spectrum of germline mutations in both genes encompasses more than 3,000, mostly family specific mutations (4). In some populations, the range of mutations in both BRCA genes is limited, as a result of a founder effect. In Ashkenazi (East European) Jews, three mutations in BRCA1 (185delAG (c.68_69delAG; p.Glu23Valfs; rs386833395) 5382InsC (c.5326_5327insC; p.Gln1777Profs; rs80357906)) and BRCA2 (6174delT(c.5946delT; p.Ser1982Argfs)) account for most of the mutations detected in high risk Ashkenazi families (5, 6). In addition, the same mutations can be detected in 35%, 12%, and 2.5% of ovarian, breast cancer cases and general population of the same ethnicity, respectively (7). The range of mutations in both BRCA genes in the non-Ashkenazi Jewish population is also somewhat limited. The 185delAG*BRCA1 mutation was reported in Iraqi and Balkan Jews (8), the 8765delAG*BRCA2 mutation was reported in Yemenite Jews (9), and the p.Y978X*BRCA1 mutation was found in Iraqi, Afghan and Iranian Jews (10). Currently, most oncogenetics services in Israel are genotyping for a set of 14 recurring mutations in both BRCA genes (11). For high risk women who do not harbor any of these recurring mutations, the Health basket in Israel allows for full sequencing of both genes if the residual risk for finding a BRCA mutation is 10% or higher (12).
To assess the likelihood of carrying a BRCA mutation, several algorithms have been developed and used in the clinical setting: BOADICEA (13), BRCAPRO (14), IBIS (15), MYRIAD (16), PENN2 (17). Notably, these models differ in the risk factors evaluated and the weight each factor is given when assigning BC risk or mutation carrier estimates. The outcomes of these models are the likelihood of finding a BRCA mutation and, in some models, also lifetime and the 5 and/or 10-year risk of developing breast and ovarian cancer. These algorithms are applied during onco-genetic counseling in several countries in Europe and in the United States and have been validated in several ethnically diverse populations (18). To the best of our knowledge, the accuracy and predictive value of these algorithms has not been comprehensively evaluated in Israel, except for one study that tested two models for breast cancer risk only (19). Thus, the aim of this study was to expand the number of tested risk prediction models and offer the optimal model to the Israeli population.
Patients and Methods
Study population. All individuals (males and females) who underwent oncogenetic counseling at the Oncogenetics Unit at the Sheba medical center, Tel-Hashomer throughout calendric years 2000 and 2005, were eligible for participation, if they were genotyped for the predominant BRCA mutations in the Jewish population. The oncogenetic counseling service at the Sheba medical center draws its counselees from high risk families, consecutive BC cases and ovarian cancer cases, as well as the general population. At the time of the study conduction, less than 5% of counseled individuals were drawn from the average risk population. For validation of the accuracy of the models to predict breast cancer the inclusion criteria were women who were breast cancer free at the time of initial counseling. The study was approved by the ethics committee of the Sheba Medical center and was carried in accordance with the approved protocol.
Prediction of carrying a BRCA1 or BRCA2 mutation. The risk for harboring a mutation in either the BRCA1 or the BRCA2 genes was calculated for all eligible participants, regardless of gender. To validate the accuracy of the models in predicting BRCA mutation in the BOADICEA, BRCAPRO, IBIS, MYRIAD, PENN2 algorithms, we used two separate thresholds; one at 10% and another set at 15%. These predictions were compared with the results of genotyping for the predominant BRCA mutations, using a genotyping strategy previously described (20).
Prediction of breast cancer. The five and (whenever possible) ten year and lifetime risks for developing breast cancer were calculated for all eligible female participants who were cancer free at the time of initial counseling. To validate the accuracy of the models in predicting breast cancer diagnoses by using the BOADICEA, BRCAPRO, and IBIS models, two thresholds were tested-one set at 15% and another at 20%. These outcomes were compared with the observed rates of breast cancer diagnoses derived from the Israeli national cancer registry (INCR) (https://www.health.gov.il/UnitsOffice/HD/ICDC/ICR/Pages/default.aspx) by cross-referencing the ID numbers of all participants with the list of breast cancer diagnoses reported to the INCR, last updated for the calendric year 2016 in 2017.
Statistical methods. The sensitivity and specificity of each of the assessed models was calculated for carrying a BRCA mutation and for breast cancer prediction (when applicable) separately. Furthermore, a ROC (receiver operating characteristic) analysis was performed to assess the goodness of fit for each model without additional factors. In a ROC curve the Sensitivity is plotted in function of the false positive rate (Specificity). In addition, logistic regression analysis was performed to assess the predictive value for each of the models by risk factors.
Results
Prediction of BRCA1 and BRCA2 mutations
Study population. Overall, 648 individuals participated in this study, with a majority of women participants: 282 in 2000 (13 men), and 366 in 2005 (18 men). Of participants, 398 (61.8%) were of Ashkenazi Jewish (AJ) origin; 176 (27.3%) were of Non-Ashkenazi Jewish origin (Non-AJ); 51 (7.9%) were of Mixed AJ-Non-AJ origin; 16 (2.5%) were non-Jewish origin and 3 (0.5%) were of Mixed Jewish/Non-Jewish origin. Age range at counseling was 19-85 years (mean age 50.9±11.4 years). A total of 272 women were diagnosed with breast cancer, age range=23-81 years (mean 48.3±10.7) prior to counseling and genotyping.
Prevalence of BRCA1 and BRCA2 mutations in the study cohort. Of the entire cohort, 65 of 648 mutation carriers were identified (10.03%): 42 BRCA1 and 23 BRCA2. In 2000, 18 mutation carriers were identified (12 BRCA1, 6 BRCA2) and in 2005, 47 mutation carriers were identified (30 BRCA1, 17 BRCA2) (Table I).
Sensitivity and specificity for predicting BRCA mutation. For all counseled individuals, regardless of ethnic origin, gender, or health status, assessing the sensitivity rates at the 10% threshold, the BOADICEA and BRCAPRO models outperformed PENN2 and Myriad for predicting BRCA mutation.
Among counseled individuals who were BC free at the time of consultation (n=373), assessing the sensitivity rates at the 10% threshold, using the BRCAPRO model outperformed all other models tested (Table II).
ROC analysis. Among all counseled individuals, the BOADICEA and BRCAPRO algorithms outperformed Myriad and Penn2 for predicting a mutation in BRCA1/2 genes. The differences between these two algorithms were small (AUC=0.741, AUC=0.738, respectively). BOADICEA was the best predictor for women who were cancer free at the time of consultation (AUC=0.714), whereas Penn2 was best suited for individuals diagnosed with breast cancer until genetic testing was performed (AUC=0.778). For individuals of AJ origin (men and women), BOADICEA and BRCAPRO outperformed the other models (AUC=0.746, AUC=0.738, respectively) and for non-AJ, the best model was Myriad (AUC=0.727). For cancer-free AJ individuals, IBIS outperformed the other models (AUC=0.792).
Relevant characteristics of study participant by BRCA1/2 carrier status.
Logistic regression analysis. Following logistic regression analysis, an additional ROC analysis was carried out to assess the goodness of fit for each model with the risk factors. All models improved their performances and Penn2 improved its accuracy the most, for all individuals (0.691 to 0.807) and for those who were initially breast cancer free (0.579 to 0.778). For individuals diagnosed with breast cancer at time of consultation the Myriad algorithm showed the most improved performance (0.722 to 0.823). The main risk factors that were not sufficiently accounted for or were totally disregarded in these algorithms were AJ ethnic origin (except for the myriad model), ovarian cancer prior to consultation, and number of pancreatic cancer cases in the family. These risk factors were found to have a significant impact on calculating the risk of carrying a BRCA mutation.
Accuracy of breast cancer risk prediction
Study population.Of 648 participants in this study, 617 (95.2%) were women of whom 345 (55.9%) were cancer free at the time of consultation.
Breast cancer risk. After cross referencing the data of these 374 cancer free women with the database of the INCR, updated in August 2017, 25/374 were diagnosed with breast cancer during 12 to 17 years follow-up, of whom 4 were BRCA1 mutation carriers: 8 were diagnosed up to 5 years after consultation; another 12 (a total of 20) were diagnosed up to 10 years after consultation and the other five were diagnosed more than 10 years after consultation.
Sensitivity and specificity. None of the algorithms predicted that any of the women diagnosed with breast cancer would be diagnosed after 5 or 10 years (sensitivity 0%). Lifetime sensitivity rates at 20% threshold using the algorithms ranged from 12% to 44% (Table III).
ROC analysis. No model was proven accurate in predicting breast cancer at 5 or 10 years after counseling (data not shown).
Discussion
In this study, for all counseled individuals and specifically for AJ the BOADICEA and BRCAPRO (AUC=0.741, AUC=0.738 respectively) algorithms outperformed Myriad IBIS and PENN2 in BRCA mutation prediction. The differences in prediction accuracy between these two algorithms were small and clinically insignificant. BOADICEA was the best BRCA carrier predictor for women who were cancer free at the time of counseling (AUC=0.714), whereas PENN2 was best suited for individuals already diagnosed with BC by the time they were initially counselled (AUC=0.778). These models have previously been validated in ethnically diverse populations, primarily Caucasian European and North American, and all were reportedly accurate in predicting the presence of BRCA1/2 mutation with both BRCAPRO and BOADICEA branded as most accurate for the German (AUC=0.80, 0.79, respectively) and British (AUC=0.76, 0.77 respectively) populations (18, 21). In a Brazilian study, BOADICEA outperformed BRCAPRO and Myriad in predicting BRCA1/2 mutation (AUC=0.87, 0.77, and 0.73 respectively) (22). A study conducted in the USA assessed the accuracy of the BOADICEA, BRCAPRO and Myriad models for predicting the risk of carrying a BRCA1/2 mutation in Ashkenazi Jews, reported that both BOADICEA and Myriad were equally accurate (AUC=0.788, 0.750, respectively) (23). A review that compared the accuracy of the models, including the 5 models tested in the current study in predicting the risk of carrying a BRCA mutation, concluded that BRCAPRO and IBIS outperformed the other models for clinical use in high risk populations in North America and Europe (24).
Algorithms' performance by threshold for all counselees and counselees not diagnosed with BC at initial consultation.
Algorithms' performance by threshold for 5-year, 10-year, and lifetime BC risk.
The current study is in line with these findings as both the BOADICEA and BRCAPRO algorithms were found to have a high prediction value in Israel as in other tested populations (18, 21-27). In the current study the Myriad algorithm was the least accurate in predicting the risk of carrying a BRCA1/2 mutation in Ashkenazi Jews, unlike the results reported for the same algorithm in the USA (23), whereas it outperformed the other models for the non-Ashkenazi Jewish population. One plausible reason for these inconsistent results between the Israeli and the American study is that the composition of the non-Ashkenazim in Israel were mostly Jews and in the USA these were non-Jews. Penn2, reportedly accurate in other studies, primarily in the North American population (26), was found less accurate in this study than the other tested algorithms, except for women diagnosed with breast cancer by the time they were genotyped, a fact that was not reported in previous studies (18, 21-27).
For assessing breast cancer risk, none of the models was accurate in predicting 5 or 10 year breast cancer risk with a 20% threshold, which is the accepted threshold (28). Even after lowering the threshold to 15%, the accuracy rates were very low with AUC values of around 0.5 for all the tested models. For lifetime risks, with a 20% threshold, none of the models' sensitivity rates were clinically satisfactory with BRCAPRO displaying an extremely low sensitivity rate of 12%. While in the current study none of the models displayed “clinical grade accuracy” in predicting breast cancer risk, studies from Europe and North America found the same three tested models (BRCAPRO, BOADICEA, IBIS) to be accurate for predicting breast cancer risk (23). Likely explanations for this discrepancy include a small number of analyzed individuals herein, the relatively short follow up of these women, and perhaps differences in risk factors in Jewish compared with non-Jewish women, factors that are differently weighted in these algorithms.
Some insights that emerged from this study may need to be implemented in the next version of these algorithms. Ovarian cancer diagnosis prior to genotyping, number of pancreatic cancer cases, and Ashkenazi Jewish ancestry all need to be better weighted by these models to improve BRCA1/2 mutation prediction. Several studies tested the accuracy of some of the models for women with ovarian cancer. A study in Brazil found that BOADICEA outperformed BRCAPRO and Myriad for these women (AUC=0.87, 0.77 and 0.73 respectively) (22). Another study from USA found BRCAPRO not to be accurate enough (AUC=0.81) and concluded that “Patients with ovarian cancer classified as low risk by BRCAPRO are more likely to test positive than predicted” (27). A possible explanation for the results regarding pancreatic cancer is that one of the most common mutations in BRCA2 found in pancreatic cancer patients is the 6174delT mutation which is one of the three main AJ mutations. Since Ashkenazi Jews have been one of the most frequently studied ethnic groups regarding the implication and association of BRCA2 mutations with familial pancreatic cancer, it seems plausible that this over-representation may have skewed the results as to the effect of the presence pancreatic cancer in a family as a predictor of carrying a BRCA2 mutation (29, 30).
The limitations of this study should be pointed out and acknowledged. The study population is limited in numbers and solely derived from referral to a single Oncogenetics Unit in central Israel. Hence, the makeup of the study participants do not necessarily reflect the true spectrum of the diverse ethnic makeup of the Israeli population. Another limitation is the different referral patterns to counseling so this cannot be viewed as a “pure” high risk population assessment. There was no distinction in data analysis between high risk and moderate risk population. Lastly the short follow up makes any conclusions regarding the accuracy of these models in terms of BC risk tentative at best.
In conclusion, for the Israeli population BRCAPRO and BOADICEA provide an accurate assessment tool for predicting the risk of carrying a BRCA mutation, but none of the algorithms has an acceptable predictive value for breast cancer risk. This latter hurdle may benefit from a different weighted analysis in any or all algorithms.
Acknowledgements
This work was partially funded by a Grant from the Maccabi HMO to Eitan Friedman; This work was carried out in partial fulfilment of the duties for a Master's Degree by Efrat Schwartz Kenan at the Department of Public Health' Ben-Gurion University of the Negev, Beer Sheba.
Footnotes
Conflicts of Interest
All Authors declare that they have no conflict of interest with the data presented herein.
- Received June 8, 2018.
- Revision received June 22, 2018.
- Accepted June 25, 2018.
- Copyright© 2018, International Institute of Anticancer Research (Dr. George J. Delinasios), All rights reserved
References
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.