Abstract
Background/Aim: Prostate multiparametric magnetic resonance imaging (mpMRI) is the reference imaging modality for extraprostatic extension of disease (EPE) assessment. We aimed to compare the diagnostic accuracy of different abbreviated MRI protocols to the standard prostate mpMRI in the identification of EPE of PCa. Patients and Methods: Fifty patients were retrospectively enrolled. Dual-pulse (dpMRI) and biparametric (bpMRI) abbreviated protocols were obtained from mpMRI. The performance of two experienced radiologists and two radiology residents was correlated with a reference standard and compared. Inter and intra-reader agreements were evaluated. Results: All protocols were strongly correlated to the reference standard (p≤0.001). A significant difference was found between dpMRI and mpMRI (p=0.009), no differences emerged between bpMRI and mpMRI (p=0.27). All readers showed moderate agreement (ĸ=0.47, ĸ=0.50 and ĸ=0.53 for dpMRI, bpMRI and mpMRI, respectively). Intra-reader agreement was good (all ĸ values ≥0.70). Conclusion: Only bpMRI showed similar diagnostic performance to mpMRI, thus appearing as a feasible alternative to the standard protocol for EPE detection.
Local staging is a crucial phase in the management of patients affected by prostate cancer (PCa). Indeed, an accurate assessment of extraprostatic extension (EPE) of the disease could modify the decision of an appropriate treatment strategy and therapeutic planning (1).
In this context, magnetic resonance imaging (MRI) is regarded as one of the best imaging modalities to achieve a correct and reliable pre-treatment local staging of PCa (2-4). In particular, multiparametric MRI (mpMRI), which is composed of three planes, T2-weighted (T2-w), axial diffusion weighted imaging (DWI) and axial dynamic contrast enhanced (DCE) sequences, is to date acknowledged as the standardized and recommended imaging protocol for the detection and local staging of PCa (5). Nevertheless, alternative short imaging protocols have been recently proposed as valuable alternatives to the standard mpMRI (6-9). Aims of these abbreviated protocols were to reduce acquisition time, costs and patients' discomfort, mainly diverging from mpMRI due to the removal of the DCE sequence. This protocol, referred as biparametric MRI (bpMRI), has been proven to have a similar diagnostic accuracy in the detection of PCa (6, 7, 9, 10). Along with bpMRI, even shorter protocols have been suggested, occasionally referred as dual-pulse MRI (dpMRI), which rely on the sole evaluation of axial T2-w and DWI sequences, further removing the assessment of coronal and sagittal T2-w images (11-13).
While both dpMRI and bpMRI have demonstrated, compared to mpMRI, a similar good diagnostic accuracy in the detection of PCa, no evidence is available regarding their possible different performances in local staging. Therefore, the aim of this study was to evaluate the diagnostic performance of different abbreviated MRI protocols compared to the standard prostate mpMRI in the identification of EPE of PCa, and also assess the “real world” reproducibility of these different MRI protocols evaluating the diagnostic performances obtained by four independent readers with different levels of expertise.
Patients and Methods
This observational, retrospective study was approved by the Institutional Review Board (Protocol number: 60/19) and the need for informed consent was waived.
Patient population. We selected 41 consecutive patients diagnosed with PCa by means of biopsy who underwent a standard mpMRI at our Institution to assess local stage prior to radical prostatectomy (RP) between January 2016 and October 2017. To avoid reader bias, we also included a second category of consecutive patients (n=15) with clinical suspicion of PCa and a negative biopsy result, who underwent a mpMRI exam to guide a re-biopsy that confirmed the absence of PCa. Six subjects were excluded due to incomplete or inadequate (e.g. presence of artefacts) mpMRI, or because RP was performed more than three months after mpMRI, leading a final number of 50 patients. The following demographic and clinical data were recorded: patients' age, Prostate Specific Antigen (PSA) levels, pathological data including Gleason Score and local stage according to the TNM classification of malignant tumors (Table I).
MRI data acquisition. A complete list of the different MRI acquisition parameters and protocols is shown in Table II. All examinations were performed on a 3 Tesla scanner without endorectal coil (Magnetom Trio, Siemens Healthcare, Erlangen, Germany).
Briefly, from the standard mpMRI images acquired according to the PI-RADS v2 recommendations (5), two different unenhanced MRI protocols were extracted, dpMRI composed of axial T2-w and a DWI (with the corresponding ADC maps) sequences, and bpMRI protocol included three-planes T2-w sequences along with an axial DWI. The different structure of MRI protocols is highlighted in Figure 1.
Image analysis. Images were independently reviewed by two experienced radiologists (ERs) highly trained in MR prostatic imaging (both with more than 5 years of experience) and two radiology residents (RRs) (both with less than 3 years of experience in prostate imaging).
Readers were asked to independently review images from each protocol of every patient included in the study, blinded to clinical information, and to assess first if an index lesion was present (i.e. PI-RADSv2 score ≥3) and, in case of positive results, if it was suggestive for prostate confined disease or EPE. The second assessment was also performed, in accordance to the PI-RADSv2 indications, evaluating the presence of both direct and indirect signs (e.g. length of capsule contact, bulging) to the make the final decision (5). For all evaluations of each reader, reviewing time expressed in seconds (sec) was recorded.
Initially, the four readers evaluated the dpMRI protocol, while bpMRI and mpMRI protocols were reviewed after a 5-weeks washout, after completing each evaluation. To further reduce readers' bias, for each protocol multiple reading sessions were organized, and the images administered to the reader were randomly ordered and grouped.
Furthermore, to evaluate intra-reader agreement, one ER and one RR re-evaluated all the images from each protocol after an additional 5-week period.
Statistical analysis. Image evaluation times for each protocol, averaged among experience groups, were compared using a two-sample t-test.
Patient population demographic and clinical information.
Fleiss' Kappa (ĸ) was calculated to evaluate agreement between readers in the assessment of each different protocol, while Cohen's ĸ was used to evaluate both the inter-reader agreement between ERs and RRs for each protocol, as well as the intra-reader agreement in the two-step evaluation performed for every MRI protocol. All agreements were interpreted, on the base of their ĸ values, using the following classification: ≤0.20: poor agreement; 0.20-0.40: fair agreement; 0.40-0.60: moderate agreement; 0.60-0.80: good agreement; ≥0.80: excellent agreement (14).
A Spearman analysis was performed to evaluate the correlation of each reader, when reviewing the different protocols, to the reference standard. Correlation strength was classified as low, moderate, high and excellent using the following correlation coefficient cut-off values respectively: ≤0.35, between 0.36 and 0.67, between 0.68 and 0.90, and ≥0.91. Data were tested to verify normal distribution using the Kolmogorov-Smirnov test. Subsequently, correlation coefficients of dpMRI and bpMRI were compared to the ones obtained with the mpMRI protocol using a paired t-test.
Measures of diagnostic performances (i.e. sensitivity, specificity, positive and negative predictive values and diagnostic accuracy) in the identification of EPE were calculated for each different reader and every MRI protocol, and these measures were then averaged across experience group readers.
Results
For all readers, mean average time for reviewing dpMRI, bpMRI and mpMRI was respectively 222, 262 and 290 sec; the time for reviewing dpMRI was significantly lower compared to mpMRI (p≤0.001), while no difference was found between bpMRI and mpMRI (p=0.06). When evaluating differences between ERs and RRs in the time needed for the evaluation of the three protocols, significant differences emerged for dpMRI (163 vs. 281 sec; p≤0.001) and bpMRI (232 vs. 293 sec; p=0.001), but not for mpMRI (270 vs. 310 sec; p=0.06). Interestingly, ERs showed a greater increase in reading time when comparing both dpMRI to bpMRI and bpMRI to mpMRI (69 and 38 sec) compared to RRs (12 and 17 sec).
MR images of a 74-year-old patient with a lesion suspicious for prostate cancer located in the right peripheral zone. From left to right, axial ADC map, axial high b-value DWI, axial T2-weighted, coronal T2-weighted, sagittal T2-weighted and axial DCE sequences are shown. The structure of the three different protocols is highlighted by braces. Interestingly, in the coronal T2-weighted sequence (4th image from left) the real maximum diameter of the suspicion lesion is revealed. Radical prostatectomy revealed a 4+4 Gleason score prostate cancer, with absence of extraprostatic extension of disease.
Technical parameters of each MRI sequence, with + and - signs marking whether the sequence is included or excluded in the protocol, respectively.
The four readers showed a moderate agreement among each protocol (ĸ=0.47, ĸ=0.50 and ĸ=0.53 for dpMRI, bpMRI and mpMRI, respectively). When agreement within groups was tested considering the experience of the readers, an increase in the agreement was found for the ERs (ĸ=0.48, ĸ=0.65 and ĸ=0.73 for dpMRI, bpMRI and mpMRI, respectively), while RRs showed a more stable, although lower agreement (ĸ=0.42, ĸ=0.42 and ĸ=0.46 for dpMRI, bpMRI and mpMRI, respectively). Finally, the intra-reader agreement analysis showed high values of concordance (all ĸ values ≥0.70). In particular, the ER obtained ĸ values of 0.70, 0.78 and 0.73 while the RR scored ĸ values of 0.79, 0.94 and 0.72 for dpMRI, bpMRI and mpMRI, respectively.
Results of the correlation analysis showed that all protocols were strongly correlated to the reference standard (Table III), with a trend towards an increase in correlation from dpMRI to mpMRI. When differences between protocols were tested, a significant difference emerged between dpMRI and mpMRI (p=0.009), while no differences emerged for the dpMRI versus bpMRI and bpMRI versus mpMRI comparisons (p=0.11 and p=0.27, respectively).
Sensitivity, specificity, positive and negative predictive value, along with diagnostic accuracy among groups of experience for each protocol are presented in Table IV. Briefly, all the protocols reached a similar high performance in terms of diagnostic accuracy, with consistently expected lower performances for the RRs compared to the ERs. Among the three protocols, dpMRI proved to have slightly lower diagnostic accuracy (ERs=83.0%; RRs=75.0%), while similar values were found for bpMRI (ERs=86.0%; RRs=75.0%) and mpMRI (ERs=87.0%; RRs=77.0%), with a subtle trend of increase when readers evaluated more complex imaging protocols (ranging from 83% to 87% for ERs and from 75% to 77% for RRs).
Results of correlation analyses between different MRI protocols and reference standard. All results are with p<0.001.
Discussion
In this study, we investigated the performance of readers with different expertise in the evaluation of three prostate MRI protocols for the identification of EPE, in patients with PCa. Our results indicate that bpMRI is comparable to mpMRI. On the other hand, dpMRI proved to have lower reproducibility and worse correlation with histopathology, suggesting that standard mpMRI could be shortened by eliminating DCE sequences, while the remaining sequences should be acquired.
The mpMRI protocol, including multiplanar T2-w, DWI and DCE sequences, is considered as the current standard for prostate MR imaging (5). Nevertheless, its composition has been debated in recent years, with particular reference to the use of DCE sequences for the detection of PCa (15-17). Indeed, it has been demonstrated that removing it from prostate MRI would significantly improve cost-effectiveness without a significant loss in terms of diagnostic accuracy (18). Moreover, this approach would also reduce the possible risks associated with multiple gadolinium contrast agent administrations (19, 20). In this context, the search of alternative unenhanced, shorter imaging protocols has been the area of interest of different research groups, although being mostly focused on PCa detection (6-9, 11, 21, 22).
In addition, of those that attempted to verify the efficiency of shortened prostate MRI protocols in the evaluation of EPE, to the best of our knowledge, none has made a direct comparison with mpMRI (23-25). Nevertheless, these studies suggested a possible role of bpMRI in this field, as confirmed in this work.
Indexes of diagnostic performances for readers averaged by experience among the three different MRI protocols.
The main result of our study is the absence of significant difference in terms of correlation with the reference standard, between mpMRI and bpMRI, as opposed to dpMRI, that showed a significantly lower correlation when compared to mpMRI. This suggests that the identification of EPE might be equally performed either with bpMRI or mpMRI, while dpMRI does not seem appropriate for this task. Therefore, we hypothesize that while DCE removal can be mitigated by the availability of two other axial sequences, providing both functional and anatomical information, the availability of sagittal and coronal T2-w images significantly impacts proper evaluation of EPE in PCa patients. Interestingly, when a direct comparison between dpMRI and bpMRI was performed, no statistically significant differences in terms of correlation with the reference standard were obtained, although we wish to highlight that mean performances were higher for bpMRI compared to dpMRI.
When investigating the role of experience in EPE detection, we found that all correlation coefficients in our study proved to be relatively high (ranging from r=0.494 to r=0.883). Nevertheless, correlation strength was moderate for the RR and high for the ER group. This result, somewhat expected, is in agreement with previous studies suggesting that experience level has an important role in PCa EPE assessment (25, 26), a complex task more suitable for readers with adequate expertise in the field. In contrast to these results, recently Jansen et al. reported the absence of a learning curve for EPE identification on mpMRI (27) probably due to the background of readers involved in the study. Indeed, as stated by the authors, only the performance over time of radiologists with an already strong expertise in prostatic imaging was evaluated.
Furthermore, the overall inter-reader agreement for all protocols in our study was moderate, again in line with previous studies (24, 28). On the other hand, when assessing reproducibility across protocols based on experience level, inter-reader agreement was good for bpMRI and mpMRI and moderate for dpMRI in the ER group, while a moderate agreement for all protocols, was obtained by the RR group. Diagnostic accuracy was also higher in the ER group compared to RRs across all evaluated protocols, as shown in Table IV. These results further corroborate our hypothesis about the incremental value of expertise in prostatic imaging. Interestingly, the diagnostic performance of mpMRI in our study differs from those found in a recent meta-analysis (29). This could be at least partially explained by the presence of PCa-free subjects in our patient population that might have contributed increasing sensitivity.
It is interesting to note that ERs showed a greater increase in reading time when comparing both dpMRI to bpMRI and bpMRI to mpMRI (69 and 38 sec) compared to RRs (12 and 17 sec). We believe that this could be related to an over-reliance of RRs on axial images, albeit the presence of longer dpMRI reading times. This is also reflected in their similar performance across all prostate imaging protocols. The performance gap that is due to experience level may be filled by the development of computer aided diagnosis software dedicated to EPE detection. Indeed, in the recent literature, radiomics and machine learning have shown promising results in this field and their added value for lesser-experienced readers should be investigated in future studies (30-32).
This study suffers from several limitations that should be acknowledged. The number of subjects included is relatively low for a multi-reader study and a greater sample size would have allowed to perform additional evaluations such as differentiating seminal vesicle, neurovascular-bundle and adjacent organs invasion; furthermore, it was not possible to separate patients in subgroups according to risk stratification (1). Conversely, a low number of patients can be partly justified by the mandatory need for radical prostatectomy as our reference standard. Although several strategies have been embraced to reduce reader bias, it is not possible to completely exclude its presence due to the relatively low number of cases and repeated reading sessions.
In conclusion, evidence collected in this study suggests that bpMRI might be adopted as a valid alternative to mpMRI for the identification of EPE; on the other hand, dpMRI does not appear as accurate for this purpose. Our results also indicate that, regardless of MRI protocol characteristics, radiologic experience significantly influences the performance of readers.
Footnotes
↵* Both authors contributed equally to this article.
Authors' Contributions
Guarantor of integrity of the entire study: Massimo Imbriaco; Study concepts and design: Arnaldo Stanzione; Andrea Ponsiglione; Literature research: Arnaldo Stanzione; Andrea Ponsiglione; Renato Cuocolo; Clinical studies: Stefano Giusto Picchi; Salvatore Stilo; Francesco Persico; Massimiliano Creta; Nicola Longo; Data analysis: Arnaldo Stanzione; Andrea Ponsiglione; Renato Cuocolo; Sirio Cocozza; Statistical analysis: Sirio Cocozza; Manuscript preparation: Arnaldo Stanzione; Andrea Ponsiglione; Renato Cuocolo; Sirio Cocozza; Massimo Imbriaco; Manuscript editing: Nicola Longo; Massimo Imbriaco.
Conflicts of Interest
The Authors have no conflicts of interest to declare regarding this study.
- Received June 27, 2019.
- Revision received July 1, 2019.
- Accepted July 2, 2019.
- Copyright© 2019, International Institute of Anticancer Research (Dr. George J. Delinasios), All rights reserved
References
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.