Abstract
Background: This was a validation study of a regional register of oral cancer in Örebro, Sweden. The purpose was to assess the rate of errors in baseline, and treatment, and the completeness and accuracy of data on recurrences. Materials and Methods: A total of 653 cases with squamous cell cancer in the oral cavity were identified from the register. A randomized sample of 73 (11%) was selected, and a set of relevant data was compared to medical records. Results: Data on patient and tumour characteristics showed high accuracy, with 98% correct data and more than 99% of treatment data were correct. Follow-up data had a higher rate of errors, with 23% of recurrences not recorded, 13.6% misclassified, and 9.1% of cases showing errors in timing of the recurrence. Conclusion: data concerning patients, tumour status, and treatment in the Regional Head and Neck Register in Örebro are highly accurate. However, the follow-up data contain a higher rate of errors, that must be taken into consideration when evaluating outcome after treatment.
There is a strong tradition of keeping healthcare registers in Sweden. Such registers provide good opportunities for research and quality control, not least when it comes to cancer treatment. However, registry-based research always carries a risk of introducing bias because of unknown errors in the registry. It is therefore important to validate the registry data in order to increase the knowledge on its accuracy (1-4).
This article reports on a validation study of the regional Register for Head and Neck Cancer at Örebro University Hospital. This single-institution register was initiated in 1988 and has a rigorous clinical follow-up scheme that reflects the daily clinical reality of working clinicians. Baseline data and results from follow-up are rigorously registered. Since this register is based on a population from a well-defined geographical area that has undergone diagnostic work-up and treatment according to well-established guidelines and with long-term follow-up of outcome, this register is of importance to the scientific community.
This study focused on oral cancer, which constitutes about 430 cases per year in Sweden (0.7% of all cancers), although the incidence is rising (5, 6). As this is an uncommon disease and the treatment is complex and multi-disciplinary, it is often centralized to specialized institutions. The University Hospital of Örebro has a long tradition of diagnostic work-up and treatment of all types of head and neck cancer. The hospital's primary catchment area, together with referrals from the neighbouring counties makes the total population of the uptake area to about 1.7 million. During the time covered by this study, the most common treatment of oral squamous cell cancer (SCC) at our Centre was surgical excision of the primary tumour, and neck dissection in cases that were clinically node positive. In all but the very low-risk cases, this was followed by adjuvant radiotherapy of the primary tumour site as well as the neck.
Materials and Methods
The register of head and neck cancer. By 30th June 2016, the register covered 3,077 cases of head and neck cancer, including 1,069 cases of oral cancer of which 995 cases were SCC. From its start to the present day, the registration has been performed by one oncologist and two Ear, Nose, and Throat specialists. Data on tumour and patient characteristics are recorded in connection with the multi-disciplinary Tumour Board, which decides the suggested treatment for every patient. The treatment is determined according to written guidelines distributed in the surrounding counties, to ensure evidence-based and equivalent cancer care. Diagnostic workup is performed both in the patient's home county and in Örebro, but the multidisciplinary conference is always held in Örebro. All surgery is performed in Örebro, but some of the referring counties are responsible for radiotherapy, and most of the follow-up takes place in the hospital nearest the patient's home. The home-county clinics will then send data about radiotherapy and follow-up to the register in Örebro. To prevent errors and missing data in the register, the keeper of the register actively requests follow-up data from the hospital where follow-up takes place. Patient follow-up is recorded for 5 years; after this, missing follow-up data will not be actively sought. This register has previously been used as the basis for several meeting abstracts and published peer-reviewed articles (7-9).
The purpose of this article was to establish the validity of this register for use in research on oral SCC. This validation focuses on cases with a diagnosis of oral SCC between 1 January 2000 and 30 June 2016.
The study was approved by the Ethics Committee in Uppsala, Sweden (ref: 2016/539; date: 18 January 2017).
Validation. A random sample of cases was extracted from the register, and the registry data for these cases were compared with those of patient records. Relevant medical records were collected locally in Örebro in digital or paper form. Source data not available locally in Örebro were retrieved from the hospital were the patient was treated/followed-up.
Variables. The variables selected for validation are presented in Table I. Three of the variables were in the form of dates: the date of registration, the date of surgery, and the date of recurrence. Date of registration has been defined in slightly different ways during the time the register has been used, but it generally refers to the date of the diagnostic biopsy. The date of the recurrence was also often difficult to pinpoint exactly. We therefore allowed both of these dates to vary within one month either side without being classified as incorrect. In order to gain a better understanding of the variation within this allowance, the mean difference between the recorded date and the most accurate date we were able to establish from patient records was calculated. For the date of surgery, the error margin was instead set to 1 week. We believe that the variation that we have allowed regarding dates would have only a minor effect on results in our planned future use of the register.
Statistical methods. The sample size for this validation was calculated under the assumption that 95% of the registry entries were correct and with the aim of validating this with 95% certainty. The sample was also stratified for the six different referring counties, as there might have been differences in the quality of the follow-up registration between the different counties. The total sample for validation was calculated as 73, or 11% of the complete set of 653 cases.
Results
Patient and tumour characteristics. The pre-specified set of variables for validation included six different variables dealing with patient and tumour characteristics. As there were 73 cases in the sample, 432 entries were checked; one entry was missing and eight were inaccurate, meaning that 98% of the entries were correct and 66 out of the 73 cases (90%) had no inaccurate entry concerning patient and tumour characteristics. The inaccurate entries are presented in Table II. Descriptive data on variation of registration date compared to date of biopsy or earliest established date with confirmed cancer, as well as on variation of registered recurrence date compared to best established recurrence date from patient records, are presented in Table III.
Treatment data. According to the register, 61 out of the 73 cases in the sample uw surgery as part of their primary treatment. Data on surgery in the register were correct in all cases. The register also recorded 49 cases as having external radiotherapy and 11 as having brachytherapy. One case with postoperative external radiotherapy at the radiotherapy clinic in the patient's home county had not been recorded in the register. The registry had >99% accuracy for surgery and radiotherapy together.
Follow-up data. The greatest difficulty in maintaining accurate data in a register is often concerning events during follow-up after primary treatment. As expected, this was also the case in the present register. During the 5 years of follow-up, 17 recurrences were recorded in the register. In the validation process, a total of 10 errors were found. Five recurrences had taken place that were not recorded at all, which means that the true number of recurrences was 22 and hence that 23% (5/22) of recurrences were missing from the register. The five missing cases corresponded to 6.8% of the whole validation sample. In addition, three recurrences were classified inaccurately and two recurrences were recorded more than 1 month later than they were actually diagnosed. All errors are presented in Table IV.
Of the five recurrences that were missing from the register, two had minor local recurrence of tongue cancer and underwent salvage surgery. The third case had two different primary oral cancers within a short time. The patient later had a regional recurrence, and it was assumed that the first primary was the origin of this recurrence. This might be the reason that normal follow-up procedures did not record this recurrence. The fourth case had a major recurrence shortly after finishing primary treatment. Finally, the fifth case had a regional recurrence in 2005.
There were also three cases where the site of first recurrence was classified differently in the register from that in patient records. It can sometimes be difficult to define the site of first recurrence, as the disease might be progressing rapidly to a more advanced state and the situation might be differently interpreted by the person managing the register. However, two out of the three cases seemed most likely to be clerical errors.
Discussion
Keeping a highly accurate quality register is essential for the ability to estimate treatment results and complication rates in any healthcare facility. Accurate registers are essential for quality control, resource allocation, and development of cancer care processes and research. However, as all registers have inherent limitations, the content of a register has to be validated before being relied upon (2).
Many of the register validation studies that are published concern registers on a national level, where data are collected from many different centres (10, 11). In this study, the register is kept and updated at the same hospital that provided all the registered cases with at least some of their cancer care. The register is managed by a small number of motivated staff. This probably contributes to the fact that the data concerning patient and tumour characteristics as well as treatment are highly accurate. It is rare that healthcare registries can achieve an accuracy of 98% as seen in the present case. The few incorrect data entries were probably mostly the result of human error. However, it is disappointing that despite high ambitions, the follow-up data are not as accurate as we would hope, and are of only moderate quality.
When looking into the cases in this study, it became apparent that several of the patients comprised complex cases, with multiple coinciding or sequential oral tumours, or primary tumours involving multiple subsites of the oral cavity. This sometimes makes diagnosis, staging, and classification of recurrences difficult and to some degree subjective. It is not always possible to determine whether a lesion is a local recurrence or a second primary, and this might impair the accuracy of registry data (12).
The strength of this study is that it establishes with high certainty that the baseline data in the register are accurate and can be used for research purposes, regardless of time period and county of referral. The study also revealed missing and incorrect data regarding recurrences; 22 out of the 73 cases in the sample had a recurrence, and 10 of these had a significant error in the registration.
One weakness of this study is that it cannot give much information as to why recurrences were missed despite the substantial effort made to keep the follow-up data accurate. In addition, the sample size was not large enough to give any reliable data on the rate of errors in the follow-up. Another weakness is that we did not assess the coverage of the register; however, we see no practical way of doing this. The best solution would be to implement automatic linking of relevant data from electronic patient records to healthcare registers, but depending on the complexity of the IT environment, this will take time to realize.
Another register validation study covering roughly the same geographical area as the register validated here was performed by Wadsten et al. (13), concerning registration of ductal carcinoma in situ in a breast cancer quality register. The study found that the proportion of correct baseline values for individual variables ranged from 92% to 99%, but that the situation was considerably worse for registration of recurrences, where only 65% of actual recurrences had been reported to the register. The situation matches the findings in our validation quite well, although the accuracy of our register is overall somewhat higher. This supports our theory that it is easier to maintain a high-quality register when few centres and only a few designated persons are involved in keeping the register.
Linder et al. reported from a validation study of the much larger Swedish National Register for Oesophageal and Gastric Cancer, which is a nationwide register with many participating hospitals. They found higher rates of missing and inaccurate data compared to our register: 6.5% of entries were inaccurate and 2.4% were missing. On the other hand, they showed an impressive accuracy in registered dates: 84.5% of cases had exact agreement on date of diagnosis (14). In our register, only 22% of cases were this accurate, although most of the discrepancies in the remaining 78% were very small.
Tumour staging in the registry was initially based on the TNM-6 system (15) and later on the TNM-7 system (16). The staging of oral cancer has not changed over these two systems. We validated the staging data by comparing registry data with patient records, thus reflecting the staging achieved with the diagnostics used at the time. Although there were no changes in the staging criteria, the diagnostic workup changed during the time covered by the register. It has been shown that the introduction of positron-emission tomography/computed tomography (CT) leads to a shift towards more advanced stages in some cases (17-20). The use of magnetic resonance imaging, increased quality of CT imaging, and the use of ultrasound guided fine-needle aspiration in the staging procedure have also probably contributed to this stage migration. That is, cases of a certain TNM stage from an earlier period might not be directly comparable with cases of the same TNM stage but from a later period.
The overall high accuracy makes this register a valuable source for research concerning patient and tumour characteristics and treatment data on patients with oral SCC in our region. However, the less accurate follow-up data implies that re-abstraction from patients' medical records seems unavoidable when studying recurrence. Alternatively, it might be preferred to look at overall survival, as dates of death can be collected from the highly valid Swedish population register, and data from the register on death causes kept by the Swedish Board of Health and Welfare can also be used (21-23). Finally, this study makes available data that can be used for comparison in future validation studies of local and regional healthcare quality registers.
In conclusion, data concerning patients, tumour status, and treatment in the Regional Head and Neck Register in Örebro are highly accurate. However, the follow-up data contain a higher rate of errors, that must be taken into consideration when evaluating outcome after treatment.
Acknowledgements
This work was supported by funding from Forskningskommittén in Region Örebro län.
Footnotes
This article is freely accessible online.
- Received November 16, 2018.
- Revision received November 28, 2018.
- Accepted November 29, 2018.
- Copyright© 2019, International Institute of Anticancer Research (Dr. George J. Delinasios), All rights reserved