Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Validation of forearm fracture diagnoses in administrative patient registers

Validation of forearm fracture diagnoses in administrative patient registers Summary The validity of forearm fracture diagnoses recorded in five Norwegian hospitals was investigated using image reports and medical records as gold standard. A relatively high completeness and correctness of the diagnoses was found. Algorithms used to define forearm fractures in administrative data should depend on study purpose. Purpose In Norway, forearm fractures are routinely recorded in the Norwegian Patient Registry (NPR). However, these data have not been validated. Data from patient administrative systems (PAS) at hospitals are sent unabridged to NPR. By using data from PAS, we aimed to examine (1) the validity of the forearm fracture diagnoses and (2) the usefulness of washout periods, follow-up codes, and procedure codes to define incident forearm fracture cases. Methods This hospital-based validation study included women and men aged ≥ 19 years referred to five hospitals for treat- ment of a forearm fracture during selected periods in 2015. Administrative data for the ICD-10 forearm fracture code S52 (with all subgroups) in PAS and the medical records were reviewed. X-ray and computed tomography (CT) reports from examinations of forearms were reviewed independently and linked to the data from PAS. Sensitivity and positive predictive values (PPVs) were calculated using image reports and/or review of medical records as gold standard. Results Among the 8482 reviewed image reports and medical records, 624 patients were identified with an incident forearm fracture during the study period. The sensitivity of PAS registrations was 90.4% (95% CI: 87.8–92.6). The PPV increased from 73.9% (95% CI: 70.6–77.0) in crude data to 90.5% (95% CI: 88.0–92.7) when using a washout period of 6 months. Using pro- cedure codes and follow-up codes in addition to 6-months washout increased the PPV to 94.0%, but the sensitivity fell to 69.0%. Conclusion A relatively high sensitivity of forearm fracture diagnoses was found in PAS. PPV varied depending on the algorithms used to define cases. Choice of algorithm should therefore depend on study purposes. The results give useful measures of forearm fracture diagnoses from administrative patient registers. Depending on local coding practices and treat- ment pathways, we infer that the findings are relevant to other fracture diagnoses and registers. Keywords Validation · Forearm fracture · Sensitivity · Positive predictive value Introduction Compared to the UK, Sweden, and Australia, a high inci- dence rate of forearm fractures has been reported in Oslo, Nor- Register data are frequently used for research, surveillance of way, where the diagnoses were verified by manually reviewing disease, and planning of healthcare services. However, hospital- medical records and X-ray reports [5]. Only one study has been based register data are collected for administrative purposes, published on forearm fractures in the Norwegian Patient Registry which affects the type of data available and data quality [1 ]. (NPR), but no validation of the diagnoses was performed [6]. The quality of register data varies depending on the data source, On the other hand, hip fracture diagnoses in the NPR have been diagnosis, and year [2, 3]. It is recommended that validation extensively studied [7, 8] and validated [9]. Hip fracture patients studies are performed before using register data for research [4]. are routinely treated surgically and thereby better captured in the hospital-based registries. In contrast, only 30% of forearm frac- tures are surgically treated in Norway, with a variation between Extended author information available on the last page of the article Vol.:(0123456789) 1 3 111 Page 2 of 9 Archives of Osteoporosis (2023) 18:111 hospitals of 16–40% [6]. Hence, forearm fractures are more chal- are treated in primary care only and therefore not reported lenging to capture in register data than hip fractures. to the NPR [10]. It is mandatory for the hospitals to report Most forearm fractures in Norway are treated at hospitals, monthly about patients treated at hospitals and emergency but some fractures may be treated in primary care only (e.g., units to NPR. Data are sent unabridged from PAS to NPR in rural areas with a decentralized X-ray service). In a study [16], and hospital discharge data can therefore be used as a comparing data from NPR and the primary health care regis- replacement for NPR data, minimizing the transfer of data ter, 93% of forearm fractures were captured by NPR whereas between institutions. 7% of all forearm fracture registrations were identified only in primary care (after wash-out and exclusion of records coded Data from patient administrative systems as follow-up visits) [10]. This is probably an overestimate, at the hospitals because some diagnoses in primary care represents suspected fractures that could not be separated from true fracture events. The Norwegian Capture the Fracture Initiative (NoFRACT) Medical record review at five primary care facilities located far is a multi-center study including patients treated at seven from hospitals found that 60% of the forearm fracture registra- hospitals in four health regions of Norway [11]. Patients tions were incident [10], but it is not known whether this figure aged ≥ 19 years treated for a forearm fracture were manually is representative for all primary care facilities treating forearm retrieved from PAS at five of the seven NoFRACT hospitals fractures in Norway. The background for this validation study in the following assigned periods: Oslo University Hospital, is the Norwegian Capture the Fracture Initiative (NoFRACT), Ullevål 2 Feb–1 Mar 2015, Drammen Hospital 2 Feb–22 a multi-center study at seven hospitals in four health regions Mar 2015, Bærum Hospital 2 Feb–26 Apr 2015, Univer- of Norway, that investigated the effect of a fracture liaison sity Hospital of North Norway, Tromsø 2 Feb–26 Apr 2015, service (FLS) on subsequent fragility fracture rates [11]. The and Molde Hospital 2 Feb–20 Sept 2015. The periods were outcome data in NoFRACT will be obtained from NPR and tailored to obtain approximately 200 forearm fracture reg- consequently there is a need for validation of hospital data. istrations from each hospital. The periods will hereafter be Incorrect medical coding can lead to both over- and under- referred to as “the study period.” reporting of specific diagnoses in administrative data. Another Patients were identified in PAS by a search for ICD- challenge in register-based research of fractures is to correctly 10 codes S52 with all sub-groups (forearm fractures) and define incident versus prevalent cases when the same patient S62.8 (fracture of other and unspecified parts of wrist and has multiple registrations of the same diagnosis code [12–14]. hand) (Fig. 1). None of the patients with an S62.8 registra- To handle this, it is common to introduce a washout period, tion (n = 4) had an incident forearm fracture, and they were which means that a patient can only count once within a spe- excluded from further analyses. We collected patient-level cic fi time window. However, other logged information can also information using the national identity number; age, sex, be used to improve the correctness of the administrative data, whether the fracture was coded as a main or additional diag- such as procedure codes, whether the S52 code is the main or nosis, date of examination, date of discharge, record date, additional diagnosis, and follow-up codes [12, 13]. name of the first treatment unit (if sustained in Norway), The aim of this study was to examine the sensitivity and date of surgery (if applicable), procedure-codes, and ICD-10 positive predictive value (PPV) of the forearm fracture diag- diagnosis codes indicating follow-up visits. nosis (S52) in Patient Administrative Systems (PAS) at five hospitals, using X-ray reports and/or medical record reviews as the gold standard. In addition, we wanted to examine the Data from conventional X‑ray and CT reports optimal washout period for identification of incident forearm fractures, and the usefulness of procedure codes and follow-up Radiological data was obtained independently of the PAS codes to improve the data quality. data. This data source was extracted from data systems at the departments of radiology at each hospital, using radiologic codes and keywords from both CT and conventional X-ray. Material and methods All CT and conventional X-ray examinations of the upper extremities were included in our search (except fingers). To Treatment of forearm fractures in Norway capture delayed registration and identify fractures sustained before the study period, conventional X-ray and CT reports In Norway, most patients with forearm fracture are treated were retrieved within the assigned study periods ± 1 week. at hospitals which report data to the NPR. These data are The Norwegian Classification of Radiological Procedures widely used for research [15]. In the larger cities, there are (NCRP) has been used since 2012, and we planned to search hospital-affiliated outpatient units that also report data to for the NCRP codes for examinations of the hand, wrist, the NPR. However, approximately 7% of forearm fractures 1 3 Archives of Osteoporosis (2023) 18:111 Page 3 of 9 111 Fig. 1 Flow diagram of data in the study of forearm fracture diagnoses (S52) from image reports and the patient admin- istrative system (PAS) in five hospitals forearm, elbow, and upper arm. However, the coding of these ICD‑10 diagnosis codes and surgical procedure NCRPs were incomplete and additional searches therefore codes included the keywords “forearm,” “wrist,” “hand,” “scaph- oid,” and “upper extremity.” Registrations in PAS were categorized as either main or All reports were manually checked to verify whether the additional diagnosis. The ICD-10 codes Z09.4 (follow-up patient sustained (1) an incident forearm fracture within the of fracture), Z09.0 (follow-up non-cancer), Z09.8 (follow- study period, (2) a prevalent forearm fracture before the up for specified diagnosis), Z46.7 (follow-up orthopedic study period, (3) no fracture, or (4) other type of fracture. equipment), Z47.0 (follow-up, removal of osteosynthesis Information on the national identity number, age, sex, date material), Z47.8 (follow-up, removal or revision of casting of the X-ray/CT, and reports were obtained. or fixation), Z88.8 (follow-up, complications after surgery), T92.1 (follow-up, sequela of fracture of arm), and T92.2 Combining data from X‑ray and medical records— (follow-up, sequela of wrist and hand) were considered defining the “gold standard” “follow-up code” (Table 1). We used the NOMESCO (Nordic Medico-Statistical The data from PAS and from the image reports were merged Committee) Classification of Surgical Procedures (NCSP) by the national identity number. The gold standard in this groups of codes, NCJ (Fracture surgery of elbow and fore- study was defined as all incident forearm fractures verified arm), NDJ (Fracture surgery of wrist and hand), TND from X-ray/CT reports or review of medical records. In cases (Minor procedures in wrist and hand), and TNC (Minor where X-ray reports were missing or the information was procedures in elbow and forearm) with all subgroups. inconclusive, fractures were verified by an additional review of the medical records. In cases where the image reports Statistical analyses were unclear, the results were compared to medical records, where a fracture could be clinically verified or excluded. Sensitivity was calculated by dividing the number of veri- In cases where no medical records were available and the fied incident forearm fractures identified in PAS (“true outcome was inconclusive (n = 13), orthopedic surgeons and positives”) by the gold standard. Positive predictive clinicians were consulted. value (PPV) was calculated by dividing the number of 1 3 111 Page 4 of 9 Archives of Osteoporosis (2023) 18:111 Table 1 Codes and criteria Term or code Explanation for diagnosis, washouts, and procedures used to create Crude cases All patients with a S52 code within the study period were included different algorithms to define Washout (3, 6, or 12 months) Prevalent S52 cases 3, 6, or 12 months prior to the registration were omitted forearm fracture cases in patient Main diagnosis Only records with S52 coded as main diagnosis were included administrative system Procedure codes Only patients with codes indicating incident fracture were included Follow-up codes Records with follow-up codes were excluded Procedure codes with all subgroups were included. The surgical procedure codes NCJ, NDJ, TND, and TNC were used “true positives” by the total number of patients identified Incident forearm fractures in PAS in PAS with S52 codes. Sensitivity and PPV with 95% confidence intervals were calculated by using the STATA A total of 1230 forearm fracture diagnose in 763 patients module “diagti.” were registered in PAS (Fig. 1). Of these 763 patients, 564 The sensitivity and PPV by different aspects of coding had a verified incident forearm fracture, 148 had a prevalent practice were investigated including timing of registra- forearm fracture, and 51 had no forearm fracture (Fig. 1). A tions, and combinations of codes (Table 1). For the crude total of 499 patients had one registration, while the remain- comparison of fracture registrations in PAS versus the ing patients had between two and 10 registrations of a fore- gold standard, all forearm fracture registrations in PAS arm fracture in the study period. The length of the study which were found to be prevalent fractures according to period (range 27–230 days) and number of patients (range the gold standard, were coded as “no fracture.” We per- 112–196) varied according to size of the hospital (Table 3). formed separate calculations where prevalent registra- Regarding patients with verified fractures in PAS, the tions within a period of 3, 6, and 12 months prior to the days between the PAS registration and the verified date PAS registration were omitted from the numerator and ranged between 0 and 92 days; 80.8% were registered on the the denominator, whereas the PAS registrations prior to same day, 90.4% within 14 days and 95.1% within 30 days. the washout were coded as “no fracture.” The rationale for doing this is that washout in register data means that Sensitivity and positive predictive values patients are only allowed to count once within a specified period. PAS covered 564 of the 624 verified incident forearm frac- Microsoft Excel 2016 (Microsoft, Redmond, WA, USA) tures and the sensitivity of the forearm fracture diagnosis was used for data collection and Stata 16 (StataCorp, Col- was 90.4% (95% CI 87.8–92.6). Among 60 patients with lege Station, TX, USA) for data cleaning, merging, and verified fractures not recorded in PAS, 9 had their primary analysis. treatment in another hospital. Other reasons for missing in PAS included miscoding (n = 1), external referral (n = 2), Results Table 2 Distribution of the verified incident forearm fracture sub- types in 624 patients at five Norwegian hospitals Verified incident forearm fractures—gold standard Fracture subtype n % A total of 7256 image reports from 5440 patients were S52.0 Fracture of upper end of ulna 25 4 retrieved from the five hospitals, and 555 of these patients S52.1 Fracture of upper end of radius 49 8 had a verified forearm fracture (Fig.  1). A total of 624 S52.2 Fracture of shaft of ulna 6 1 patients with incident forearm fractures were verified S52.3 Fracture of shaft of radius 14 2 either by image reports or medical records, the gold stand- S52.4 Fracture of shafts of both ulna and radius 0 0 ard. There were 6 patients with simultaneous bilateral S52.5 Fracture of lower end of radius 481 77 forearm fractures registered on the same day. No other S52.6 Fracture of lower end of both ulna and radius 29 5 patients had more than one forearm fracture event during S52.7 Multiple fractures of forearm 1 0 the study period. Women constituted 73% of the patients, S52.8 Fracture of other parts of forearm 18 3 and 77% of the fractures were located at the distal fore- S52.9 Fracture of forearm, part unspecified 1 0 arm (Table 2). The median age at fracture was 62 years [interquartile range (IQR): 52–73] in women, and 47 years a Verified by the gold standard (x-ray and CT report and/or medical [IQR: 33–64] years in men. records) 1 3 Archives of Osteoporosis (2023) 18:111 Page 5 of 9 111 Table 3 The study period, total number of registrations of S52 diag- patients with uncertain fracture status and the sensitivity noses and patients in the patient administrative system (PAS) at five was 91.3% (95% CI: 88.8–93.4%) and PPV = 91.0% (95% hospitals CI: 88.4–93.1%). Patients Use of washout periods for PPV calculation Period Registrations Main diagnoses Total (days) (n) (n) (%) (n) As shown in Table 5, washout periods of 3, 6, or 12 months Bærum 83 203 138 98 141 separated cases with prevalent from incident forearm frac- tures similarly. Eight prevalent fractures were sustained Drammen 48 250 122 72 169 Molde 230 200 104 93 112 more than 6 months before the PAS entry date, and if includ- ing a washout period of 6 months, we would categorize the Oslo 27 234 192 98 196 Tromsø 83 343 126 87 145 140 prevalent fractures correctly, increasing the PPV of the incident S52 fractures from 73.9 to 90.5% (Table 5). Total 471 1230 682 89 763 Total number of registrations in PAS Additional use of follow‑up codes and procedure codes for PPV calculation and that the fracture diagnosis was not verified until the sec- ond visit (n = 3). There were 6 patients with undetermined Among 564 patients with a verified incident fracture in PAS, date of fracture (possibly prevalent fractures). In 39 patients, 16 patients (2.8%) had registered follow-up codes. In records we were not able to find the reasons why they were miss- with S52 diagnoses verified as prevalent fractures, 62.4% ing in PAS. The highest proportion of unexplained missing had a follow-up code. Using follow-up codes in addition registrations in PAS was found at the hospital in Tromsø to 6 months washout increased the PPV from 74 to 88% (n = 13) where several remote locations have X-ray available (Fig. 2). in primary care, and digitally transfer images to the hospital In 682 patients with S52 as the main diagnosis, 6% had for consultation. no verified incident S52 fracture. Of 763 registrations with The 564 of 763 patients with verified incident forearm S52 codes in PAS, 682 (89%) were registered as the main fracture in PAS, resulted in a crude PPV of 73.9% (95% diagnosis code and 81 (11%) as an additional diagnosis code CI: 70.6–77.0) (Table 4). The main reason for miscoding (Table 3). Of these 81 diagnoses, 25 (31%) had an incident (n = 199) was that 148 (74%) of the patients had a verified S52 fracture, 47 (58%) had a prevalent S52 fracture, and 9 prevalent S52-fracture (Table 4). Most of the prevalent frac- (11%) had no verified S52 fracture. Among the 25 patients tures (n = 136, 92%) were sustained during the 3 months with incident S52 fractures as additional diagnosis, 12 before, whereas the remaining 12 were sustained more than (48%) patients had been to a primary care physician, were 3 months before the study period. In 51 miscoded registra- treated outside the hospital, or sustained the fracture abroad. tions, the patients had other fractures (S42, S62, S72, S82, Excluding patients with fracture as an additional diagnosis S92), or other injures but no incident forearm fracture veri- reduced the sensitivity and the PPV slightly (Fig. 2). Addi- fied by X-ray and CT report review or in medical records. tional use of follow-up codes increased the PPV more than In a sensitivity analysis with 6  months washout, we washout only but reduced the sensitivity. excluded 6 patients with uncertain date of fracture and 4 Table 5 Positive predictive value (PPV) using different washout peri- ods to define incident forearm fractures in the patient administrative Table 4 Type of miscoding of 199 registrations of S52 diagnosis system (PAS) codes in the patient administrative system at five hospitals, not veri- Registra- Verified Miscoded PPV fied as incident forearm fracture by x-ray or medical records tions in fractures Miscoded registrations n (%) PAS Fracture of shoulder and upper arm (S42) 3 2 Crude comparison 763 564 199 73.9 Fracture at wrist and hand level (S62) 11 6 3-month washout 627 564 63 90.0 Fracture of femur (S72) 2 1 6-month washout 623 564 59 90.5 Fracture of lower leg, including ankle (S82) 4 2 12-month washout 622 564 58 90.7 Fracture of foot, except ankle (S92) 1 1 Registrations in PAS (without prevalent fractures during washout) Other injuries (suspected fracture, strain, sprain etc.) 30 15 Incident fractures verified by X-ray/CT or medical journal review Prevalent fractures (sustained before study period) 148 74 Sum of miscoded registrations and prevalent fractures with different Total 199 100 washout criteria 1 3 111 Page 6 of 9 Archives of Osteoporosis (2023) 18:111 Fig. 2 Estimated sensitivity and positive predictive value (PPV) by different algorithms to define forearm fracture cases (algo- rithm details in Table 1) There were 297, 69, and 123, patients with a TND, TNC, more challenging to capture due to their diversity in treat- or NCJ code, whereas no patients had an NDJ procedure ment, place of treatment, and that hip fractures are routinely code. A total of 453 (80.3%) of the verified fractures in PAS treated surgically. had one or more of the procedure codes during the study In order to obtain estimates near the true value when cal- period. Additional use of surgical procedure codes increased culating incidence rates, it is advantageous if the false nega- the PPV more than washout alone, but the sensitivity was tive and the false positive cases balance out [1]. For studies lower. Using all available codes gave excellent PPV, but the of incidence rates of forearm fracture, the best combination sensitivity was reduced (Fig. 2). of sensitivity and PPV was found with 6-month washout only. On the other hand, if the data are to be used for other purposes, for example when studying an association in a Discussion cohort study, it might be more appropriate to use the algo- rithm that produced the highest PPV. Although the results In this study, forearm fracture registrations at five Nor - for 3- and 6-month washout were similar, it is reasonable to wegian hospitals were validated using reports from either use a 6-month washout as both treatment and fracture heal- X-ray, CT,or medical records as the gold standard. The sen- ing usually are terminated by then. sitivity was 90% and the crude PPV of 74% increased to The PPV is a valid measure of validity in the current 90% by using a washout period of 6 months. Registration study, but it only covers one dimension of registration prac- of follow-up codes and procedure codes was incomplete, tice. Estimating sensitivity is challenging in this type of and coding practices varied between the hospitals. The PPV study as it ideally would need to cover all patients seeking increased when using a combination of washout period of healthcare treatment during the study period [21]. This was 6 months, follow-up codes, and procedure codes but at the not feasible, and the sensitivity might be overestimated. The expense of the sensitivity. proportion of patients with miscoded S52 registrations (veri- To the best of our knowledge, a validation of forearm fied non-arm fractures) was 0.9%, and the proportion of true fracture diagnosis on a national level has not been performed S52 fractures miscoded as non-arm fractures is likely to be in Norway or any other Scandinavian country. However, a low. Another weakness regarding the sensitivity calculations subset of 471 incident distal radius fractures in the Skåne is that some patients with subsequent forearm fractures in Health Care register in Sweden, was validated by reviewing register data will only be counted once when introducing a medical records as the gold standard [17]. The sensitivity of washout period and that was not accounted for in our cal- the register data was 90% and PPV 94%. Our results of 90% culations. However, a study from Iceland of 2364 medical sensitivity and PPV are in line with the findings, although records verified incident forearm fractures reported that our PPV was somewhat lower. In a population-based vali- 11% of patients with a first forearm fracture sustained a sec- dation study of humeral fractures in the Danish National ond forearm fracture within 10 years [22]. Given a similar Patient Registry (DNPR) in 2017–2020 [18], the PPV was risk of subsequent forearm fractures in Norway, we would 89.3%, whereas another study of all orthopedic diagnoses exclude approximately 1.5% of incident forearm fractures in the DNPR in a 2-week period in 2006 reported a PPV of when using a 6-month washout period. Hence, the sensitiv- 86% [19]. In a validation study of 1000 hip fracture patients ity after 6-month washout is likely to be 1.5% lower than the in Norway, the PPV was 98.2% [9]. Likewise, a study from estimated 90.4% that we reported in this study. On the other the Finish patient register found that 98.1% of hip fractures hand, in the current study, nine of the patients not found had a hip fracture diagnosis [20]. The lower validity of in PAS (1.4%) were treated in other hospitals and would non-hip fractures probably reflects that other fractures are probably be captured if using data from the NPR. Although 1 3 Archives of Osteoporosis (2023) 18:111 Page 7 of 9 111 at the University of Oslo and at the hospitals. The data collected in data from PAS is said to be transferred unabridged to the this study were not available to other investigators or data managers NPR, some minor data processing seems to take place when than those included in project description (listed in the data protection combining data from all Norwegian hospitals. Future studies impact assessment). The investigators at the hospitals who collected should compare hospital data with data from NPR (collected data were only able to upload data and not access any other data in the safe research platform at the University of Oslo. Access to data from hospitals) on an individual level to obtain direct infor- for other investigators can be obtained after approval from Norwegian mation about the data from NPR. Directorate of Health, evaluation by the Norwegian Centre for Research A strength of this study is that we reviewed more than Data, and approvals from the data protection officers at the hospitals 8000 medical records and X-ray and CT reports from five and the University of Oslo. hospitals in Norway in 2015 and estimated both sensitivity Declarations and PPVs. We included data from only five of 44 hospitals treating forearm fractures in Norway. Still, the data are likely Ethics approval All procedures performed in studies involving human to be representative for the country as the included hospi- participants were in accordance with the ethical standards of the insti- tals vary in size and are located in all four health regions tutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. of Norway. Changes in classification systems and the use The study was evaluated by the Norwegian Centre for Research Data of more sensitive diagnostic methods over time (diagnostic (project number 587591). After performing a data protection impact drift) may hamper the interpretation of secular trends in inci- assessment (DPIA), the project was approved by the data protection dence rates. A limitation is that we only validated data from officer at the University of Oslo as pursuant to the General Data Pro- tective Regulation EU 2016/679. Exemption from consent for qual- the year 2015 and were unable to investigate any changes in ity assurance was obtained from the Norwegian Directorate of Health medical coding practice over time. It would have been useful (ref:19/3103-4). The project was also approved by the data protection to have data from three or more years, but due to methodo- officer at all hospitals. All data was uploaded directly from the patient logic challenges and limited resources this was not possible. administrative systems (PAS) and from the x-ray/CT reports into the research platform “Services for Sensitive Data” (TSD), which meets the requirements of Norwegian law regarding safe handling and stor- ing of sensitive data. All data merging, handling, and analyses were performed within this safe system. Conclusion Conflicts of interest FF reports speaking fees from UCB and Amgen; The data from the patient administrative system from five TTB reports speaker fees from UCB, Amgen, Roche, and Pharma hospitals in Norway showed a sensitivity of approximately Prim. Advisory board for UCB; JEG reports lecture fees from Or- 90% and the PPV varied between 74 and 95% depending tomedic Norway and LINK Norway; EMA reports advisory board for UCB and Amgen; RMJ reports consulting fees from Norwegian Direc- on the algorithm used to define incident forearm fractures. torate of Health, Norwegian Diabetes Association, Northern Norway The results are valuable measures of the validity of forearm Regional Health Authority and lecture fees from Novo Nordisk, Eli- fracture diagnoses obtained from administrative patient reg- Lilly, Astra-Zeneca, Sanofi; EFE reports consultant fees from UCB, isters. Depending on local coding practices and treatment Amgen, Gilead, Pharma Medico and honoraria from Amgen, UCB, and Novo Nordisk. The other authors declared that they have no con- pathways, our estimates are relevant for other fracture diag- flict of interest. noses and research based on register data in other countries. Open Access This article is licensed under a Creative Commons Attri- Acknowledgements We would like to acknowledge the contribu- bution 4.0 International License, which permits use, sharing, adapta- tion of Merete Finjarn, Anine Johansen, Cecilie Wagenheim, Ingrid tion, distribution and reproduction in any medium or format, as long Mikaelsen, Ann Kristin Schanche, and Olav Wetteland in data as you give appropriate credit to the original author(s) and the source, collection. provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are Author contribution CD, TKO, LBS, ÅB, and TTB were involved in included in the article's Creative Commons licence, unless indicated the the conception and design of the study. CD, TKO, LBS, TTB, ÅB, otherwise in a credit line to the material. If material is not included in JMS, CA, TB, TW, GH, FIN, AKH, RMJ, EF, GP, AAR, JK, EFE, the article's Creative Commons licence and your intended use is not LN, FF, WF, EMA, and JEG were involved in the acquisition of data, permitted by statutory regulation or exceeds the permitted use, you will or analysis and interpretation of data. The conventional X-ray and CT need to obtain permission directly from the copyright holder. To view a reports were reviewed independently by CD and TKO. In cases with copy of this licence, visit http://cr eativ ecommons. or g/licen ses/ b y/4.0/ . uncertain diagnoses, clinicians (ÅB and TTB) and orthopaedic sur- geons (JEG, LBS, and FF) were consulted. TKO, CD, and ÅB drafted the manuscript and the other authors revised it critically. All authors have approved the final version of the manuscript. References Funding Open access funding provided by University of Oslo (incl Oslo University Hospital) 1. Thygesen LC, Ersboll AK (2014) When the entire population is the sample: strengths and limitations in register-based epide- Data availability Availability of data depend on permissions from the miology. Eur J Epidemiol 29:551–558. https:// doi. org/ 10. 1007/ Norwegian Directorate of Health, evaluation by the Norwegian Cen- s10654- 013- 9873-0 tre for Research Data, and approval from the data protection officer 1 3 111 Page 8 of 9 Archives of Osteoporosis (2023) 18:111 2. Schmidt M, Schmidt SA, Sandegaard JL, Ehrenstein V, Pedersen 12. Brophy S, John G, Evans E, Lyons RA (2006) Methodological L, Sorensen HT (2015) The Danish National Patient Registry: a issues in the identification of hip fractures using routine hospital review of content, data quality, and research potential. Clin Epi- data: a database study. Osteoporos Int 17:405–409. https://doi. or g/ demiol 7:449–490. https:// doi. org/ 10. 2147/ CLEP. S9112510. 1007/ s00198- 005- 2038-6 3. Arts DG, De Keizer NF, Scheffer GJ (2002) Defining and improv - 13. Langley J, Stephenson S, Cryer C, Borman B (2002) Traps for the ing data quality in medical registries: a literature review, case unwary in estimating person based injury incidence using hospital study, and generic framework. J Am Med Inform Assoc 9:600– discharge data. Inj Prev 8:332–337. https://doi. or g/10. 1136/ ip.8. 4. 332 611. https:// doi. org/ 10. 1197/ jamia. m1087 14. Boufous S, Williamson A (2003) Reporting of the incidence 4. Ehrenstein V, Petersen I, Smeeth L, Jick SS, Benchimol EI, Lud- of hospitalised injuries: numerator issues. Inj Prev 9:370–375. vigsson JF, Sorensen HT (2016) Helping everyone do better: a https:// doi. org/ 10. 1136/ ip.9. 4. 370 call for validation studies of routinely recorded health data. Clin 15. Laugesen K, Ludvigsson JF, Schmidt M, Gissler M, Valdimarsdottir Epidemiol 8:49–51. https:// doi. org/ 10. 2147/ CLEP. S1044 48 UA, Lunde A, Sorensen HT (2021) Nordic health registry-based 5. Lofthus CM, Frihagen F, Meyer HE, Nordsletten L, Melhuus K, research: a review of health care systems and key registries. Clin Falch JA (2008) Epidemiology of distal forearm fractures in Oslo. Epidemiol 13:533–554. https:// doi. org/ 10. 2147/ CLEP. S3149 59 Norway Osteoporos Int 19:781–786. h t t p s : / / d o i . o r g / 1 0 . 1 0 0 7 / 16. Varmdal T, Bakken IJ, Janszky I, Wethal T, Ellekjaer H, Rohweder s00198- 007- 0499-5 G, Fjaertoft H, Ebbing M, Bonaa KH (2016) Comparison of the 6. Kvernmo HD, Otterdal P, Balteskard L (2017) Treatment of wrist validity of stroke diagnoses in a medical quality register and an fractures 2009–14. Tidsskr Nor Laegeforen 137. https:// doi. org/ administrative health register. Scand J Public Health 44:143–149. 10. 4045/ tidss kr. 17. 0065https:// doi. org/ 10. 1177/ 14034 94815 621641 7. Omsland TK, Holvik K, Meyer HE, Center JR, Emaus N, Tell GS, 17. Rosengren BE, Karlsson M, Petersson I, Englund M (2015) The Schei B, Tverdal A, Gjesdal CG, Grimnes G, Forsmo S, Eisman 21st-century landscape of adult fractures: cohort study of a com- JA, Sogaard AJ (2012) Hip fractures in Norway 1999–2008: time plete adult regional population. J Bone Miner Res 30:535–542. trends in total incidence and second hip fracture rates: a NORE-https:// doi. org/ 10. 1002/ jbmr. 2370 POS study. Eur J Epidemiol 27:807–814. https://doi. or g/10. 1007/ 18. Karimi D, Houkjaer L, Gundtoft P, Brorson S, Viberg B (2023) s10654- 012- 9711-9 Positive predictive value of humeral fractures in the Danish 8. Sogaard AJ, Holvik K, Meyer HE, Tell GS, Gjesdal CG, Emaus N, National Patient Registry. Dan Med J 70(4):A10220612 Grimnes G, Schei B, Forsmo S, Omsland TK (2016) Continued decline 19. Lass P, Lilholt J, Thomsen L, Lundbye-Christensen S, Enevoldsen in hip fracture incidence in Norway: a NOREPOS study. Osteoporos H, Simonsen OH (2006) The quality of diagnosis and procedure Int 27:2217–2222. https:// doi. org/ 10. 1007/ s00198- 016- 3516-8 coding in Orthopaedic surgery Northern Jutland. Ugeskr Laeger 9. Hoiberg MP, Gram J, Hermann P, Brixen K, Haugeberg G (2014) 168:4212–4215 The incidence of hip fractures in Norway -accuracy of the national 20. Sund R, Nurmi-Luthje I, Luthje P, Tanninen S, Narinen A, Kes- Norwegian patient registry. BMC Musculoskelet Disord 15:372. kimaki I (2007) Comparing properties of audit data and routinely https:// doi. org/ 10. 1186/ 1471- 2474- 15- 372 collected register data in case of performance assessment of hip 10. Dahl C, Ohm E, Solbakken SM, Anwar N, Holvik K, Madsen C, fracture treatment in Finland. Methods Inf Med 46:558–566. Frihagen F, Bjornerem A, Igland Nissen F, Solberg LB, Omsland https:// doi. org/ 10. 1160/ me0382 TK (2023) Forearm fractures - are we counting them all? An 21. Fox MP, Lash TL, Bodnar LM (2020) Common misconceptions attempt to identify and include the missing fractures treated in about validation studies. Int J Epidemiol 49:1392–1396. https:// primary care. Scand J Prim Health Care 1–10. https:// doi. org/ 10. doi. org/ 10. 1093/ ije/ dyaa0 90 1080/ 02813 432. 2023. 22310 28 22. Kanis JA, Johansson H, Oden A, Harvey NC, Gudnason V, 11. Andreasen C, Solberg LB, Basso T, Borgen TT, Dahl C, Wisloff T, Sanders KM, Sigurdsson G, Siggeirsdottir K, Fitzpatrick LA, Hagen G, Apalset EM, Gjertsen JE, Figved W, Hubschle LM, Stutzer Borgstrom F, McCloskey EV (2018) Characteristics of recurrent JM, Elvenes J, Joakimsen RM, Syversen U, Eriksen EF, Nordsletten fractures. Osteoporos Int 29:1747–1757. https://doi. or g/10. 1007/ L, Frihagen F, Omsland TK, Bjornerem A (2018) Effect of a fracture s00198- 018- 4502-0 liaison service on the rate of subsequent fracture among patients with a fragility fracture in the Norwegian Capture the Fracture Initiative Publisher's note Springer Nature remains neutral with regard to (NoFRACT): a trial protocol. JAMA Netw Open 1:e185701. https:// jurisdictional claims in published maps and institutional affiliations. doi. org/ 10. 1001/ jaman etwor kopen. 2018. 5701 1 3 Archives of Osteoporosis (2023) 18:111 Page 9 of 9 111 Authors and Affiliations 1 2 3,4,5 6 4,7 Tone Kristin Omsland  · Lene B. Solberg  · Åshild Bjørnerem  · Tove T. Borgen  · Camilla Andreasen  · 8 9 10 11,12 13,14 15,16 Torbjørn Wisløff  · Gunhild Hagen  · Trude Basso  · Jan‑Erik Gjertsen  · Ellen M. Apalset  · Wender Figved  · 17 3,4,7 4,7 4,18 1 1 Jens M. Stutzer  · Frida I. Nissen  · Ann K. Hansen  · Ragnar M. Joakimsen  · Elisa Figari  · Geoffrey Peel  · 1 4 19,20 2,16 16,21 1 Ali A. Rashid  · Jashar Khoshkhabari  · Erik F. Eriksen  · Lars Nordsletten  · Frede Frihagen  · Cecilie Dahl * Tone Kristin Omsland Department of Orthopaedic Surgery, Haukeland University t.k.omsland@medisin.uio.no Hospital, Bergen, Norway Department of Clinical Medicine, University of Bergen, Department of Community Medicine and Global Health, Bergen, Norway Institute of Health and Society, University of Oslo, Blindern, Po box 1130, 0318 Oslo, Norway Bergen Group of Epidemiology and Biomarkers in Rheumatic Disease, Department of Rheumatology, Division of Orthopaedic Surgery, Oslo University Hospital, Haukeland University Hospital, Bergen, Norway Oslo, Norway Department of Global Public Health and Primary Care, Department of Obstetrics and Gynaecology, University University of Bergen, Bergen, Norway Hospital of North Norway, Tromsø, Norway Department of Orthopaedic Surgery, Vestre Viken Hospital Department of Clinical Medicine, The Arctic University Trust, Bærum Hospital, Gjettum, Norway of Norway, Tromsø, Norway Institute of Clinical Medicine, University of Oslo, Oslo, Norwegian Research Centre for Women’s Health, Oslo Norway University Hospital, Oslo, Norway Department of Orthopaedic Surgery, Møre and Romsdal Department of Rheumatology, Vestre Viken Hospital Trust, Hospital Trust, Hospital of Molde, Molde, Norway Drammen Hospital, Drammen, Norway Department of Medicine, University Hospital of North Department of Orthopaedic Surgery, University Hospital Norway, Tromsø, Norway of North Norway, Tromsø, Norway Pilestredet Park Specialist Centre, Oslo, Norway Health Services Research Unit, Akershus University Hospital, Lørenskog, Norway Faculty of Dentistry, University of Oslo, Oslo, Norway 9 21 Department of Health Services, Norwegian Institute Department of Orthopaedic Surgery, Østfold Hospital Trust, of Public Health, Oslo, Norway Grålum, Norway Department of Orthopaedic Surgery, St. Olavs University Hospital, Trondheim, Norway 1 3 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Archives of Osteoporosis Springer Journals

Validation of forearm fracture diagnoses in administrative patient registers

Download PDF
9 pages

Loading next page...
 
/lp/springer-journals/validation-of-forearm-fracture-diagnoses-in-administrative-patient-Wv0Qke0dfE

References (30)

Publisher
Springer Journals
Copyright
Copyright © The Author(s) 2023
ISSN
1862-3522
eISSN
1862-3514
DOI
10.1007/s11657-023-01322-x
Publisher site
See Article on Publisher Site

Abstract

Summary The validity of forearm fracture diagnoses recorded in five Norwegian hospitals was investigated using image reports and medical records as gold standard. A relatively high completeness and correctness of the diagnoses was found. Algorithms used to define forearm fractures in administrative data should depend on study purpose. Purpose In Norway, forearm fractures are routinely recorded in the Norwegian Patient Registry (NPR). However, these data have not been validated. Data from patient administrative systems (PAS) at hospitals are sent unabridged to NPR. By using data from PAS, we aimed to examine (1) the validity of the forearm fracture diagnoses and (2) the usefulness of washout periods, follow-up codes, and procedure codes to define incident forearm fracture cases. Methods This hospital-based validation study included women and men aged ≥ 19 years referred to five hospitals for treat- ment of a forearm fracture during selected periods in 2015. Administrative data for the ICD-10 forearm fracture code S52 (with all subgroups) in PAS and the medical records were reviewed. X-ray and computed tomography (CT) reports from examinations of forearms were reviewed independently and linked to the data from PAS. Sensitivity and positive predictive values (PPVs) were calculated using image reports and/or review of medical records as gold standard. Results Among the 8482 reviewed image reports and medical records, 624 patients were identified with an incident forearm fracture during the study period. The sensitivity of PAS registrations was 90.4% (95% CI: 87.8–92.6). The PPV increased from 73.9% (95% CI: 70.6–77.0) in crude data to 90.5% (95% CI: 88.0–92.7) when using a washout period of 6 months. Using pro- cedure codes and follow-up codes in addition to 6-months washout increased the PPV to 94.0%, but the sensitivity fell to 69.0%. Conclusion A relatively high sensitivity of forearm fracture diagnoses was found in PAS. PPV varied depending on the algorithms used to define cases. Choice of algorithm should therefore depend on study purposes. The results give useful measures of forearm fracture diagnoses from administrative patient registers. Depending on local coding practices and treat- ment pathways, we infer that the findings are relevant to other fracture diagnoses and registers. Keywords Validation · Forearm fracture · Sensitivity · Positive predictive value Introduction Compared to the UK, Sweden, and Australia, a high inci- dence rate of forearm fractures has been reported in Oslo, Nor- Register data are frequently used for research, surveillance of way, where the diagnoses were verified by manually reviewing disease, and planning of healthcare services. However, hospital- medical records and X-ray reports [5]. Only one study has been based register data are collected for administrative purposes, published on forearm fractures in the Norwegian Patient Registry which affects the type of data available and data quality [1 ]. (NPR), but no validation of the diagnoses was performed [6]. The quality of register data varies depending on the data source, On the other hand, hip fracture diagnoses in the NPR have been diagnosis, and year [2, 3]. It is recommended that validation extensively studied [7, 8] and validated [9]. Hip fracture patients studies are performed before using register data for research [4]. are routinely treated surgically and thereby better captured in the hospital-based registries. In contrast, only 30% of forearm frac- tures are surgically treated in Norway, with a variation between Extended author information available on the last page of the article Vol.:(0123456789) 1 3 111 Page 2 of 9 Archives of Osteoporosis (2023) 18:111 hospitals of 16–40% [6]. Hence, forearm fractures are more chal- are treated in primary care only and therefore not reported lenging to capture in register data than hip fractures. to the NPR [10]. It is mandatory for the hospitals to report Most forearm fractures in Norway are treated at hospitals, monthly about patients treated at hospitals and emergency but some fractures may be treated in primary care only (e.g., units to NPR. Data are sent unabridged from PAS to NPR in rural areas with a decentralized X-ray service). In a study [16], and hospital discharge data can therefore be used as a comparing data from NPR and the primary health care regis- replacement for NPR data, minimizing the transfer of data ter, 93% of forearm fractures were captured by NPR whereas between institutions. 7% of all forearm fracture registrations were identified only in primary care (after wash-out and exclusion of records coded Data from patient administrative systems as follow-up visits) [10]. This is probably an overestimate, at the hospitals because some diagnoses in primary care represents suspected fractures that could not be separated from true fracture events. The Norwegian Capture the Fracture Initiative (NoFRACT) Medical record review at five primary care facilities located far is a multi-center study including patients treated at seven from hospitals found that 60% of the forearm fracture registra- hospitals in four health regions of Norway [11]. Patients tions were incident [10], but it is not known whether this figure aged ≥ 19 years treated for a forearm fracture were manually is representative for all primary care facilities treating forearm retrieved from PAS at five of the seven NoFRACT hospitals fractures in Norway. The background for this validation study in the following assigned periods: Oslo University Hospital, is the Norwegian Capture the Fracture Initiative (NoFRACT), Ullevål 2 Feb–1 Mar 2015, Drammen Hospital 2 Feb–22 a multi-center study at seven hospitals in four health regions Mar 2015, Bærum Hospital 2 Feb–26 Apr 2015, Univer- of Norway, that investigated the effect of a fracture liaison sity Hospital of North Norway, Tromsø 2 Feb–26 Apr 2015, service (FLS) on subsequent fragility fracture rates [11]. The and Molde Hospital 2 Feb–20 Sept 2015. The periods were outcome data in NoFRACT will be obtained from NPR and tailored to obtain approximately 200 forearm fracture reg- consequently there is a need for validation of hospital data. istrations from each hospital. The periods will hereafter be Incorrect medical coding can lead to both over- and under- referred to as “the study period.” reporting of specific diagnoses in administrative data. Another Patients were identified in PAS by a search for ICD- challenge in register-based research of fractures is to correctly 10 codes S52 with all sub-groups (forearm fractures) and define incident versus prevalent cases when the same patient S62.8 (fracture of other and unspecified parts of wrist and has multiple registrations of the same diagnosis code [12–14]. hand) (Fig. 1). None of the patients with an S62.8 registra- To handle this, it is common to introduce a washout period, tion (n = 4) had an incident forearm fracture, and they were which means that a patient can only count once within a spe- excluded from further analyses. We collected patient-level cic fi time window. However, other logged information can also information using the national identity number; age, sex, be used to improve the correctness of the administrative data, whether the fracture was coded as a main or additional diag- such as procedure codes, whether the S52 code is the main or nosis, date of examination, date of discharge, record date, additional diagnosis, and follow-up codes [12, 13]. name of the first treatment unit (if sustained in Norway), The aim of this study was to examine the sensitivity and date of surgery (if applicable), procedure-codes, and ICD-10 positive predictive value (PPV) of the forearm fracture diag- diagnosis codes indicating follow-up visits. nosis (S52) in Patient Administrative Systems (PAS) at five hospitals, using X-ray reports and/or medical record reviews as the gold standard. In addition, we wanted to examine the Data from conventional X‑ray and CT reports optimal washout period for identification of incident forearm fractures, and the usefulness of procedure codes and follow-up Radiological data was obtained independently of the PAS codes to improve the data quality. data. This data source was extracted from data systems at the departments of radiology at each hospital, using radiologic codes and keywords from both CT and conventional X-ray. Material and methods All CT and conventional X-ray examinations of the upper extremities were included in our search (except fingers). To Treatment of forearm fractures in Norway capture delayed registration and identify fractures sustained before the study period, conventional X-ray and CT reports In Norway, most patients with forearm fracture are treated were retrieved within the assigned study periods ± 1 week. at hospitals which report data to the NPR. These data are The Norwegian Classification of Radiological Procedures widely used for research [15]. In the larger cities, there are (NCRP) has been used since 2012, and we planned to search hospital-affiliated outpatient units that also report data to for the NCRP codes for examinations of the hand, wrist, the NPR. However, approximately 7% of forearm fractures 1 3 Archives of Osteoporosis (2023) 18:111 Page 3 of 9 111 Fig. 1 Flow diagram of data in the study of forearm fracture diagnoses (S52) from image reports and the patient admin- istrative system (PAS) in five hospitals forearm, elbow, and upper arm. However, the coding of these ICD‑10 diagnosis codes and surgical procedure NCRPs were incomplete and additional searches therefore codes included the keywords “forearm,” “wrist,” “hand,” “scaph- oid,” and “upper extremity.” Registrations in PAS were categorized as either main or All reports were manually checked to verify whether the additional diagnosis. The ICD-10 codes Z09.4 (follow-up patient sustained (1) an incident forearm fracture within the of fracture), Z09.0 (follow-up non-cancer), Z09.8 (follow- study period, (2) a prevalent forearm fracture before the up for specified diagnosis), Z46.7 (follow-up orthopedic study period, (3) no fracture, or (4) other type of fracture. equipment), Z47.0 (follow-up, removal of osteosynthesis Information on the national identity number, age, sex, date material), Z47.8 (follow-up, removal or revision of casting of the X-ray/CT, and reports were obtained. or fixation), Z88.8 (follow-up, complications after surgery), T92.1 (follow-up, sequela of fracture of arm), and T92.2 Combining data from X‑ray and medical records— (follow-up, sequela of wrist and hand) were considered defining the “gold standard” “follow-up code” (Table 1). We used the NOMESCO (Nordic Medico-Statistical The data from PAS and from the image reports were merged Committee) Classification of Surgical Procedures (NCSP) by the national identity number. The gold standard in this groups of codes, NCJ (Fracture surgery of elbow and fore- study was defined as all incident forearm fractures verified arm), NDJ (Fracture surgery of wrist and hand), TND from X-ray/CT reports or review of medical records. In cases (Minor procedures in wrist and hand), and TNC (Minor where X-ray reports were missing or the information was procedures in elbow and forearm) with all subgroups. inconclusive, fractures were verified by an additional review of the medical records. In cases where the image reports Statistical analyses were unclear, the results were compared to medical records, where a fracture could be clinically verified or excluded. Sensitivity was calculated by dividing the number of veri- In cases where no medical records were available and the fied incident forearm fractures identified in PAS (“true outcome was inconclusive (n = 13), orthopedic surgeons and positives”) by the gold standard. Positive predictive clinicians were consulted. value (PPV) was calculated by dividing the number of 1 3 111 Page 4 of 9 Archives of Osteoporosis (2023) 18:111 Table 1 Codes and criteria Term or code Explanation for diagnosis, washouts, and procedures used to create Crude cases All patients with a S52 code within the study period were included different algorithms to define Washout (3, 6, or 12 months) Prevalent S52 cases 3, 6, or 12 months prior to the registration were omitted forearm fracture cases in patient Main diagnosis Only records with S52 coded as main diagnosis were included administrative system Procedure codes Only patients with codes indicating incident fracture were included Follow-up codes Records with follow-up codes were excluded Procedure codes with all subgroups were included. The surgical procedure codes NCJ, NDJ, TND, and TNC were used “true positives” by the total number of patients identified Incident forearm fractures in PAS in PAS with S52 codes. Sensitivity and PPV with 95% confidence intervals were calculated by using the STATA A total of 1230 forearm fracture diagnose in 763 patients module “diagti.” were registered in PAS (Fig. 1). Of these 763 patients, 564 The sensitivity and PPV by different aspects of coding had a verified incident forearm fracture, 148 had a prevalent practice were investigated including timing of registra- forearm fracture, and 51 had no forearm fracture (Fig. 1). A tions, and combinations of codes (Table 1). For the crude total of 499 patients had one registration, while the remain- comparison of fracture registrations in PAS versus the ing patients had between two and 10 registrations of a fore- gold standard, all forearm fracture registrations in PAS arm fracture in the study period. The length of the study which were found to be prevalent fractures according to period (range 27–230 days) and number of patients (range the gold standard, were coded as “no fracture.” We per- 112–196) varied according to size of the hospital (Table 3). formed separate calculations where prevalent registra- Regarding patients with verified fractures in PAS, the tions within a period of 3, 6, and 12 months prior to the days between the PAS registration and the verified date PAS registration were omitted from the numerator and ranged between 0 and 92 days; 80.8% were registered on the the denominator, whereas the PAS registrations prior to same day, 90.4% within 14 days and 95.1% within 30 days. the washout were coded as “no fracture.” The rationale for doing this is that washout in register data means that Sensitivity and positive predictive values patients are only allowed to count once within a specified period. PAS covered 564 of the 624 verified incident forearm frac- Microsoft Excel 2016 (Microsoft, Redmond, WA, USA) tures and the sensitivity of the forearm fracture diagnosis was used for data collection and Stata 16 (StataCorp, Col- was 90.4% (95% CI 87.8–92.6). Among 60 patients with lege Station, TX, USA) for data cleaning, merging, and verified fractures not recorded in PAS, 9 had their primary analysis. treatment in another hospital. Other reasons for missing in PAS included miscoding (n = 1), external referral (n = 2), Results Table 2 Distribution of the verified incident forearm fracture sub- types in 624 patients at five Norwegian hospitals Verified incident forearm fractures—gold standard Fracture subtype n % A total of 7256 image reports from 5440 patients were S52.0 Fracture of upper end of ulna 25 4 retrieved from the five hospitals, and 555 of these patients S52.1 Fracture of upper end of radius 49 8 had a verified forearm fracture (Fig.  1). A total of 624 S52.2 Fracture of shaft of ulna 6 1 patients with incident forearm fractures were verified S52.3 Fracture of shaft of radius 14 2 either by image reports or medical records, the gold stand- S52.4 Fracture of shafts of both ulna and radius 0 0 ard. There were 6 patients with simultaneous bilateral S52.5 Fracture of lower end of radius 481 77 forearm fractures registered on the same day. No other S52.6 Fracture of lower end of both ulna and radius 29 5 patients had more than one forearm fracture event during S52.7 Multiple fractures of forearm 1 0 the study period. Women constituted 73% of the patients, S52.8 Fracture of other parts of forearm 18 3 and 77% of the fractures were located at the distal fore- S52.9 Fracture of forearm, part unspecified 1 0 arm (Table 2). The median age at fracture was 62 years [interquartile range (IQR): 52–73] in women, and 47 years a Verified by the gold standard (x-ray and CT report and/or medical [IQR: 33–64] years in men. records) 1 3 Archives of Osteoporosis (2023) 18:111 Page 5 of 9 111 Table 3 The study period, total number of registrations of S52 diag- patients with uncertain fracture status and the sensitivity noses and patients in the patient administrative system (PAS) at five was 91.3% (95% CI: 88.8–93.4%) and PPV = 91.0% (95% hospitals CI: 88.4–93.1%). Patients Use of washout periods for PPV calculation Period Registrations Main diagnoses Total (days) (n) (n) (%) (n) As shown in Table 5, washout periods of 3, 6, or 12 months Bærum 83 203 138 98 141 separated cases with prevalent from incident forearm frac- tures similarly. Eight prevalent fractures were sustained Drammen 48 250 122 72 169 Molde 230 200 104 93 112 more than 6 months before the PAS entry date, and if includ- ing a washout period of 6 months, we would categorize the Oslo 27 234 192 98 196 Tromsø 83 343 126 87 145 140 prevalent fractures correctly, increasing the PPV of the incident S52 fractures from 73.9 to 90.5% (Table 5). Total 471 1230 682 89 763 Total number of registrations in PAS Additional use of follow‑up codes and procedure codes for PPV calculation and that the fracture diagnosis was not verified until the sec- ond visit (n = 3). There were 6 patients with undetermined Among 564 patients with a verified incident fracture in PAS, date of fracture (possibly prevalent fractures). In 39 patients, 16 patients (2.8%) had registered follow-up codes. In records we were not able to find the reasons why they were miss- with S52 diagnoses verified as prevalent fractures, 62.4% ing in PAS. The highest proportion of unexplained missing had a follow-up code. Using follow-up codes in addition registrations in PAS was found at the hospital in Tromsø to 6 months washout increased the PPV from 74 to 88% (n = 13) where several remote locations have X-ray available (Fig. 2). in primary care, and digitally transfer images to the hospital In 682 patients with S52 as the main diagnosis, 6% had for consultation. no verified incident S52 fracture. Of 763 registrations with The 564 of 763 patients with verified incident forearm S52 codes in PAS, 682 (89%) were registered as the main fracture in PAS, resulted in a crude PPV of 73.9% (95% diagnosis code and 81 (11%) as an additional diagnosis code CI: 70.6–77.0) (Table 4). The main reason for miscoding (Table 3). Of these 81 diagnoses, 25 (31%) had an incident (n = 199) was that 148 (74%) of the patients had a verified S52 fracture, 47 (58%) had a prevalent S52 fracture, and 9 prevalent S52-fracture (Table 4). Most of the prevalent frac- (11%) had no verified S52 fracture. Among the 25 patients tures (n = 136, 92%) were sustained during the 3 months with incident S52 fractures as additional diagnosis, 12 before, whereas the remaining 12 were sustained more than (48%) patients had been to a primary care physician, were 3 months before the study period. In 51 miscoded registra- treated outside the hospital, or sustained the fracture abroad. tions, the patients had other fractures (S42, S62, S72, S82, Excluding patients with fracture as an additional diagnosis S92), or other injures but no incident forearm fracture veri- reduced the sensitivity and the PPV slightly (Fig. 2). Addi- fied by X-ray and CT report review or in medical records. tional use of follow-up codes increased the PPV more than In a sensitivity analysis with 6  months washout, we washout only but reduced the sensitivity. excluded 6 patients with uncertain date of fracture and 4 Table 5 Positive predictive value (PPV) using different washout peri- ods to define incident forearm fractures in the patient administrative Table 4 Type of miscoding of 199 registrations of S52 diagnosis system (PAS) codes in the patient administrative system at five hospitals, not veri- Registra- Verified Miscoded PPV fied as incident forearm fracture by x-ray or medical records tions in fractures Miscoded registrations n (%) PAS Fracture of shoulder and upper arm (S42) 3 2 Crude comparison 763 564 199 73.9 Fracture at wrist and hand level (S62) 11 6 3-month washout 627 564 63 90.0 Fracture of femur (S72) 2 1 6-month washout 623 564 59 90.5 Fracture of lower leg, including ankle (S82) 4 2 12-month washout 622 564 58 90.7 Fracture of foot, except ankle (S92) 1 1 Registrations in PAS (without prevalent fractures during washout) Other injuries (suspected fracture, strain, sprain etc.) 30 15 Incident fractures verified by X-ray/CT or medical journal review Prevalent fractures (sustained before study period) 148 74 Sum of miscoded registrations and prevalent fractures with different Total 199 100 washout criteria 1 3 111 Page 6 of 9 Archives of Osteoporosis (2023) 18:111 Fig. 2 Estimated sensitivity and positive predictive value (PPV) by different algorithms to define forearm fracture cases (algo- rithm details in Table 1) There were 297, 69, and 123, patients with a TND, TNC, more challenging to capture due to their diversity in treat- or NCJ code, whereas no patients had an NDJ procedure ment, place of treatment, and that hip fractures are routinely code. A total of 453 (80.3%) of the verified fractures in PAS treated surgically. had one or more of the procedure codes during the study In order to obtain estimates near the true value when cal- period. Additional use of surgical procedure codes increased culating incidence rates, it is advantageous if the false nega- the PPV more than washout alone, but the sensitivity was tive and the false positive cases balance out [1]. For studies lower. Using all available codes gave excellent PPV, but the of incidence rates of forearm fracture, the best combination sensitivity was reduced (Fig. 2). of sensitivity and PPV was found with 6-month washout only. On the other hand, if the data are to be used for other purposes, for example when studying an association in a Discussion cohort study, it might be more appropriate to use the algo- rithm that produced the highest PPV. Although the results In this study, forearm fracture registrations at five Nor - for 3- and 6-month washout were similar, it is reasonable to wegian hospitals were validated using reports from either use a 6-month washout as both treatment and fracture heal- X-ray, CT,or medical records as the gold standard. The sen- ing usually are terminated by then. sitivity was 90% and the crude PPV of 74% increased to The PPV is a valid measure of validity in the current 90% by using a washout period of 6 months. Registration study, but it only covers one dimension of registration prac- of follow-up codes and procedure codes was incomplete, tice. Estimating sensitivity is challenging in this type of and coding practices varied between the hospitals. The PPV study as it ideally would need to cover all patients seeking increased when using a combination of washout period of healthcare treatment during the study period [21]. This was 6 months, follow-up codes, and procedure codes but at the not feasible, and the sensitivity might be overestimated. The expense of the sensitivity. proportion of patients with miscoded S52 registrations (veri- To the best of our knowledge, a validation of forearm fied non-arm fractures) was 0.9%, and the proportion of true fracture diagnosis on a national level has not been performed S52 fractures miscoded as non-arm fractures is likely to be in Norway or any other Scandinavian country. However, a low. Another weakness regarding the sensitivity calculations subset of 471 incident distal radius fractures in the Skåne is that some patients with subsequent forearm fractures in Health Care register in Sweden, was validated by reviewing register data will only be counted once when introducing a medical records as the gold standard [17]. The sensitivity of washout period and that was not accounted for in our cal- the register data was 90% and PPV 94%. Our results of 90% culations. However, a study from Iceland of 2364 medical sensitivity and PPV are in line with the findings, although records verified incident forearm fractures reported that our PPV was somewhat lower. In a population-based vali- 11% of patients with a first forearm fracture sustained a sec- dation study of humeral fractures in the Danish National ond forearm fracture within 10 years [22]. Given a similar Patient Registry (DNPR) in 2017–2020 [18], the PPV was risk of subsequent forearm fractures in Norway, we would 89.3%, whereas another study of all orthopedic diagnoses exclude approximately 1.5% of incident forearm fractures in the DNPR in a 2-week period in 2006 reported a PPV of when using a 6-month washout period. Hence, the sensitiv- 86% [19]. In a validation study of 1000 hip fracture patients ity after 6-month washout is likely to be 1.5% lower than the in Norway, the PPV was 98.2% [9]. Likewise, a study from estimated 90.4% that we reported in this study. On the other the Finish patient register found that 98.1% of hip fractures hand, in the current study, nine of the patients not found had a hip fracture diagnosis [20]. The lower validity of in PAS (1.4%) were treated in other hospitals and would non-hip fractures probably reflects that other fractures are probably be captured if using data from the NPR. Although 1 3 Archives of Osteoporosis (2023) 18:111 Page 7 of 9 111 at the University of Oslo and at the hospitals. The data collected in data from PAS is said to be transferred unabridged to the this study were not available to other investigators or data managers NPR, some minor data processing seems to take place when than those included in project description (listed in the data protection combining data from all Norwegian hospitals. Future studies impact assessment). The investigators at the hospitals who collected should compare hospital data with data from NPR (collected data were only able to upload data and not access any other data in the safe research platform at the University of Oslo. Access to data from hospitals) on an individual level to obtain direct infor- for other investigators can be obtained after approval from Norwegian mation about the data from NPR. Directorate of Health, evaluation by the Norwegian Centre for Research A strength of this study is that we reviewed more than Data, and approvals from the data protection officers at the hospitals 8000 medical records and X-ray and CT reports from five and the University of Oslo. hospitals in Norway in 2015 and estimated both sensitivity Declarations and PPVs. We included data from only five of 44 hospitals treating forearm fractures in Norway. Still, the data are likely Ethics approval All procedures performed in studies involving human to be representative for the country as the included hospi- participants were in accordance with the ethical standards of the insti- tals vary in size and are located in all four health regions tutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. of Norway. Changes in classification systems and the use The study was evaluated by the Norwegian Centre for Research Data of more sensitive diagnostic methods over time (diagnostic (project number 587591). After performing a data protection impact drift) may hamper the interpretation of secular trends in inci- assessment (DPIA), the project was approved by the data protection dence rates. A limitation is that we only validated data from officer at the University of Oslo as pursuant to the General Data Pro- tective Regulation EU 2016/679. Exemption from consent for qual- the year 2015 and were unable to investigate any changes in ity assurance was obtained from the Norwegian Directorate of Health medical coding practice over time. It would have been useful (ref:19/3103-4). The project was also approved by the data protection to have data from three or more years, but due to methodo- officer at all hospitals. All data was uploaded directly from the patient logic challenges and limited resources this was not possible. administrative systems (PAS) and from the x-ray/CT reports into the research platform “Services for Sensitive Data” (TSD), which meets the requirements of Norwegian law regarding safe handling and stor- ing of sensitive data. All data merging, handling, and analyses were performed within this safe system. Conclusion Conflicts of interest FF reports speaking fees from UCB and Amgen; The data from the patient administrative system from five TTB reports speaker fees from UCB, Amgen, Roche, and Pharma hospitals in Norway showed a sensitivity of approximately Prim. Advisory board for UCB; JEG reports lecture fees from Or- 90% and the PPV varied between 74 and 95% depending tomedic Norway and LINK Norway; EMA reports advisory board for UCB and Amgen; RMJ reports consulting fees from Norwegian Direc- on the algorithm used to define incident forearm fractures. torate of Health, Norwegian Diabetes Association, Northern Norway The results are valuable measures of the validity of forearm Regional Health Authority and lecture fees from Novo Nordisk, Eli- fracture diagnoses obtained from administrative patient reg- Lilly, Astra-Zeneca, Sanofi; EFE reports consultant fees from UCB, isters. Depending on local coding practices and treatment Amgen, Gilead, Pharma Medico and honoraria from Amgen, UCB, and Novo Nordisk. The other authors declared that they have no con- pathways, our estimates are relevant for other fracture diag- flict of interest. noses and research based on register data in other countries. Open Access This article is licensed under a Creative Commons Attri- Acknowledgements We would like to acknowledge the contribu- bution 4.0 International License, which permits use, sharing, adapta- tion of Merete Finjarn, Anine Johansen, Cecilie Wagenheim, Ingrid tion, distribution and reproduction in any medium or format, as long Mikaelsen, Ann Kristin Schanche, and Olav Wetteland in data as you give appropriate credit to the original author(s) and the source, collection. provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are Author contribution CD, TKO, LBS, ÅB, and TTB were involved in included in the article's Creative Commons licence, unless indicated the the conception and design of the study. CD, TKO, LBS, TTB, ÅB, otherwise in a credit line to the material. If material is not included in JMS, CA, TB, TW, GH, FIN, AKH, RMJ, EF, GP, AAR, JK, EFE, the article's Creative Commons licence and your intended use is not LN, FF, WF, EMA, and JEG were involved in the acquisition of data, permitted by statutory regulation or exceeds the permitted use, you will or analysis and interpretation of data. The conventional X-ray and CT need to obtain permission directly from the copyright holder. To view a reports were reviewed independently by CD and TKO. In cases with copy of this licence, visit http://cr eativ ecommons. or g/licen ses/ b y/4.0/ . uncertain diagnoses, clinicians (ÅB and TTB) and orthopaedic sur- geons (JEG, LBS, and FF) were consulted. TKO, CD, and ÅB drafted the manuscript and the other authors revised it critically. All authors have approved the final version of the manuscript. References Funding Open access funding provided by University of Oslo (incl Oslo University Hospital) 1. Thygesen LC, Ersboll AK (2014) When the entire population is the sample: strengths and limitations in register-based epide- Data availability Availability of data depend on permissions from the miology. Eur J Epidemiol 29:551–558. https:// doi. org/ 10. 1007/ Norwegian Directorate of Health, evaluation by the Norwegian Cen- s10654- 013- 9873-0 tre for Research Data, and approval from the data protection officer 1 3 111 Page 8 of 9 Archives of Osteoporosis (2023) 18:111 2. Schmidt M, Schmidt SA, Sandegaard JL, Ehrenstein V, Pedersen 12. Brophy S, John G, Evans E, Lyons RA (2006) Methodological L, Sorensen HT (2015) The Danish National Patient Registry: a issues in the identification of hip fractures using routine hospital review of content, data quality, and research potential. Clin Epi- data: a database study. Osteoporos Int 17:405–409. https://doi. or g/ demiol 7:449–490. https:// doi. org/ 10. 2147/ CLEP. S9112510. 1007/ s00198- 005- 2038-6 3. Arts DG, De Keizer NF, Scheffer GJ (2002) Defining and improv - 13. Langley J, Stephenson S, Cryer C, Borman B (2002) Traps for the ing data quality in medical registries: a literature review, case unwary in estimating person based injury incidence using hospital study, and generic framework. J Am Med Inform Assoc 9:600– discharge data. Inj Prev 8:332–337. https://doi. or g/10. 1136/ ip.8. 4. 332 611. https:// doi. org/ 10. 1197/ jamia. m1087 14. Boufous S, Williamson A (2003) Reporting of the incidence 4. Ehrenstein V, Petersen I, Smeeth L, Jick SS, Benchimol EI, Lud- of hospitalised injuries: numerator issues. Inj Prev 9:370–375. vigsson JF, Sorensen HT (2016) Helping everyone do better: a https:// doi. org/ 10. 1136/ ip.9. 4. 370 call for validation studies of routinely recorded health data. Clin 15. Laugesen K, Ludvigsson JF, Schmidt M, Gissler M, Valdimarsdottir Epidemiol 8:49–51. https:// doi. org/ 10. 2147/ CLEP. S1044 48 UA, Lunde A, Sorensen HT (2021) Nordic health registry-based 5. Lofthus CM, Frihagen F, Meyer HE, Nordsletten L, Melhuus K, research: a review of health care systems and key registries. Clin Falch JA (2008) Epidemiology of distal forearm fractures in Oslo. Epidemiol 13:533–554. https:// doi. org/ 10. 2147/ CLEP. S3149 59 Norway Osteoporos Int 19:781–786. h t t p s : / / d o i . o r g / 1 0 . 1 0 0 7 / 16. Varmdal T, Bakken IJ, Janszky I, Wethal T, Ellekjaer H, Rohweder s00198- 007- 0499-5 G, Fjaertoft H, Ebbing M, Bonaa KH (2016) Comparison of the 6. Kvernmo HD, Otterdal P, Balteskard L (2017) Treatment of wrist validity of stroke diagnoses in a medical quality register and an fractures 2009–14. Tidsskr Nor Laegeforen 137. https:// doi. org/ administrative health register. Scand J Public Health 44:143–149. 10. 4045/ tidss kr. 17. 0065https:// doi. org/ 10. 1177/ 14034 94815 621641 7. Omsland TK, Holvik K, Meyer HE, Center JR, Emaus N, Tell GS, 17. Rosengren BE, Karlsson M, Petersson I, Englund M (2015) The Schei B, Tverdal A, Gjesdal CG, Grimnes G, Forsmo S, Eisman 21st-century landscape of adult fractures: cohort study of a com- JA, Sogaard AJ (2012) Hip fractures in Norway 1999–2008: time plete adult regional population. J Bone Miner Res 30:535–542. trends in total incidence and second hip fracture rates: a NORE-https:// doi. org/ 10. 1002/ jbmr. 2370 POS study. Eur J Epidemiol 27:807–814. https://doi. or g/10. 1007/ 18. Karimi D, Houkjaer L, Gundtoft P, Brorson S, Viberg B (2023) s10654- 012- 9711-9 Positive predictive value of humeral fractures in the Danish 8. Sogaard AJ, Holvik K, Meyer HE, Tell GS, Gjesdal CG, Emaus N, National Patient Registry. Dan Med J 70(4):A10220612 Grimnes G, Schei B, Forsmo S, Omsland TK (2016) Continued decline 19. Lass P, Lilholt J, Thomsen L, Lundbye-Christensen S, Enevoldsen in hip fracture incidence in Norway: a NOREPOS study. Osteoporos H, Simonsen OH (2006) The quality of diagnosis and procedure Int 27:2217–2222. https:// doi. org/ 10. 1007/ s00198- 016- 3516-8 coding in Orthopaedic surgery Northern Jutland. Ugeskr Laeger 9. Hoiberg MP, Gram J, Hermann P, Brixen K, Haugeberg G (2014) 168:4212–4215 The incidence of hip fractures in Norway -accuracy of the national 20. Sund R, Nurmi-Luthje I, Luthje P, Tanninen S, Narinen A, Kes- Norwegian patient registry. BMC Musculoskelet Disord 15:372. kimaki I (2007) Comparing properties of audit data and routinely https:// doi. org/ 10. 1186/ 1471- 2474- 15- 372 collected register data in case of performance assessment of hip 10. Dahl C, Ohm E, Solbakken SM, Anwar N, Holvik K, Madsen C, fracture treatment in Finland. Methods Inf Med 46:558–566. Frihagen F, Bjornerem A, Igland Nissen F, Solberg LB, Omsland https:// doi. org/ 10. 1160/ me0382 TK (2023) Forearm fractures - are we counting them all? An 21. Fox MP, Lash TL, Bodnar LM (2020) Common misconceptions attempt to identify and include the missing fractures treated in about validation studies. Int J Epidemiol 49:1392–1396. https:// primary care. Scand J Prim Health Care 1–10. https:// doi. org/ 10. doi. org/ 10. 1093/ ije/ dyaa0 90 1080/ 02813 432. 2023. 22310 28 22. Kanis JA, Johansson H, Oden A, Harvey NC, Gudnason V, 11. Andreasen C, Solberg LB, Basso T, Borgen TT, Dahl C, Wisloff T, Sanders KM, Sigurdsson G, Siggeirsdottir K, Fitzpatrick LA, Hagen G, Apalset EM, Gjertsen JE, Figved W, Hubschle LM, Stutzer Borgstrom F, McCloskey EV (2018) Characteristics of recurrent JM, Elvenes J, Joakimsen RM, Syversen U, Eriksen EF, Nordsletten fractures. Osteoporos Int 29:1747–1757. https://doi. or g/10. 1007/ L, Frihagen F, Omsland TK, Bjornerem A (2018) Effect of a fracture s00198- 018- 4502-0 liaison service on the rate of subsequent fracture among patients with a fragility fracture in the Norwegian Capture the Fracture Initiative Publisher's note Springer Nature remains neutral with regard to (NoFRACT): a trial protocol. JAMA Netw Open 1:e185701. https:// jurisdictional claims in published maps and institutional affiliations. doi. org/ 10. 1001/ jaman etwor kopen. 2018. 5701 1 3 Archives of Osteoporosis (2023) 18:111 Page 9 of 9 111 Authors and Affiliations 1 2 3,4,5 6 4,7 Tone Kristin Omsland  · Lene B. Solberg  · Åshild Bjørnerem  · Tove T. Borgen  · Camilla Andreasen  · 8 9 10 11,12 13,14 15,16 Torbjørn Wisløff  · Gunhild Hagen  · Trude Basso  · Jan‑Erik Gjertsen  · Ellen M. Apalset  · Wender Figved  · 17 3,4,7 4,7 4,18 1 1 Jens M. Stutzer  · Frida I. Nissen  · Ann K. Hansen  · Ragnar M. Joakimsen  · Elisa Figari  · Geoffrey Peel  · 1 4 19,20 2,16 16,21 1 Ali A. Rashid  · Jashar Khoshkhabari  · Erik F. Eriksen  · Lars Nordsletten  · Frede Frihagen  · Cecilie Dahl * Tone Kristin Omsland Department of Orthopaedic Surgery, Haukeland University t.k.omsland@medisin.uio.no Hospital, Bergen, Norway Department of Clinical Medicine, University of Bergen, Department of Community Medicine and Global Health, Bergen, Norway Institute of Health and Society, University of Oslo, Blindern, Po box 1130, 0318 Oslo, Norway Bergen Group of Epidemiology and Biomarkers in Rheumatic Disease, Department of Rheumatology, Division of Orthopaedic Surgery, Oslo University Hospital, Haukeland University Hospital, Bergen, Norway Oslo, Norway Department of Global Public Health and Primary Care, Department of Obstetrics and Gynaecology, University University of Bergen, Bergen, Norway Hospital of North Norway, Tromsø, Norway Department of Orthopaedic Surgery, Vestre Viken Hospital Department of Clinical Medicine, The Arctic University Trust, Bærum Hospital, Gjettum, Norway of Norway, Tromsø, Norway Institute of Clinical Medicine, University of Oslo, Oslo, Norwegian Research Centre for Women’s Health, Oslo Norway University Hospital, Oslo, Norway Department of Orthopaedic Surgery, Møre and Romsdal Department of Rheumatology, Vestre Viken Hospital Trust, Hospital Trust, Hospital of Molde, Molde, Norway Drammen Hospital, Drammen, Norway Department of Medicine, University Hospital of North Department of Orthopaedic Surgery, University Hospital Norway, Tromsø, Norway of North Norway, Tromsø, Norway Pilestredet Park Specialist Centre, Oslo, Norway Health Services Research Unit, Akershus University Hospital, Lørenskog, Norway Faculty of Dentistry, University of Oslo, Oslo, Norway 9 21 Department of Health Services, Norwegian Institute Department of Orthopaedic Surgery, Østfold Hospital Trust, of Public Health, Oslo, Norway Grålum, Norway Department of Orthopaedic Surgery, St. Olavs University Hospital, Trondheim, Norway 1 3

Journal

Archives of Osteoporosis – Springer Journals

Published: Aug 24, 2023

Keywords: Validation; Forearm fracture; Sensitivity; Positive predictive value

There are no references for this article.