Validity of the Diagnosis of Acute Myocardial Infarction in Korean National Medical Health Insurance Claims Data: The Korean Heart Study (1)

The importance of valid data has increased due to the high impact of cardiovascular diseases on mortality and morbidity. Medical insurance claims (MIC) data are one of the largest sources of outcome data in the form of International Classification of Diseases (ICD) codes.1) Because enrollment is mandatory for all Koreans, the Korean national health insurance system covers nearly all of the country's population (96.3% in 2007) with the exception of some medical assistance beneficiaries. If the validity of MIC data is acceptable for research purposes, it can be used as a comprehensive and economical source of outcome information.

International Classification of Diseases codes have been used in research for years; however, accuracy issues have been raised in recent studies. Questions regarding validity have been suggested as a result of ambiguities and inconsistencies in ICD-coded diagnoses.2) Coding errors are also regarded as an issue with regards to data reliability.3) Despite these concerns, the reported accuracy of ICD codes appeared to be acceptable in previous reports. In Korea, the accuracy rate of ICD codes for cerebrovascular disease was 83% for 425 men, according to insurance claims.4) However, the accuracy of ICD codes for cardiovascular disease is not well-known. Particularly, acute myocardial infarction (AMI), a fatal disease, has doubled in the past 11 years (1997-2007).5) Therefore, in this study we evaluated the validity of ICD codes for AMI according to Korean national MIC data.

Among the 673 mailed questionnaires, we received 482 responses (71.6%). The most common respondents were resident trainees in internal medicine departments (Table 1). Among the respondents, 95.0% were working at hospitals affiliated with medical colleges. A large number of the cases (75.9%) were from hospitals located in the Seoul area (Table 2). AMI, defined as I21, I210-I219 on the MIC ICD-10 codes, was reported in 107 cases (93 men, 26-73 years of age). Mean age (standard deviation) of the patients with AMI was 49.5 (10.1) years. Unspecified forms of AMI (I219) accounted for 48.6% (Table 3). The accuracy of AMI was 71.4%, according to the WHO criteria (1997-2000, n=24, kappa=0.46) and 73.1% according to ESC/ACC criteria (2001-2007, n=83, kappa=0.74), after excluding unavailable cases (Table 4 and 5).

Table 1
Characteristics of the respondents in the event validation project

• Click for larger image

Table 2
Number of cases based on the characteristics of the hospitals participating in the event validation project (n=482)

• Click for larger image

Table 3
Subgroup of acute myocardial infarction (I21-) based on ICD-10 code frequency

• Click for larger image

Table 4
Accuracy of ICD codes for acute myocardial infarction using the WHO criteria in medical insurance claims data among ICD-10 codes I21- (1997-2000)

• Click for larger image

Table 5
Accuracy of ICD codes for acute myocardial infarction using ESC/ACC criteria for medical insurance claim data with an ICD-10 code of I21- (2001-2007)

• Click for larger image

An age of 50 years or older was the only factor related to AMI inaccuracy (odds ratio, 4.6; 95% confidence interval, 1.2-17.7, p=0.0270) after adjusting for gender, hospital location, respondents, ICD code recorder, and respondents who made the diagnosis related to the ICD codes (Table 6).

Table 6
Potential variables related to the inaccuracy of ICD codes for acute myocardial infarction among patients diagnosed since January 2001 by ESC/ACC criteria (n=83)

• Click for larger image

The accuracy and reliability of AMI diagnoses in the Korean Heart Study was evaluated to assess validity. The accuracy of AMI was 71.4% (WHO criteria, 1997-2000) to 73.1% (ESC/ACC criteria, 2001-2007). Reliability between each of the AMI criteria and ICD codes was good to fair in terms of the kappa values (0.46 or 0.74) according to the suggestion of Fleiss that values >0.75 indicate excellent reliability, those 0.40-0.75 indicate fair to good, and <0.40 indicate poor.11)

The importance of MIC data has been growing, particularly in epidemiological studies due to the huge population included.12) However, issues regarding the reliability of MIC data have been raised continuously. Hospitals may record codes incorrectly in some cases due to reimbursement issues.13) Admission duration and the presence of complex co-morbidities also add to confusion in understanding the discharge data.14), 15) There are several known reasons associated with coding errors, including misspecification, miscoding, and re-sequencing or substitution of the major diagnosis with a secondary diagnosis for the purpose of reimbursement.16) The ICD codes alone include ambiguities and inconsistencies.2)

Although the ICD-coding system has certain limitations, the usefulness of the coding has also been reported, and it has been used in various studies to date. In an intervention study, the reliability of ICD-10 coding was assessed among three groups of students, staff, and coding specialists, and concordance results were fair to moderate.17)

Studies on the reliability of ICD codes in the field of cardiovascular research have been reported. Validity of ischemic heart disease coding was evaluated in Canada where physician service claims and discharge records using the ICD-9 code were concordant in 69.3% of AMI cases.18)

In the Republic of Korea, the National Insurance System covers nearly all of the population. Claims and payments are handled by an online electronic system. Some conflicts occur with the payment system between the insurance payer and hospitals, so we could not exclude the possibility of systematic miscoding; however, efforts to reduce coding errors have been introduced by the payer.

Several findings regarding stroke and diabetes have been obtained among reports on the accuracy of ICD codes in Korea. Park et al.4) reported an 83% accuracy for a cerebrovascular diseases diagnosis based on ICD codes in 425 cases among Korean men in 2000. In the National Diagnosis Related Group Validation Study, the accuracy of mixed diseases was 79.2%.13) In the diabetes epidemiology report in Korea, the accuracy of admitted cases was 87.2%, and that of outpatients was 72.3%.19) These results for other diseases seem to be comparable with our AMI accuracy results. It has been suggested that the accuracy of various mixtures of many diagnoses cannot be summarized or generalized due to the unique clinical situations and specific conditions of each disease.20), 21) In an evaluation of the appropriateness of AMI diagnostic coding in Boston, 41.7% of the cases failed to qualify in teaching hospitals because of inappropriate coding results, whereas the number was only 9.1% in non-teaching facilities.22) Many of our participants were from teaching hospitals, and the coding results were acceptable. However, the differences in qualification status among hospitals may not be comparable among countries due to huge variations in medical care systems.

In this study, we used WHO and ESC/ACC criteria to assess the AMI diagnoses. ESC/ACC criteria were introduced in 2000 and include creatine kinase MB.8) The WHO criteria, which have also been widely used, was the core method in the Multinational Monitoring of Trends and Determinants in Cardiovascular Disease project.9), 10)

Several limitations of this study need to be acknowledged. We could not collect any information regarding the reasons for coding errors or on the various coding pathways in multiple hospitals. For example, the possibility of accidental AMI coding rather than sudden death cannot be excluded. Questionnaires were sent instead of a direct review of the medical records. The number of cardiologists or staff who directly input the ICD codes was small, and the number of no responses was large. Many of the participant hospitals were located in city areas and were teaching hospitals, so we could not expand our results to all hospitals or clinics. We evaluated all of the AMI cases using our approach, but they were not recruited randomly. We had a relatively small number of subjects, particularly of women (n=14, 13.1% of the 107 AMI cases). The reliability of the AMI diagnosis was fair to good regardless of these potential differences among our samples. However, we cannot exclude the possibility of selection bias, because the participants were spontaneous examinees of health examination centers, although data from similar health examination centers has been used in previous cardiology studies.23), 24)

In conclusion, the accuracy of the AMI diagnosis using ICD-10 codes in Korean MIC data was >70%, and reliability was good to fair. An age of 50 years or older was the only factor related to inaccuracy of AMI diagnoses; therefore, more attention is required when re-coding ICD data of older patients. Further studies should be performed with regard to the validity of AMI MIC data and other cardiovascular diseases using a larger group.