MONICA Manual, Part IV:Event Registration

Section 4: ECG coding quality assurance methods

November 1990

This section provides a description of the quality assurance methods to be applied to the ECG coding in the WHO MONICA Project.

Contents

• Introduction
• 1. Guidelines for Internal Quality Control
• 2. External Quality Control
• Annex

• Copyright notice
• Manual acknowledgements
• Document identification:
  • URL:http://www.ktl.fi/publications/monica/manual/part4/iv-4.htm
  • URN:NBN:fi-fe19981159

Queries/comments on this section should be addressed to:

Earlier versions

• WHO MONICA Project. MONICA Manual. CVD/MNC/Version 1.1, December 1986; Sect. 9.
• WHO MONICA Project. MONICA Manual. Geneva: World Health Organization, Cardiovascular Diseases Unit; November 1990; Part IV, Sect. 3.

Introduction

The reliability of Minnesota coding within the WHO MONICA Project is achieved by internal and external quality control measures. Internal quality control includes all measures applied within one MONICA Collaborating Centre aimed at uniform coding by different coders and consistency in using the code system. External quality control aims at standardization of Minnesota coding between the MONICA Collaborating Centres and is carried out by the MONICA Quality Control Centre for ECG Coding, Budapest.

1. Guidelines for internal quality control

1. Each ECG is coded preferably by two independent coders who have previously been trained in the use of the Minnesota code system, unaware of the results of each other. The coders should also, ideally, be unaware of the clinical diagnosis made on the case and the result of any cardiac enzyme determinations. Research has shown that coders like to agree with the clinicians involved with the case, and knowledge of the clinical diagnoses and interpretations of the electrocardiogram biases the coding towards finding more pathological abnormality than is actually present. The codes established by the two coders separately are compared. If only one coder is available, all ECGs should be re-coded after a suitable interval.
2. In the case of a discrepancy between the two coders, a third senior coder or supervisor re-codes the ECG without knowing the codes of the two other coders. After having established his/her own code, the three coders discuss the case and come to a final decision.
3. In order to check intra-coder consistency, a random sample of 10% of the ECGs is re-cycled every year. The results of the two readings are compared and the causes of discrepancies carefully analysed with the involvement of the supervisor.
4. In the case of more than one Reporting Unit (RU) within one MCC, 20 ECGs are coded by the supervisors of all RUs every half a year. The two most experienced supervisor coders previously establish their reference codes. After all the RUs' supervisors have coded the ECGs, they discuss the discrepancies and come to a consensus on the codes. The same 20 ECGs are then coded by all the coders within one RU and the codes discussed by the coders and the supervisor coder.

2. External quality control

1. Training seminars for coders are held regularly by the Quality Control Centre for ECG Coding according to the needs and requests of the MCCs. The aim is to give an opportunity to all MCCs to participate in such courses.

   The Training Seminars are led by the head of the Quality Control Centre. They are open to a limited number of participants (preferably not more than 10 to 12 at each occasion) chosen from those who give notice about their willingness to participate. The results of the previous test exercises are also taken into consideration when choosing from candidates. All participants should have some experience in Minnesota Coding.

   The topic of the Seminars are those items of the Minnesota code system which are relevant for the MONICA Project. These are 1, 4, 5, 9.2 and suppressing codes. Both technical and theoretical problems are discussed and great emphasis is placed on actual coding.

2. A test set of ECGs is collected every year by the Quality Control Centre for ECG Coding. This set is reproduced and dispatched to the MCCs by the MONICA Management Centre. The MCCs establish their Minnesota codes of the ECGs in the set and send the codes back to the Quality Control Centre.

   The "reference codes" of the ECG set are established by the Quality Control Centre in cooperation with the Minnesota Coding Laboratory, and/or a panel of coders of proven ability. After receipt of the codes from the MCCs, a computer analysis is carried out - the underlying equations are shown in the appendix. This is followed by a review of each individual set of codes in comparison to the reference codes. A short report of the performance of each MCC is prepared and sent to the respective MONICA Collaborating Centre.

   After receipt of the results from each MCC the performance of the individual Centres is compared. For this purpose formula "B" (see Annex) is used, which compares the numbered all discrepancies to the total numbered in ECGs with reference codes. The average of the mistake rates of the three lead groups for Q-codes is taken and separately the average of averages of the mistake rates for ST and T codes. Finally, the average of these two values is calculated.

   Agreement between the Centre codes and the reference codes is considered good if the calculated value, i.e. the percentage of discrepancy, is less than 30%, and acceptable if it is between 30% and 50%. In those cases with disagreement of more than 50% improvement in Minnesota coding is desirable.

3. The quality control of the MONICA Quality Control Centre itself is provided, where possible, through the Minnesota Coding Laboratory in Minneapolis. If this cannot be maintained, it will be done through a panel of external coders of proven ability, or Task Force, who will help to provide a pool of high quality electrocardiograms with agreed codes. The panel will help to arbitrate in case of disagreement between the Quality Control Centre and the individual MCCs.

Annex: Formulas used in ECG quality control

Values were taken from these equations:

(Using code 1)