WWW-publications from the WHO MONICA Project

Quality Assessment of Coronary Event Registration Data in the WHO MONICA Project

January 1999

Markku Mähönen1, Hanna Tolonen1, Kari Kuulasmaa1, Hugh Tunstall-Pedoe2 and Philippe Amouyel3 for the WHO MONICA Project4

1 MONICA Data Centre, National Public Health Institute, Helsinki, Finland
2 Cardiovascular Epidemiology Unit, University of Dundee, Ninewells Hospital, Dundee, Scotland
3 Faculty of Medicine and Pasteur Institute of Lille, Lille, France
4 Annex: Sites and key personnel of the WHO MONICA Project


© Copyright World Health Organization (WHO) and the WHO MONICA Project investigators 1999. All rights reserved.

This document includes the main findings of unpublished reports: :


Acknowledgements

Thanks are due to Alun Evans and Hermann Wolf who commented on the text.

The MONICA Centres are funded predominantly by regional and national governments, research councils, and research charities. Coordination is the responsibility of the World Health Organization (WHO), assisted by local fund raising for congresses and workshops. WHO also supports the MONICA Data Centre (MDC) in Helsinki. Not covered by this general description is the ongoing generous support of the MDC by the National Public Health Institute of Finland, and a contribution to WHO from the National Heart, Lung, and Blood Institute, National Institutes of Health, Bethesda, Maryland, USA for support of the MDC and the Quality Control Centre for Event Registration in Dundee. The completion of the MONICA Project is generously assisted through a Concerted Action Grant from the European Community. Likewise appreciated are grants from ASTRA Hässle AB, Sweden, Hoechst AG, Germany, Hoffmann-La Roche AG, Switzerland, the Institut de Recherches Internationales Servier (IRIS), France, and Merck & Co. Inc., New Jersey, USA, to support data analysis and preparation of publications.


Contents

1. Introduction

To achieve the goals set for the MONICA Project (1), the quality of the acute coronary event and coronary death registers is of paramount importance. The key issues are:

This report addresses these issues. It follows the approach of the first quality assessment report (5), the second report (6) and the quality assessment in the publication "Myocardial infarction and coronary deaths in the WHO MONICA Project" (2). The current report covers the full MONICA event registration period, including the years covered by the earlier quality assessment reports.

In the specification of the calculations for this quality assessment report the names of the data items in the Core Data Transfer Format - Coronary Events (3) have been used. The terminology used in this report was developed for MONICA event registration in the MONICA Manual (3), with later refinements in the collaborative publications (2). It is important to note that the reliability of event rates depends also on the quality of demographic data.

2. Material and Methods

2.1 Populations

The report considers the Reporting Unit Aggregates (RUAs) which are seen as potential candidates for units of analysis of the MONICA coronary event data. The RUAs, their abbreviations and Reporting Units (RUs) are listed in Table 1. Some of the RUAs have several versions distinguished by suffix ´a´ and ´b´. Different combinations of Reporting Units (RUs) may be used for analyses concerning coronary events only, and others for analyses involving both coronary event and risk factor data. The reason for the distinction is that some RUs of some RUAs were not included in every risk factor survey. Therefore, the different RUAs from AUS-PER, GER-BER, GER-BRE, GER-EGE, ICE-ICE, RUS-MOI and RUS-NOC the RUAs may include the same RUs. Furthermore, the combination (GER-AUG) of GER-AUR and GER-AUU has also been considered. Altogether 49 RUAs are considered.

2.2 Periods of events considered

Only full calendar years of registration are considered. Individual years for which data were received in the MONICA Data Centre (MDC), were excluded if:

The years considered for each RUA are shown in Table 1.

2.3 Age and sex

The quality assessment concerns the age group 25-64. The age was calculated in full years of the date of onset, except for mortality comparison, where date of death was used (Table 6). When calculating the age, day 99 was interpreted as 15 and day/month 99/99 as 30/06. No age standardization was used. Data for men and women were combined for the analysis.

2.4 Other inclusion criteria for the data

Individual records have not been included in the analysis if:

Otherwise, all data available in the MDC were used in the analysis, regardless of their quality. The total number of coronary events available in the MDC are shown by calendar year in Table 2.

2.5 Sources of information

The report is based on the data which the MDC has received from the MCCs on coronary events (Form 01, see MONICA Manual (3)), on mortality (Form C (ICD-8) and Form E (ICD-9), see MONICA Manual (4)), and on other information received from the MCCs.

3. Serial number inventory and routine data checking status

Table 3 shows a summary of serial number inventory, which is based on a linkage of the coronary event data and the serial number inventory data (3) received by the MDC. Its purpose is to check that the MDC database has exactly those records which it should have according to the MCC. Ideally, all entries of the last four columns should be zero. Otherwise there is a possibility that some records have been lost or duplicated  from the time of the data's ascertainment. There are a large number of discrepancies in the serial number inventory for AUS-PER, BEL-LUX, HUN-BUD, HUN-PEC, LTU-KAU and NEZ-AUC.

When the coronary event data were received in the MDC they were checked routinely for the constraints specified in Appendix 1. All violations of the constraints were reported to the MCC for their correction or elucidation. Data values outside the constraint limits were acceptable, but the MCC had to check that the values were not unusual due to data errors. The MCCs were only asked to revise data if they were incorrect. The current unresolved constraint violations are shown in Appendix 2 and summarized by calendar year in Table 4. There is only a small number of unresolved constraint violations for any RUA. The number is more than ten for BEL-CHA, BEL-GHE, FIN-NKA, GER-EGE, GER-RHN, HUN-BUD, and SWE-GOT.

The data are also checked periodically in the MDC for events which have been registered more than once. Table 5 shows the number of suspected duplicate registrations which have neither been confirmed as duplicate registrations nor as separate events. There is only a small number of unresolved suspected duplicate registrations for any RUA. The number is more than ten for BEL-CHA, GER-AUG and GER-EGE.

The data with unresolved constraint violations or duplicate suspicions have been included in the analysis for this report.

4. Coverage of registration

4.1 Fatal events

According to the MONICA Manual (3),  suspect coronary events are collected from death certificates, hospital medical records and  community health services. All deaths with the underlying cause of death coded to 410-414 must be registered even if not found to be a coronary case on examination.

Data should have been submitted to the MDC on:

To be certain that all coronary deaths are registered, in each RUA the number of fatal events registered with a CHD death certificate diagnosis (CLIND1-3 coded 410-414) should be greater than the number of coronary deaths reported in routine (official) mortality statistics (greater because CHD deaths in routine statistics include only those having an ICD code 410-414 as the underlying cause of death). The number of these fatal events registered in MONICA is the numerator and the number of CHD deaths in the routine statistics is the denominator in the CHD/CHD ratio shown in   Table 6 . The ratio is less than one for at least one year in AUS-PER (1988); BEL-CHA (1983-1987); BEL-GHE (1983-1991); BEL-LUX (1985-1987); CHN-BEI (1984-1986); DEN-GLO (1987-1988, 1990-1991); GER-AUR (1985-1986, 1994); GER-AUU (1986); GER-BREa (1992);  GER-RHN (1984-1987); HUN-BUD (1982-1984, 1986, 1988); HUN-PEC (1984-1989); ITA-BRI (1987); NEZ-AUC (1983-1988, 1990-1992); POL-WAR (1988-1989, 1991-1994); RUS-MOC (1990-1991, 1993); RUS-MOIa (1987-1988, 1990, 1992); RUS-MOIb (1988, 1990, 1992); RUS-NOCa (1989, 1991, 1993); RUS-NOIa (1987); UNK-BEL (1986, 1988, 1991-1992).

In some RUAs the access to the death certificates or the final official code of death is restricted, which may be the reason why some deaths coded as CHD are not registered. Therefore, we also compared the number of all registered fatal events to the number of CHD deaths by routine statistics (Total/CHD ratio: the number of events in MONICA register is the numerator and the number of deaths in the routine mortality statistics is the denominator) (Table 6) to see if the restricted availability of routine code of death is compensated by other registrations. In many MCCs, the breadth of registration is large, although the ratio of MONICA CHD deaths/routine statistics CHD deaths is below 1.0. Total/CHD ratio is less than one in at least one year in BEL-LUX (1985); CHN-BEI (1984-1986); GER-AUR (1994); GER-RHN (1987); HUN-BUD (1986); HUN-PEC (1985-1989); RUS-MOC (1990-1991); RUS-MOIa (1987-1990, 1992); RUS-MOIb (1988,1990, 1992); RUS-NOCa (1989,1991, 1993); SWE-GOT (1985); UNK-BEL (1986, 1991-1992).

In many of these populations the discrepancy is small, and could be explained by errors in the routine mortality statistics, or a different way of determining the year of death in the register and in the routine statistics (UK and Australia).

Especially worrying is the situation in MCCs where low CHD/CHD ratio is insufficiently compensated for by broader total fatal registration or otherwise explained ( see Section 10 for specific comments on each RUA):

The comparison with routine mortality statistics was not possible because of missing or incomplete routine mortality statistics data in the following RUAs:

There is a clear error in routine mortality statistics data in 1993 in RUS-NOCa and RUS-NOIa.

4.2 Non-fatal events

According to the MONICA Manual (3), the suspect coronary events are collected from hospital medical records and from community health services. All clinically diagnosed myocardial infarctions should be registered.

Data should have been submitted to the MDC on:

The case identification should be wide enough to catch practically all clinically diagnosed myocardial infarctions which would get the diagnostic category DIACAT 1 (definite MI) in MONICA. Unlike for fatal events, there are no independent data sources against which the coverage of the non-fatal registrations could be checked. However, there are two indicators in the data which may reflect problems in the coverage of non-fatal registrations:

Proportion of non-fatal events with DIACAT 2 or 4:

In order to capture all events with MONICA DIACAT 1, the selection of clinical diagnoses used for registration will in practice be such that it will also produce a significant proportion of DIACAT 2 (possible MI) and 4 (no MI). Therefore, the case ascertainment for definite coronary events may be incomplete, if the proportion of events belonging to DIACAT 2 or 4 is small.

Table 7 shows the proportion of different diagnostic categories of non-fatal events. The proportion of possible and not AMIs is particularly small (below 20%) in the following RUAs: BEL-CHA (1983-1986); BEL- GHE (1983-1987, 1992); CHN-BEI (1985); DEN-GLO (1982); GER-RHN (1987); ICE-ICEa (1989-1990, 1992); ICE-ICEb (1990-1992); NEZ-AUC (1992); SWE-GOT (1992-1993); YUG-NOSa (1984-1987, 1990-1992). The reason for the low proportion in BEL-CHA, BEL-GHE and YUG-NOS is that DIACAT 4 events are not sent to the MDC. In ICE-ICEa and ICE-ICEb the reason may be related to low numbers and clinical practice patterns. In SWE-GOT data for the MI register are collected only from clinically suspected MIs with the clinical diagnoses of 410-411. (See Section 10 for specific comments for each RUA.)

Case fatality:

If there is under-ascertainment of non-fatal events, it may lead to an unusually high case fatality. Case fatality, calculated using MONICA definition I: (F1+F2+F9)/(F1+F2+F9+NF1), for men and women combined, is shown in Table 7.

Case-fatality was over 65% in CHN-BEI (1990-1991, 1993); HUN-BUD (1982-1987, 1989); POL-TAR (1984-1993); RUS-MOC (1991, 1993); RUS-MOIa (1988-1989, 1992-1993); RUS-MOIb (1992-1993); RUS-NOCa (1990, 1992); RUS-NOCb (1989, 1990); RUS-NOI (1987-1988, 1993).

The particularly high case fatalities in HUN-BUD and POL-TAR are explained in Section 10.

5. Availability and quality of data for diagnostic classification

5.1 Fatal events

The results of autopsy give valid data for the diagnostic classification of fatal events. The frequency of autopsies done is shown in Table 8.

There was a large variation in the frequency of autopsies between the MCCs. However, within each RUA the frequency of autopsies was fairly stable except in DEN-GLO, GER-EGEa, GER-EGEb and SWE-GOT where there was a drop in the frequency of autopsies, and YUG-NOS where there was first a rise in the frequency of autopsies and later a marked decline.

In events where autopsy was not carried out or the results of autopsy are not available (NECSUM= 8 or 9), other diagnostic information is very important for reliable classification of fatal events. For such fatal events, the availability of data on symptoms, the number of ECGs, the availability of data on enzymes, on previous CHD and previous MI are shown in Table 8.

For events with no autopsy, the RUAs in which the range in the yearly proportion of the availability of data was over 40% are shown in Table A.

Table A. Over 40% changes in the availability of data on symptoms, previous CHD and previous MI for fatal events with no autopsy
RUA symptoms previous CHD previous MI
low vs high trend low vs high trend low vs high trend
BEL-CHAa     1990 vs 1983 no 1990 vs 1983 no
BEL-GHEa         1991 vs 1985 down
DEN-GLOa 1987 vs 1990 no 1983 vs 1990 no    
GER-BREa 1992 vs 1985 down        
GER-BREb 1992 vs 1985 down        
GER-RHNa 1984 vs 1988 up 1984 vs 1987 up 1984 vs 1988 up
HUN-BUDa 1988 vs 1984 down 1983 vs 1986 no 1984 vs 1987 no
HUN-PECa 1984 vs 1989 up 1984 vs 1988 no 1984 vs 1989 no
LTU-KAUa 1992 vs 1987 no        
RUS-MOCa 1985 vs 1991 no     1985 vs 1991 up
RUS-MOIa 1985 vs 1989 no     1985 vs 1991 up
RUS-MOIb 1985 vs 1987 no     1985 vs 1991 up
SWE-GOTa 1993 vs 1984 down 1985 vs 1993 no 1994 vs 1985 no
YUG-NOSa     1984 vs 1994 up    

Changes in the proportion of DIACAT=9 events (Table 7) were notable in GER-EGEa and GER-EGEb (rising trend), GER-RHN (varies from 15% to 63%), HUN-PEC (varies from 0% to 38%) and SWE-GOT (a rise in 1990-1994).

5.2 Non-fatal events

The availability of the data has been very good in non-fatal events, as shown in Table 8. However, the availability of data from year to year varies in some RUAs. In DEN-GLO, 1982 deviates from other years (lower availability of data on ECGs); in FRA-TOU there is a drop in the availability of data on ECGs; in GER-EGEa, GER-EGEb, GER-RHN, HUN-BUD and HUN-PEC the availability of data on ECG varies from year to year; in RUS-NOCa, RUS-NOCb and RUS-NOI the availability of data on more than three ECGs drops to zero in 1988. (See also Section 10 for specific comments and clarification for each RUA.)

6. Availability of data on previous MI

The data item on previous MI is important when the first ever coronary events need to be identified. This concerns for example the calculation of incidence rates. The proportions of different categories of data on previous MI are given in the Table 9.

The proportion of indeterminate codes (4 (yes, from ECG) and 9 (insufficient data)) was over 30 % in:

In addition to these the proportion of indeterminate codes was over 10 % in:

It is important to note that these proportions are calculated for all registered events. If we consider only non-fatal definite MIs, the proportions of indeterminate events are very low in nearly all RUAs. On the other hand, for fatal events the proportions are higher than those shown in Table 9.

7. Results from test cases and ECG test coding

The MONICA Quality Control Centre for event registration in Dundee, Scotland, circulated test case histories to the MCCs. The test case histories firstly focused on training of coders and later on assessing the comparability between the MCCs.  The results of test case history exercises are summarized in Appendix 3. Likewise, the MONICA Quality Control Centre for ECG Coding in Budapest, Hungary, circulated test sets of ECGs for coding by the MCCs. The results of the ECG test coding are summarized in Appendix 4.

8. Internal quality control

It was suggested in the MONICA Manual (3) that the identification of events should start with a wider range of death certificate and hospital record diagnosis ICD codes. Event identification could proceed with a narrower range, if it had been established (and documented) in a pilot study (and reviews) that only a negligible number of definite coronary events were excluded by adopting a narrower range of ICD codes.

The results of the initial pilot studies and later repetitions are known for AUS-PER, ICE-ICE and UNK-GLA only.

9. Summary

Table 10 gives an overall summary of the data quality for each RUA by year. Data for the RUAs and years coded "2" have sufficient quality for calculation of attack rates and case fatality. Code "2*" indicates that   the data do not fulfil all the criteria for the good score "2" but the difference is minor, or there is an acceptable explanation for the deviant pattern. Code "1" indicates that a decision needs to be made separately for each publication. Score "0" is used for cases where the data are insufficiently complete for use in analyses.

After the previous quality assessment (6), many MCCs have done a lot of work in completing the data collection and clarifying the unresolved issues. There are data for ten or more years from 26 MCCs out of the 32 current MCCs. In 17 MCCs, ten or more years have score "2". There are several years in three MCCs having ten years' data for which the assessment cannot be completed because of missing data on routine mortality statistics. Four MCCs have data for nine years and will not be able to complete the tenth year: one of these with nine years data (AUS-NEW) score "2" for all nine years; there have been problems in the availability of  ECGs in FRA-TOU; and the coding of enzymes is deviant from the other MCCs in  SWI-TIC and SWI-VAF. Bremen (GER-BRE) has data for 8 years, all coded "2".   Luxembourg (BEL-LUX)  has data for seven years but routine mortality statistics are missing for four years and the quality assessment cannot be completed for this MCC.

There are still unresolved data queries in some MCCs. Clarifications which have been received from the MCCs are listed in Section 10. Several MCCs have notified that they are unable to make any data amendments.

The explanations for scores "2*",  "0" and "1" are included in Section 10. It is important that those who use the coronary event data for analyses read this section carefully so as to be able to understand the data better and make decisions about their use.

Appendix 5 defines a risk score for trends in coronary events, and shows the values of the score and its components for the RUAs and years to be considered in the the forthcoming publication of the 10-year trends.

10. Comments on individual RUAs

AUS-NEW

ICD-version used: 9/ICD9-CM (ICD9-CM used since 1st Jan 1988)
Non-fatal case finding method: Hot pursuit
Access to death certificates: Full

Comments:

AUS-PER

ICD-version used: ICD9/ICD9-CM
Non-fatal case finding method: Cold pursuit
Access to death certificates: Full

Comments:

BEL-CHA

ICD-version used: 9
Non-fatal case finding method: Hot pursuit
Access to death certificates: Restricted

Comments:

BEL-GHE

ICD-version used: 9
Non-fatal case finding method: Hot pursuit
Access to death certificates: Restricted

Comments:

BEL-LUX

ICD-version used: 9
Non-fatal case finding method: Mixed
Access to death certificates: Restricted

Comments:

CAN-HAL

ICD-version used: 9
Non-fatal case finding method: Hot pursuit
Access to death certificates: Restricted

Comments:

CHN-BEI

ICD-version used: 9
Non-fatal case finding method: Cold pursuit
Access to death certificates: Full

Comments:

CZE-CZE

ICD-version used: 9
Non-fatal case finding method: Mixed
Access to death certificates: Full

Comments:

DEN-GLO

ICD-version used: 8
Non-fatal case finding method: Cold pursuit
Access to death certificates: Full

Comments:

FIN-KUO

ICD-version used: 8/9 (During 1983-1986 ICD 8, from 1987 ICD 9)
Non-fatal case finding method: Hot pursuit
Access to death certificates: Full

Comments:

FIN-NKA

ICD-version used: 8/9 (During 1983-1986 ICD 8, from 1987 ICD 9)
Non-fatal case finding method: Hot pursuit
Access to death certificates: Full

Comments:

FIN-TUL

ICD-version used: 8/9 (During 1983-1986 ICD 8, from 1987 ICD 9)
Non-fatal case finding method: Hot pursuit
Access to death certificates: Full

Comments:

FRA-LIL

ICD-version used: 9
Non-fatal case finding method: Mixed
Access to death certificates: Restricted

Comments:

FRA-STR

ICD-version used: 9
Non-fatal case finding method: Cold pursuit
Access to death certificates: Restricted

Comments:

FRA-TOU

ICD-version used: 9
Non-fatal case finding method: Cold pursuit
Access to death certificates: Restricted

Comments:

GER-AUG

ICD-version used: 9
Non-fatal case finding method: Hot pursuit
Access to death certificates: Restricted

Comments:

GER-BRE

ICD-version used: 9
Non-fatal case finding method: Cold pursuit
Access to death certificates: Full

Comments:

GER-EGE

ICD-version used: 9
Non-fatal case finding method: Mixed
Access to death certificates: Full

Comments:

GER-RHN

ICD-version used: 9
Non-fatal case finding method: Hot pursuit
Access to death certificates: Restricted

Comments:

HUN-BUD

ICD-version used: 9
Non-fatal case finding method: Hot pursuit
Access to death certificates: Full

Comments:

HUN-PEC

ICD-version used: 9
Non-fatal case finding method: Hot pursuit
Access to death certificates: Full

Comments:

ICE-ICE

ICD-version used: 9
Non-fatal case finding method: Cold pursuit
Access to death certificates: Full

Comments:

ITA-BRI

ICD-version used: 9
Non-fatal case finding method: Hot pursuit
Access to death certificates: Full

Comments:

ITA-FRI

ICD-version used: 9
Non-fatal case finding method: Cold pursuit
Access to death certificates: Full

Comments:

LTU-KAU

ICD-version used: 9
Non-fatal case finding method: Cold pursuit
Access to death certificates: Full

Comments:

NEZ-AUC

ICD-version used: 9
Non-fatal case finding method: Hot pursuit
Access to death certificates: Full

Comments:

POL-TAR

ICD-version used: 9
Non-fatal case finding method: Cold pursuit
Access to death certificates: Restricted

Comments:

POL-WAR

ICD-version used: 9
Non-fatal case finding method: Cold pursuit
Access to death certificates: Restricted

Comments:

RUS-MOC

ICD-version used: 9
Non-fatal case finding method: Cold pursuit
Access to death certificates: Full

Comments:

RUS-MOI

ICD-version used: 9
Non-fatal case finding method: Cold pursuit
Access to death certificates: Full

Comments (see the comments for RUS-MOC, too):

RUS-NOC

ICD-version used: 9
Non-fatal case finding method: Hot pursuit
Access to death certificates: Full

Comments:

RUS-NOI

ICD-version used: 9
Non-fatal case finding method: Hot pursuit
Access to death certificates: Full

Comments (see the comments for RUS-NOC, too):

SPA-CAT

ICD-version used: 9
Non-fatal case finding method: Cold pursuit
Access to death certificates: Full

Comments:

SWE-GOT

ICD-version used: 8/9 (ICD 8 until 1986, from 1987 ICD 9 )
Non-fatal case finding method: Hot pursuit
Access to death certificates: Full

Comments:

SWE-NSW

ICD-version used: 8/9 (ICD 8 until 1986, from 1987 ICD 9 )
Non-fatal case finding method: Cold pursuit
Access to death certificates: Full

Comments:

SWI-TIC

ICD-version used: 8/9 (ICD 8 is used for non-hospitalized fatal events and ICD 9 for hospitalized events)
Non-fatal case finding method: Cold pursuit
Access to death certificates: Very restricted

Comments:

SWI-VAF

ICD-version used: 8/9 (ICD 8 is used for non-hospitalized fatal events and ICD 9 for hospitalized events)
Non-fatal case finding method: Cold pursuit
Access to death certificates: Very restricted

Comments:

UNK-BEL

ICD-version used: 9
Non-fatal case finding method: Mixed
Access to death certificates: Full

Comments:

UNK-GLA

ICD-version used: 9
Non-fatal case finding method: Cold pursuit
Access to death certificates: Full

Comments:

USA-STA

ICD-version used: 9-CM
Non-fatal case finding method: Cold pursuit
Access to death certificates: Full

Comments:

YUG-NOS

ICD-version used: 9
Non-fatal case finding method: Hot pursuit
Access to death certificates: Full

Comments:

References to publications

  1. Tunstall-Pedoe H for the WHO MONICA Project. The World Health Organization MONICA Project (Monitoring Trends and Determinants in Cardiovascular Disease): A major international collaboration. J Clin Epidemiol 1988;41:105-14.
  2. Tunstall-Pedoe H, Kuulasmaa K, Amouyel P, Arveiler D, Rajakangas A-M, Pajak A for the WHO MONICA Project. Myocardial infarction and coronary deaths in the World Health Organization MONICA Project. Registration procedures, event rates and case fatality in 38 populations from 21 countries in 4 continents. Circulation 1994;90:583-612.
  3. WHO MONICA Project. MONICA Manual. Part IV: Event registration. Section 1: Coronary event registration data component. (March 1999). Available from: URL: http://www.ktl.fi/publications/monica/manual/part4/iv-1.htm, URN:NBN:fi-fe19981154.
  4. WHO MONICA Project. MONICA Manual. Part II: Annual Statistics. Section 1: Population demographics and mortality data component. (December 1998). Available from: URL: http://www.ktl.fi/publications/monica/manual/part2/ii-1.htm, URN:NBN:fi-fe19981149.

References to internal MONICA documents

  1. Pajak A, Kuulasmaa K, Tunstall-Pedoe H, Martin I, Rajakangas A-M for the WHO MONICA Project. Coronary event registration data quality report. MONICA Memo 173A, February 1990.
  2. Mähönen M, Rajakangas A-M,  Kuulasmaa K, Tunstall-Pedoe H for the WHO MONICA Project. WHO MONICA Project - quality assessment of coronary event data for 1980-1990.  MONICA Memo 257A, February 1994.