November 1990
This section provides a description of the quality assurance methods to be applied to the Event Registration procedures in the MONICA Project and should be read in association with Part IV.1 and Part IV.2.
Quality control procedures for event registration are more difficult to specify and to carry out than for other aspects of the MONICA Project. Whilst laboratory procedures, ECG coding and procedures in population surveys can be specified centrally, event registration is grafted on to the existing medical care system. It has to coexist with it, adapt itself to it and be prepared to evolve with it. Registration systems cannot be effective if the medical care system is disorganized, fragmented and inefficient. Patients will be missed and misdiagnosed, records will be untraced, laboratory reports misfiled, wrongly dated or lost. However, a registration system should be as efficient as is possible under the constraints imposed by a particular medical and medico-legal system. Diplomacy and persistence may lead to changes that make registration more efficient or less laborious.
Procedures will vary depending on whether events are identified and registered as a result of their admission to hospital ("hot pursuit") or through use of post discharge diagnostic lists to obtain case notes retrospectively ("cold pursuit"). Whichever methods are used (and there are intermediate methods, such as identifying cases through searching hospital wards to obtain a list which is then used to raise the case notes subsequently) certain general principles should be applied.
Part of the routine of registration is the use of death certificates covering certain age groups and certain disease groups in residents of the MONICA population. There are several different procedures for doing this.
Once an event has been identified through the death certificate, the obtaining of further medical or medico-legal data should follow a systematic routine. In centres where death certificates are obtained very late, alternative sources of notification will be used, such as search of local newspapers, or medico-legal sources. The source of first notification may differ, but the total potential of sources of further information is probably the same - usual medical attendant, hospital records, emergency services, medico-legal sources and eyewitnesses or relatives.
Quality assurance of data depends on a systematic routine, and use of all readily available data. There must be rules formulated and observed for coping with vague, inadequate, and even contradictory data from different sources and systems for recording when arbitrary decisions have had to be made.
Most MCCs will not be using the interviewing of relatives as a matter of routine. However, where it is done the interviewers should be trained and tested. (See below under hot pursuit). However, the duplicate interviewing of bereaved relatives is not feasible, so it may be necessary to use dummy cases for training.
As previously described there are several variations on this method of identifying events. In the most extreme form this involves identifying patients acutely in hospital and obtaining the information directly from them whilst they are under acute care, interviewing them directly. The problem with this method is that the interview technique is very difficult to standardize so that descriptions of symptoms may vary with the observer. Periods of staff shortages or holidays may lead to loss of cases that cannot be recovered and a large team is needed to search the wards for cases. However, some information may be more complete than that which can be obtained from case notes. For this method staff should be trained by sitting in on interviews, by tape recording and by duplicate interviewing, and a proportion of all interviews should be duplicated on a random basis indefinitely.
Notification of events should be instituted on a routine basis that should not depend too much on the personal relations between the MONICA team and the clinical staff. For example, asking for patients may be less efficient than checking admission books on the wards. Busy clinical staff may find it easy to deny they have patients, or may genuinely forget them, or may get the age and domicile wrong, and it is then possible to lose cases.
Methods of registration that depend on the enthusiasm and devotion to duty of MONICA staff may not be practical over a 10-year period and it is better if regular, even bureaucratic, routines can be established.
While the extreme forms of hot pursuit involve getting the information from the patient acutely, an alternative is to use the hot pursuit method to identify the patients of interest and to mark their notes or list them for review later. This method depends on the ability of the hospital clinical and record services to obtain the notes that are requested for the MONICA staff. An efficient reliable routine is needed for picking up the case notes at an identifiable point in their processing.
One problem with hot pursuit methods is that the diagnosis may not have been made when the case is notified, so that many more cases are potentially notified than deserve full registration. Consistency demands that there is some method of quality assurance built into the registration system so that a decision on which cases are investigated and which ignored is not an arbitrary one that can be made, and not checked up on. There must be explicit rules and accountability and audit, or there is a danger that pressure of work, or enthusiasm, will decide whether doubtful cases are registered or not. As time goes by considerable trends could occur as registration staff change, or become bored or overworked.
Registration of patients on the wards involves taking registration documents out of the MONICA offices, or having several sorts of documents. A security routine is necessary to ensure that documents do not get lost, filed in patients' case notes, or otherwise mislaid. A tracking system must be instituted to show which documents are being taken where, and who has them.
It is difficult to check up on a hot pursuit system several months later, but some centres will use discharge lists as a backup method to ensure that the hot pursuit method had detected all the diagnosed cases. Residents hospitalized outside the area, and these late-detected cases mean that a proportion of events will always have to be registered by cold pursuit, weeks or months later.
Use of discharge diagnoses rather than hospital admission is a more bureaucratic system of identifying events for the MONICA study. Its advantage is that it can be done months or years after the event; its disadvantage is that the information in the case notes must be reasonably complete and the notes themselves must be accessible.
Discharge listings are often computerized. It is important that during the pilot studies and then periodically, the MONICA team ensures that every hospital admission leads to a discharge diagnosis. Medical care systems in which the patients are charged for their treatment are more likely to have complete discharge records than those where there is less financial incentive for the hospital to make its records 100% complete.
The description of the quality control methods for computerized versus manual discharge lists, and how to extract the events is very much the same for hospital events as for death certificates (See 1.1).
Once events have been identified, a routine is needed for obtaining case notes and extracting information from them. The extraction of medical data from medical records is a skilled business and medical staff do not often realize how both complex and disorganized medical records can be. While different operations can be done by those with different levels of skills, it is essential that all delegated jobs are supervised and subjected to random duplicate checks and audit, and that those carrying them out are fully trained, feel competent and know when to ask for help. In those Centres in which certification of staff is used, it is needed in this context, but in some cultures it is not considered appropriate.
Events should be coded from hospital case notes in a suitable environment for unhurried accurate work, not on a corner of a table in a dark corner of the records department. Some case notes are much more bulky than others and information may be concealed out of context. Staff should not work to simple deadlines therefore but should be encouraged to spend time looking through the contents of the folders. Problems should be discussed with a supervisor and those extracting the data should be encouraged to copy down or photocopy parts of the notes that they find difficult to follow. If necessary, notes should be obtained more than once to check on difficult items.
Difficult coding problems should be discussed regularly and any arbitrary decisions should be logged in a problem book so that they can be looked back on a long time later. Any problems that may be of general interest should be discussed on a Centre, or national level, and, unless it is too rare to matter, can be communicated to the Dundee Quality Control Centre for Event Registration. Of particular interest in this context are new treatments or diagnostic techniques or interventions that threaten the long term stability of coding.
In most MCCs virtually all patients with suspected acute myocardial infarction in the MONICA age groups are managed in hospital, but in some MCCs, general practitioners manage some cases at home and occasionally patients refuse to go into hospital. Unless a good communication system is maintained with general practitioners, events such as these will be lost, but at the same time, they may be too rare to be worth troubling general practitioners for at all frequently. A routine must be established so that events stand a similar chance of being detected year by year. Alternatively, a special effort could be made once every three years to see what small numbers were being missed, but then excluding them from the main analysis. This latter method is permissible if the total proportion of events revealed by this method is less than 5%. It means that rare cases can be excluded, but a periodic survey is needed to ensure that they are still rare.
In most cases registration of events at home will be on the basis of hot pursuit, as documentation and investigation of cases by general practitioners may be inadequate for cold pursuit. Few (but an increasing number) have a diagnostic index.
These events will be both fatal and non-fatal and their frequency will depend on local circumstances. Local residents may travel within their own country for work and holiday, and also outside their country for work and holiday. The problems within the country are different from outside it. Routines should be established to cover these events as far as possible. An increase in holidays abroad over ten years could introduce a bias in registration.
External quality control is an extension of, but not a replacement for internal quality control. The methods used in external control should be used regularly within each MCC which has the advantage of using local records in the local language.
Event registration quality control material is produced in the MONICA Quality Control Centre for Event Registration in Dundee. It covers Coronary Events, Acute Coronary Care and Stroke Events. Case histories are produced from local material, from material sent in by MCCs, and also by introducing items into histories to test particular coding problems.
Material is circulated about once a year for coding in the form of case histories and coding forms. The histories are translated into the local language in the MONICA centre. They should then be coded on to the test case history forms (Coronary events, Acute coronary care, Stroke events) by those responsible for coding within the centre. Results should be sent back to Dundee within the deadline so that a report can be prepared. The external quality control exercises serve two overlapping functions:
The Dundee Centre generates a general report addressed to all MONICA Collaborating Centres and specific reports for individual MCCs. Clarifications to coding instructions are discussed either at the Principal Investigators' Meeting, or with the Steering Committee, and these are circulated as MONICA Memos, often for discussion first, and implementation afterwards if approved.
Another form of external quality control which should be implemented within centres for internal control also is the potential for requesting validation of a particular case record by the production of the original documents from which the coding was done. The possibility for doing this has been agreed by the principal investigators so that the MONICA Data Centre, Management Centre or Quality Control Centre could implement it, and internal systems should be designed to cope with such a potential request.