Contents        Previous (Part III Section 5)        Next (Part III Section 7)

6. Data management

Data management is one of the corner stones of a well-organized risk factor survey. Good data management ensures that:

1. the data which will be available for data analyses and reporting are:
   • complete, i.e. no data collected from the survey subjects were lost.
   • correct, i.e. there are only justifiable differences between the values which were originally measured and the values in the final database.
   • verifiable in the sense that the relationship between the original data collected and the final data in the database can be described.
2. the data analysis will be:
   • done using the correct data and other information which are relevant for the data analysis.
   • done without errors.
   • documented in such a way that the whole analysis or a part of it can be repeated later. If the documentation is not done properly, it may be difficult or impossible to reach the same results when similar analyses are repeated in other situations.
3. the confidentiality of the data is secure. It is particularly important that the information which connects the survey data to the personal identification of the subject, will be available only to persons who have authorized access to such data.

Point one above involves data collection, checking, error correction, documentation and back-up of the study database. Point two concerns the analysis of the data from the database, to obtain survey results. Separation of the data management into two stages is highly recommended. If the database is not completed and the quality of the data are not documented before the data are analyzed for the survey results, it is likely that the data analysis will reveal problems in the data which should have been detected earlier in the preparation of the database. This usually results in much longer delays in the final analysis than if more care had been taken in preparation of the database. Furthermore, use of unchecked and uncorrected data will lead to incorrect results.

Well-planned data management facilitates good quality data and ready availability of the data for analysis. Poorly planned data management is usually expensive, increases risk of loss of data, decreases data quality and delays the data analysis.

We will consider here issues which have been found crucial for survey data management. There are good methods for many of the stages of the data management, often ranging from manual methods to computerized ones. The choice of the methods will depend on the local facilities, existing practice and the expertise available, and will not be considered here.

6.1 Sample selection and recruitment

The first data management issue in a survey is usually the management of the documentation of the survey planning. The next main data management issue relates to sample selection and recruitment. As a minimum, the following information needs to be recorded for every subject selected for the sample:

• Subject identification. This is a unique code which will be used to identify the subject throughout the survey and data management. It is a key item, which has to be correct at all stages. Therefore, it is strongly recommended that it includes a checking code, so that the accuracy of any single subject identification code can be checked at any stage. For data confidentiality reasons, the subject identification should be such that the identity of the person on whom the measurements were made cannot be derived from it (see Section 6.5).
• Sampling information. If the sample was stratified, the information on the strata in which each subject was selected needs to be recorded. If multi-stage sampling was used, the identification of all sampling units of the subjects have to be recorded.
• Address information, such as the person's name, address and any other information will be needed to contact that person.
• Often, additional information such as sex and age are also available from the sampling frame.

The subjects selected in the sample form the basis for the control of the data completeness throughout the data management process. The survey history of every subject should be verifiable from the final data base.

At the recruitment stage, attempts made to contact each subjects need to be monitored, and the eventual success of the recruitment needs to be recorded. For the subjects who did not participate in the survey examinations, the reason should be recorded using a classification which at least includes the following categories:

1. The subject has died before the scheduled examination date.
2. The subject has moved out of the survey area before the scheduled examination date. In a national survey the area may be the country, but in a geographically stratified or multi-stage sampling a smaller area may be relevant here.
3. The subject's age group or sex do not correspond to those in which he or she was selected to the survey. Such situations relate typically to errors in the sampling frame.
4. The subject refused to take part in the survey.
5. The subject was unable to attend the survey due to medical reasons.
6. No information is available to classify the subject to the other categories. This usually relates to situations where there is no evidence that the subject had received any of the invitations.

Categories 'a' to 'c' are often called ineligible to the survey, whereas  'd' and 'e' are true respondents. The eligibility of category 'f' is often uncertain, and this category should be allocated sparingly.

6.2 Survey measurements, data transfer and keying

Recording the survey measurements and getting the data from the different survey sites to a common database is a challenging part of the data management. All steps where data are transferred from one form to another or from one place to another require specific attention at the planning stage. Such steps include:

• completion of self-administered questionnaires
• interview
• recording the values of physical measurements
• blood sampling, processing and transfer to the laboratory
• recording the laboratory results
• transfer of paper forms and/or electronic records from the examination site or laboratory to the common database
• getting the data into electronic format

Three main challenges for data management in these steps are to ensure that:

1. no errors are made in recording the measurement results;
2. the data records are complete;
3. no records are lost or different persons' records are not mixed up.

Errors and incompleteness of the records can be prevented by good design of record forms and by routine checking of the forms and data. The earlier the errors can be detected, the easier they are to remedy. In particular, where feasible, detection of errors should be attempted when the subject is still in the interview or examination site.

Relevant data which were not obtained from the subject should never be left blank on the form, but a specific code for missing data should be used. Subsequently, the incompleteness of the data records can be detected easily as blanks in the data forms.

Computer-based data collection systems have the advantage that they reduce the number of manual data transfers and facilitate extensive data checking at an early stage. However, such systems should be used only if they have been tested in the field and found reliable. Otherwise there is an increased risk for losing records or delaying the examination schedule due to breakdown of the system.

The use of paper forms has proven to be reliable over decades, but they have the problem that on-site data checking is more difficult. If paper forms are used, the keying of the data into electronic format will need to be done carefully. The traditional method of double keying by different persons is still worth considering.

To prevent the loss of records, it is important that the subject identification becomes correctly recorded at all stages (see Section 6.1). Furthermore, whenever data or samples are transferred from one place to another, it is important that the transfer is logged properly. The recipient of the data or samples should be able to check immediately that he or she has received exactly the same records which were sent, and the person sending the data should make sure that everything was received.

6.3 Error checking, data correction, data documentation, database structure and documentation

The data should be checked as soon as possible after the data collection for:

• outlandish values, i.e. for values which have not been defined, and also for values which are possible but rare; 
• consistency between the values of different data items;
• completeness, i.e. that all data items have been recorded and no records have been missed.

A visual checking of the key items can be done at the interview or examination site even if paper forms were used, and extensive checking should take place as soon as the data have been computerized. When potential errors are detected, they should be investigated for correctness, and corrected only if it is found that they really are errors. It is a good practice to authorize only those who have made the errors to correct them, because they are usually in the best position to say if there really is an error, and they are usually the only ones who know the correct value. Each error and its possible correction should be documented.

The frequency of errors, which were not possible to remedy should be documented. The same concerns the results of the quality control during the data collection, any deviations from the survey protocol, and any other information which may be relevant in the interpretation of the results. Knowledge of these issues is essential to those who analyze the data and interpret the results. Examples of the routine error checking criteria and documentation of the quality of the data, in a multinational setting can be found in the MONICA quality assessment reports, which are available through the internet at http://www.ktl.fi/publications/monica/.

The database can be structured in different ways, depending on the available facilities and expertise, and size or complexity of the survey. Often the data management facilities of statistical computer packages (such as SAS or SPSS)  are sufficient, but specific database management systems (such as ORACLE RDB) may provide more versatility. More important than the selection of the database management system is usually the fact that expertise on data management should be present from the beginning of the planning of the survey. There are too many examples of surveys where the data quality control and the data security had to be compromised because expertise in data management was acquired too late.

6.4 Backup

All data in electronic format has to be backed up routinely for accidental breaks of the storage devices, failures in data transfer and unintentional deletion of the data files. Common situations where important data have been lost, although some back-up was in place, are:

• Loss of data data during data collection or data processing, because back-up was neglected at these early stages.
• Accidental loss of the back-up data together with the original data because the two were stored together, or the broken device or system which destroyed the original data was used to open the back-up data.
• The complete back-up data had already been replaced by the incomplete data before the loss of data had been detected. This may happen if the system for long-term back-up is incomplete.
• There was back-up, but there were insufficient documentation on its location or on the procedures needed to retrieve the data from the back-up. This frequent problem arises due to unforeseen changes in personnel.

Back-up is not only needed for data in electronic format, but also for some paper documents, such as log books for the survey examinations.

6.5 Confidentiality

Only authorized persons should have access to the data and all of them must understand the importance of the confidentiality of the data. After the data collection, the information from which a person can be identified and the code which connects this information to the subject identification of the survey records, has to be stored separate from the survey data. Usually very few people need access to the person identification information, whereas the rest of the survey data will need to be accessed by all who analyze the data. Specific precautions should also be defined for the handling and storage of paper forms in the examination site and elsewhere.

6.6 Correctness of data analysis

Routine statistical program packages usually carry out the analyses which the user instructs them to do correctly. However, any misprints in the user's instructions or misunderstandings of the language of the programs lead to incorrect results. Therefore, important analyses should be checked by another person, and the results of all analyses should be reviewed critically by the programmer and other persons involved in reporting the findings. Where possible, the results should be compared with other results of similar analysis, and the reasons for inconsistencies investigated. It is recommended that a data book is prepared as part of the preparation of the data, consisting of the distributions of all data items and basic descriptive statistics of the data. Such a data book will be an invaluable reference when checking the correctness of most later data analyses.

6.7 Analysis documentation

The documentation of certain issues relating to all data analyses which lead to reporting or publication will be valuable. The analyses may need to be repeated because of changes made to the data, or if similar analyses will be performed on other data, or if  uncertainties about the correctness of the analysis emerge. The most important issues to be documented are the:

• Person who did the analysis and the date of documentation.
• Location or other identification of the data set used for the analysis.
• Source code of the data analysis program.

The analysis documentation can be very simple, and therefore need not take much time. It can be kept on paper or in electronic form, but it should be readily available to all those who may need it.

Contents        Previous (Part III Section 5)        Next (Part III Section 7)