Contents Previous (Section 13.)
This section will deal with issues related to the design and testing of the survey questionnaire, especially wording and language of the questions as well as length of the questionnaire.
14.1.1 Wording of the questions
The correct understanding of the questions depends on the wording. If wording is imprecise or the respondent does not understand some words used in the question he/she may misinterpret the question or omit the question without answering it.
Professional jargon should be avoided in formulating the questions, because respondents are usually lay persons and are unfamiliar with the professional terms. The words used, should be easily understood and should be unambiguous. (1, 2)
In general, the questions should be short and simple. Sometimes, longer questions will need to be accompanied by explanations to ensure correct interpretation. This is especially important if questionnaire is designed to be self-administered. (1, 2)
Abbreviations should be avoided in the questions unless they are expected to be known to the respondents. (3)
Response alternatives for questions can be either open-ended or close-ended. Both alternatives have their advantages and disadvantages.
Close-ended response alternatives list the possible answers to the question. This makes the comparison of responses easier and reduces coding errors. (1)
When preparing the list of answer alternatives for a close-ended question, attention should be paid to the number of alternatives as well as to range of each alternative. Answer categories have to be mutually exclusive. Listed answers can help respondent to understand the question. (1)
The response categories have to cover all possible answers. If there are too many alternatives, the difference between alternatives may be marginal, and the respondent may find it difficult to choose the correct answer. (4)
In open-ended questions the respondent may find it easier to give the precise answer to the question than in closed-ended. In close-ended questions none of the alternatives may correspond exactly to the answer the respondent has in mind and he/she has to settle for something else. (1)
Open-ended questions can give more information than closed-ended but standardized coding of the answers is difficult. (1)
Open-ended questions are also useful if there is a known and limited, but long list of alternatives.
14.1.3 Language and translation of the questions
Questions have to be in a language familiar to the respondent, preferably in his/her mother tongue. When questions need to be translated from one language to another, special steps should be taken to ensure that original and translated questions ask exactly the same thing. (3)
WHO MONICA Project
In MONICA all centres had their local questionnaires in their own language. For multilingual countries the questionnaire was issued in more than one language.
Risk factor monitoring in the Netherlands
No information available.
Risk factor monitoring in Germany
German Federal Health Survey 1998 was the first of the German Health Surveys that recruited not only German citizens, but residents in general. Eligibility criteria required language proficiency in German, which was judged by the interviewer without a formal test. It is not know whether question explanations in other languages were available to the interviewers.
UK National Health Survey
For the Health Survey 1999 for England, which focused on the health of minority groups, all survey questionnaires were translated into seven languages: Hindi, Gujarati, Punjabi, Urdu, Bengali, Mandarin and Cantonese. The validity of the translation was "checked during piloting of the survey, but no other work was done in terms of formal verification". (5) Informants who could not carry out an interview in English were provided with an interviewer who could speak the appropriate language. (6)
NHANES III
In NHANES III interviews were conducted either in English or Spanish. (7)
What a person can remember from the past has its limits. Recent happenings are easier to remember but when a person is asked to recall events from the past, accuracy of the recall gets worse while time span expands. (8)
Long recall periods may have a telescoping effect on the responses. This means that events further in past are telescoped into the time frame of the question. (4, 8) For example, if the question asks if a person had his/her blood pressure measured in past 12 months, the respondent often places the blood pressure measurement taken 15 months ago into the time frame of 12 months.
The process of recall of events from the past can be helped by questionnaire design and process of interview. The order of questions in the questionnaire can help respondents to recall events from the past. Also giving some landmarks (holidays, known festivals etc.) can help to remember when some events happened. (4, 8) Also, use of a calendar may help a respondent to set events into the correct time frame. (4)
WHO MONICA Project
In the WHO MONICA Project different recall periods were used for different kinds of questions. In smoking questions, the present and ever smoking status is asked (see Chapter 7.). For hypertension, it is asked if a person has ever been diagnosed as hypertensive and has taken any medications in the past two weeks and had a blood pressure measured within past 12 months (see Chapter 10.). For high cholesterol, same time limits are used as for the hypertension (see Chapter 11.). For aspirin use, a two-week (see Chapter 12.) and for hormone replacement therapy a one-month (see Chapter 13.) time interval is used.
Risk factor monitoring in the Netherlands
In the Monitoring Project on Cardiovascular Disease Risk Factors in the Netherlands, the time frames for the smoking question are "present" and "ever" (see Chapter 7.). For high cholesterol, the person is asked if he/she has ever been diagnosed to have high cholesterol (see Chapter 11.)
Risk factor monitoring in Germany
The questionnaire for the German Federal Health Survey 1998 (9) uses recall periods "ever", "past 12 months", "past four weeks", and "past 7 days" for health history questions. In the section on use of medical services, a question inquires about the last time a physician was consulted. The multiple choice answers distinguish between past 4 weeks, 1 to 3 months, 4 to 12 months, 1 to 5 years, and more than 5 years. The section on nutrition probes for body weight changes during the past three years. The food frequency part of the survey requests average data for the past 12 months. The choices for consumption frequencies are: several times per day, daily, several times per week, once per week, two or three times per month, once per month, and almost never. The physical activity section has a recall period of three months. The section on leisure time activity collects information about foreign travel during the past three years. The frequency of medication use was to be specified for the past 12 months.
UK National Health Survey
In the National Health Survey in the UK, the recall periods were either "present" or "ever". Smoking status was asked both for present and ever (see Chapter 7.). For hypertension, it is asked whether the person had ever been diagnosed to have hypertension and whether he/she ever had blood pressure measured. Medication status for hypertension is determined for "present" (see Chapter 10.). The question related to high cholesterol asked if the person had ever had his/her cholesterol measured (see Chapter 11.). The "ever" time span was also used for the question about hormone replacement therapy (see Chapter 13.).
NHANES III
In the NHANES III most of the questions are asked in the time interval "ever". The recall period "during the last month" is used for the question on aspirin use (see Chapter 12.) and smoking status is determined for "present" (see Chapter 7.). For other smoking question, awareness and treatment of hypertension (see Chapter 10.), awareness and treatment of high cholesterol (see Chapter 11.) and use of hormone replacement therapy (see Chapter 13.) the persons are asked to recall whether they "ever have been told/used".
Questionnaires should begin with easy, non threatening, and simple questions. If there are complicated and sensitive questions in the beginning that the respondent finds difficult to answer then he/she may refuse to continue with the questionnaire. When complicated and sensitive questions are placed later in the questionnaire, most of the information has already been collected if the respondent would refuses to continue with the questions. Also, if the questionnaire starts with easy questions, the respondent may establish a trusting relationship with interviewer and may be more willing to answer also difficult questions if they come later during the interview. (4, 8)
A respondent should not be asked questions that are not relevant for him/her. This can be avoided using the filtering questions. (3, 4) For example, if an informant denies ever having smoked, no more smoking questions should be asked.
The order of questions can also be used to help respondent to recall past events. Grouping the questions about same topic helps to concentrate on the issue and "dig" the memory.
WHO MONICA Project
As all MONICA centres had their own local questionnaires the order of questions varied between them. However, all questionnaires started with simple demographic questions like age/date of birth and sex.
Risk factor monitoring in Germany
The questionnaire for the German Federal Health Survey 1998 started with simple demographic questions that were followed with health history questions. Embedded in the long list of possible afflictions is the sensitive issue on alcohol and drug addiction. (9)
Other surveys
The order of questions is unknown at this point in time for Risk factor monitoring in the Netherlands, UK National Health Survey, and NHANES III .
14.1.6 Length of the questionnaire
Researchers want all the questions included in the questionnaire that are needed to answer the hypothesis set for the study. This goal has to be balanced against the need to keep the questionnaire as short as possible to improve response rates and quality of data. (1) If the questionnaire is very long and complicated, respondents easily get reluctant to complete it or complete it only partially. Also in case of a long questionnaire, the respondent may get tired answering questions and towards the end of questionnaire the reliability of answers gets to be lower. (4, 10)
WHO MONICA Project
The length of the questionnaires varied greatly between MONICA centres. In some centres, the questionnaire covered only 64 MONICA questions, whereas in some other centres several hundred questions were asked.
Risk factor monitoring in the Netherlands
Questionnaire length is unknown
Risk factor monitoring in Germany
In the German Federal Health Survey 1998, the questionnaire was self-administered and had 107 questions. There was also a computer-assisted interview by a physician using a laptop computer. The number of interview questions is unknown.
UK National Health Survey
In the Health Survey for England there were two questionnaires, one filled in during the interview and the other one was self-administered. The interview questionnaire includes about 200 questions and the self-administered questionnaire about 40 questions. (11)
NHANES III
In NHANES III the respondent first completed the Household Adult Questionnaire during the home interview. This questionnaire had 21 sections, each with between 6 and 34 questions. After that, still during the home interview, respondents answered the Family Questionnaire. This questionnaire had 6 section each with 11-20 questions. During the examination at Mobile Examination Center (MEC), the respondent had to answer two more questionnaires; MEC Adult Questionnaire and 24 Hour Dietary Recall Form. The MEC Adult questionnaire had 8 sections each with 2 to 67 questions. The 24 Hour Dietary Recall Form had 10 question. (7)
With a pilot study, all aspects of the questionnaire can be tested (2), like:
Mode of administration
Questionnaires can be either self-administered or filled in by an interviewer. Self-administered questionnaires can be mailed to the respondents or delivered personally. Interviews can be either a telephone or face-to-face. In any case it is important that clear instructions are available for interviewer and respondent.
Several issues can influence the choice of administration mode. Low literacy level in the population may favour a face-to-face interview, whereas sensitive issues may better be addressed by self-administered questionnaire, which provides greater privacy for the respondent. (4)
It is advisable that self-administered questionnaires be checked with the respondents for completeness and consistency during the a subsequent visit to the examination centre.
WHO MONICA Project
In the WHO MONICA Project the questionnaire could be administered either by self-administration or interview (12). There were additional instruction for both alternatives:
In the case of self-administered procedures, the questionnaire should be checked by a technician or a nurse for completeness and consistency of answers.
In the case of direct administration, some general rules should be followed:
Interviewers should be trained and their performance evaluated and tested for precision and accuracy.
In most, but not all of the MONICA populations the questionnaire was completed during the interview (13). This choice was made in many populations because of uncertainty about the literacy level of the prospective participants.
Risk factor monitoring in the Netherlands
In the Monitoring Project on Cardiovascular Disease Risk Factors and the MORGEN-project the questionnaires were filled in at home and checked at the research centre for possible problems, completeness and correctness (14).
Risk factor monitoring in Germany
In the German Federal Health Survey 1998, the questionnaire was self-administered at the examination centre, where assistance was available, if needed (15).
UK National Health Surveys
In National Health Surveys in the UK part of the questionnaire was filled in during the interview and part was self-administered. The self-administered questionnaire was checked by the interviewer for completeness and correctness. (11)
NHANES III
In NHANES III questionnaires were filled in during the interviews. (7)
Interviewer effect
The interviewer effect in surveys is a well know phenomenon. Interviewers can influence the quality of the data collected by (16):
Also some characteristics of the interviewer like age, gender, race, educational level, social class and ethnic or religious background can have effect. In telephone interviews, the type of voice can be quite influential. Especially with sensitive issues, the characteristics of the interviewer can influence a respondent's answers. Respondents may try to answer in a socially accepted or politically correct way. (16)
The interviewer-related error can be reduced by proper training. During the training, interviewers should be given a thorough understanding of the purpose and protocol of survey. It would be best if all interviewers were provided with a manual telling them how the interview should be conducted and how to act in different situations; for example, when the respondent does not understand question or does not give a precise answer. (16)
Training of the interviewers could include lectures, manuals and practice interviews (16). During the lectures, the protocol would be discussed and to-be-trained interviewers could ask questions about the survey and protocol. The manual should cover situation arising during the interviews where uniform procedures are needed; i.e. how to explain questions to the respondent or how to probe respondent to give precise answers. During the practical training, supervisors should monitor each interviewer and their technique and give feedback when needed.
During the survey, it is also important to supervise the interviewers (16). This can be done by monitoring the work load of each interviewer. Seeing that the number of interviews per day is reasonable and does not stress the interviewer too much. Also the response rates in whole and for each question can be monitored to see how well interviewers are achieving the co-operation of the respondent. The relative proportion of missing data may provide information on the thoroughness of the interviewer.
Wording of the questions
All surveys had their questionnaire in the local language. In some MONICA centres, NHANES III, and the Health Survey for England 1999 questionnaires were translated into other languages. The equivalency of the translated versions in the Health Survey for England 1999 was checked during survey piloting.
Recall periods used in the questions varied from question to question and from survey to survey. In many questions the respondents were asked to describe their present situation but in some they were required to recall past experiences. In past recall the most common period was the whole past life, such as have person ever been doing something or been told something. However, some questionnaires had a range of recall periods.
The effect of the order of questions was not described by the surveys and it is unknown whether systematic evaluations were carried out.
Length of the questionnaires varied from less than 100 questions used in some MONICA centres up to several hundred questions used in NHANES III. In NHANES III, where the total number of questions was enormous, the respondent's task of answering was reduced by dividing the questions into several questionnaires administered at different times.
In most of the surveys the questionnaires were administered by interview. Some had at least part of the questionnaire completed by self-administration.
Most surveys provided training to the interviewers, but no information was found about interviewer performance.
References
Contents Previous (Section 13.)