WWW-publications from the WHO MONICA Project
June 1999
Pedro Marques-Vidal1, Marco Ferrario2, Kari Kuulasmaa1, Dusan Grafnetter3 and Vladislav Moltchanov1 for the WHO MONICA Project4
1 MONICA Data Centre, National Public Health Institute, Helsinki, Finland;
2 Institute of Biomedical Sciences ‘San Gerardo’, and Research Centre
on Chronic-Degenerative Diseases, University of Milan, Monza, Italy;
3 WHO Lipid Reference Centre, Institute for Clinical and Experimental Medicine
(IKEM), Prague, Czech Republic;
4 Annex: Sites and key personnel of the WHO MONICA
Project.
This document includes the main findings of unpublished reports:
Thanks are due to Hanna Tolonen for her help in preparing the tables, and to John Yarnell who commented on the text.
The MONICA Centres are funded predominantly by regional and national governments, research councils, and research charities. Coordination is the responsibility of the World Health Organization (WHO), assisted by local fund raising for congresses and workshops. WHO also supports the MONICA Data Centre (MDC) in Helsinki. Not covered by this general description is the ongoing generous support of the MDC by the National Public Health Institute of Finland, and a contribution to WHO from the National Heart, Lung, and Blood Institute, National Institutes of Health, Bethesda, Maryland, USA for support of the MDC. The completion of the MONICA Project is generously assisted through a Concerted Action Grant from the European Community. Likewise appreciated are grants from ASTRA Hässle AB, Sweden, Hoechst AG, Germany, Hoffmann-La Roche AG, Switzerland, the Institut de Recherches Internationales Servier (IRIS), France, and Merck & Co. Inc., New Jersey, USA, to support data analysis and preparation of publications.
The aim of this report is to evaluate the quality of the data on HDL cholesterol measurement in the MONICA surveys. The quality assessment consists of:
Topics which are covered in the quality report of total cholesterol measurements (1) are not reported here if the results are identical for HDL cholesterol. In such cases, reference is made to the relevant section of the quality assessment of total cholesterol measurements..
The report considers the Reporting Unit Aggregates (RUA) which are foreseen as potential candidates for units of analyses of the MONICA data. The RUAs, their abbreviations and Reporting Units (RU) are listed in Table 1.Some of the RUAs have several versions because different combinations of RUs may be used for cross-sectional and trend analyses if all RUs of the population were not included in all three or two surveys. Therefore, in AUS-PERa, GER-BREa, GER-EGEa, GER-KMSa, GER-RDMa, RUS-MOIa and RUS-NOCa there is an overlap of RUs included in the RUAs in some surveys. The RUAs are identified by the abbreviation and a version letter. For UNK-GLAa which carried out four surveys, the first (initial), third (middle) and fourth (final) survey are considered.
Compared with other survey quality assessment reports, in the current document GER-RDMa and ITA-FRIa have been split into smaller RUAs according to the different laboratories used by their different RUs in some or all of the surveys. Altogether there were 57 RUAs, of which 54 RUAs were considered for the initial survey, 44 for the middle, and 42 for the final survey. The optional HDL cholesterol measurement was not done in the initial survey for GER-ERFa.
For the quality analyses all observations within the age group 25-64 were used, except in FRA-LILa in the final survey and in AUS-NEW, BEL-LUX, FRA-STR, FRA-TOU, LTU-KAU, NEZ-AUC, POL-TAR, POL-WAR, RUS-MOC, RUS-MOIa, RUS-MOIb and SWI-TIC where the age range studied was 35-64. Age was defined as age in full years on the date of examination (see DEF1 in reference 14). No age or sex adjustment was applied to the data.
Information on methods adopted in the different RUAs was originally gathered from the site visit questionnaires (parts Taking blood samples and Preparation of plasma/serum samples, MONICA Memo 68, pages 17-33) and was updated with details mainly from questionnaires on survey procedures (Questionnaires on MONICA Population Survey Procedures, Form VI, pages 31-37). At the time of the preparation of the present report, this information was not available for four RUAs which participated in the initial surveys (GER-RHNa, ISR-TELa, MLT-MLTa and ROM-BUCa). Other information was collected directly by the WHO-RLRC or the Lipid Quality Assessment Working Group, which prepared this quality assessment report.
The external quality control data were obtained from WHO-RLRC for the Centres standardized by this laboratory and from the MONICA Collaborating Centre for Centres standardized by CDC.
Some of the laboratories measured HDL cholesterol in mmol/l with two decimals, others in mg/dl with one decimal. The external quality control data were reported to the WHO-RLRC and the actual cholesterol data to the MONICA Data Centre (MDC) in the original units. The WHO-RLRC and the MDC have used the unit mmol/l in all reports. The relationship between the units is:
1 mg/dl = 0.025864 mmol/l.
The training procedures are considered in Section 3 of the quality assessment of total cholesterol measurements (1).
Most minor sources of variability for total cholesterol also apply for HDL cholesterol. The effect of those sources has been extensively reviewed in the the quality assessment of total cholesterol measurements (1). Only sources specifically relevant for HDL cholesterol will be assessed here.
The effect of diurnal variation and fasting status has been reviewed in Section 4.1.1 of the quality assessment of total cholesterol measurements (1).
The effect of strenuous exercise has been reviewed in Section 4.1.2 of the quality assessment of total cholesterol measurements (1).
The effect of rapid changes of body weight and of several diseases has been reviewed in Section 4.1.3 of the quality assessment of total cholesterol measurements (1).
The effect of selected medications has been reviewed in Section 4.1.4 of the quality assessment of total cholesterol measurements (1).
The effect of tourniquet use has been reviewed in Section 4.1.5 of the quality assessment of total cholesterol measurements (1).
The effect of storing before centrifuging has been reviewed in Section 4.1.6 of the quality assessment of total cholesterol measurements (1).
The effect of haemolysis has been reviewed in Section 4.1.7 of the quality assessment of total cholesterol measurements (1).
The effect of the number of HDL cholesterol determinations has been reviewed in Section 4.1.8 of the quality assessment of total cholesterol measurements (1).
Although seasonal fluctuations represent a major source of variability for total cholesterol measurements (as indicated in Section 4.2.2 of the quality assessment of total cholesterol measurements (1)), this is not the case for HDL cholesterol measurements (2).
Alcohol consumption is associated with an increase in HDL cholesterol levels (3, 4, 5), while smoking is associated with a decrease (5). All MCCs have collected individual data on smoking as part of the MONICA core study. Several MCCs have performed nutritional surveys or assessed food intake, but this is not part of the MONICA core study and therefore the data are not available in the MDC. Since smokers and alcohol drinkers are part of the population, they should not be excluded from the analysis.
The effect of posture on total and HDL cholesterol levels is considered in Section 4.2.3 of the quality assessment of total cholesterol measurements (1).
The effect of plasma-serum differences on total and HDL cholesterol levels is considered in Section 4.2.4 of the quality assessment of total cholesterol measurements (1).
According to the MONICA Manual, isolation of HDL should preferably be done on fresh serum aliquots on the day of blood collection (6). If impossible, the serum or plasma for HDL cholesterol determination should be frozen at -20°C and precipitation should be performed within 14 days. Storage of fresh samples for more than three days at +4°C leads to a reduction in HDL cholesterol levels of about 8.2% to 14.9% (7, 8). Storage of frozen samples for more than 14 days at -20°C leads to a decrease in HDL cholesterol levels (7, 8), whereas storage at lower temperatures does not produce such modifications (7).
Table 2 summarises the results of the three surveys. In the initial survey, HDL isolation was performed in fresh samples in 36 RUAs and on frozen samples within 14 days in two RUAs. In nine RUAs either frozen samples were stored for more than 14 days before HDL isolation, or HDL were isolated from fresh and frozen samples. Data on HDL isolation is missing for six RUAs (GER-RHNa, HUN-PECa, ISR-TELa, ITA-LATa, MLT-MLTa and ROM-BUCa), and in one RUA (HUN-BUDa) HDL cholesterol determination was performed in samples frozen at -18°C for a maximum time of 12 weeks. In the middle survey, HDL isolation was performed in fresh samples in 35 RUAs, on frozen samples within 14 days in two RUAs, and seven RUAs either stored frozen samples for more than 14 days before HDL isolation, or isolated HDL from fresh and frozen samples. In the final survey, HDL isolation was performed in fresh samples in 30 RUAs, on frozen samples within 14 days in three RUAs, and nine RUAs either stored frozen samples for more than 14 days before HDL isolation, or isolated HDL from fresh and frozen samples.
For cross-sectional comparisons, RUAs which isolated HDL after prolonged storage (LTU-KAU) should be reported. Also, RUAs which store frozen samples can be subject to bias if vials were not kept airtight. Since there are no data available on the characteristics of the tubes used for storage, it is not possible to determine whether storage conditions of frozen samples were adequate. For trend analyses, RUAs which changed storage before isolation of HDL should be reported (FRA-LILa, FRA-TOUa, GER-EGEb, NEZ-AUCa, RUS-NOCa, RUS-NOIa and SWI-VAFa).
According to the MONICA Manual, the determination of HDL cholesterol should preferably be done on the same day as HDL isolation (6). If necessary, storage of the supernatants should not exceed 4 days at 4°C. For longer periods, storage at -20°C or lower is required. Storage should be performed in small glass tubes with leak-proof stoppers to prevent volume and concentration changes.
Table 3 summarises the results for the three survey periods: in the initial survey, 19 RUAs stored samples for less than five days at temperatures around 4°C (max 8°C), 17 RUAs stored samples at -20°C for periods from 1 to 120 weeks before analysis and 10 RUAs performed HDL cholesterol analysis both in refrigerated and frozen samples. Data are missing or incomplete for eight RUAs (BEL-LUXa, DEN-GLOa, GER-RHNa, HUN-PECa, ISR-TELa, ITA-LATa, MLT-MLTa and ROM-BUCa). In the middle survey, 19 RUAs measured HDL cholesterol on samples kept at 4-8°C for less than five days, three RUAs on samples kept at 7°C for less than 10 days, 15 RUAs on frozen samples and seven RUAs both on refrigerated and frozen samples. In the final survey, 23 RUAs measured HDL cholesterol on samples kept at 4-8°C for less than five days, 13 on frozen samples and six both on refrigerated and frozen samples.
Since almost all HDL cholesterol measurements were performed less than one year after blood collection, the magnitude of the bias is expected to be relatively small. For RUAs which analysed HDL cholesterol from frozen samples, a bias can occur if vials were not kept airtight. This also applies for RUAs which assess HDL cholesterol from refrigerated and frozen samples. Again, since there are no data available on the characteristics of the tubes used for storage, no inferences on possible biases due to inadequate storage conditions can be inferred. For trend analysis, no bias is expected if storage conditions were kept constant, whereas RUAs which changed storage conditions between surveys should be noted (FRA-LILa, FRA-TOUa, NEZ-AUCa, SWI-TICa and SWI-VAFa). Also, the very long storage period (120 weeks) in the initial survey for POL-TARa should be kept in mind for trend analysis.
HDL cholesterol measurements are performed after first removing the other lipoproteins from the sample and subsequently measuring the cholesterol content of the remaining HDL-containing fraction. Several precipitation procedures are available for HDL isolation, using one of the following reagents: heparin-Mn++ (hep-Mn++), dextran sulphate-Mg++ (Dxtr), phosphotungstate-Mg++ (PTA) and polyethylene glycol (PEG). However, because no definitive or primary reference methods exist for the separation of HDL, and because differences in the precipitation procedures can alter the population of particles precipitated, not all methods give the same result for HDL cholesterol and therefore standardisation of HDL cholesterol measurement is difficult. In the first version of the MONICA Manual of operations, use of a phosphotungstate-Mg++ (PTA) precipitation method after Burnstein, Samaille and Lopez-Virella was recommended. However, the recommendation was made more than 10 years ago and since then an improved modification of the PTA method (kit No. 543004 of Boehringer Mannheim) has been introduced. This method correlated well with the ultracentrifugation method and it was found in the WHO-RLRC to work well on most lyophilised controls. The description of the modification was included in the MONICA Manual in November 1990 (6).
The reasons that led the MONICA Project to allow the use of any of the HDL isolation methods were the following:
Table 4 summarises the methods used: in the initial survey, 22 RUAs used the PTA method, six RUAs used the modified PTA 543004 method, 15 the Hep-Mn++ method, six RUAs the PEG method, four RUAs the Dxtr method and one RUA both the PTA and the PTA 543004 methods. In the middle survey, 14 RUAs used the PTA method, nine RUAs the modified PTA 543004 method, eight RUAs the Hep-Mn++ method, nine RUAs the PEG method, three RUAs the Dxtr method, and one RUA both the PTA and the Dxtr methods. In the final survey, 12 RUAs used the PTA method, 14 the modified PTA 543004 method, seven RUAs the Dxtr method, four RUAs the Hep-Mn++ method and three RUAs the PEG method. Data are missing for two RUAs (SWI-TICa and SWI-VAFa).
The precipitation technique should be reported for cross-sectional and longitudinal studies. The comparison of six different methods for isolating HDL showed heparin-Mn++ and polyethylene glycol 6000 to give comparable results with a slope close to one and a zero intercept, while the dextran sulphate-Mg++ method had the largest proportional and constant bias with respect to those two methods (9). Hence, RUAs which shifted to/from dextran sulphate-Mg++ to/from another technique (CAN-HALa, SWE-GOTa and USA-STAa) should be reported. No major difference is expected for RUAs which did not change the precipitation technique or which changed the precipitation technique (other than dextran sulphate-Mg++) between surveys.
Some reagents and/or instruments may give results which are systematically higher or lower than those obtained with the reference method (10, 11), this phenomenon is called "matrix effect" and has been extensively discussed in Section 5.4.2 of the quality assessment of total cholesterol measurement report (1). Also, automated methods may prove more reliable and reproducible than manual ones. According to MONICA Manual (6), an enzymatic cholesterol method is preferred. However, other methods may be used (RUAs standardised with CDC or local research).
Table 4 indicates that in the initial survey, 27 RUAs used an enzymatic automated assay for HDL cholesterol, six RUAs an enzymatic manual assay, seven RUAs a direct automated assay, three RUAs a direct manual assay, 10 RUAs an extraction automated assay, and one RUA an enzymatic assay without indicating if automated or manual. In the middle survey, 26 RUAs used an enzymatic automated assay, four RUAs an enzymatic manual assay, six RUAs a direct automated assay, two RUAs a direct manual assay, three RUAs an extraction automatic assay, and two RUAs an extraction and an enzymatic assay, both automated. One RUA used an enzymatic assay (automated or manual). In the final survey, 36 RUAs used an enzymatic automated assay, one RUA an enzymatic semi-automated assay, two RUAs an enzymatic manual assay, and one RUA a direct and an enzymatic automated assay. Data are missing for two RUAs (SWI-TICa and SWI-VAFa).
The changes in the middle and final surveys were essentially towards automation and the use of enzymatic assays. In the final survey, practically all RUAs were using enzymatic assays. As for the modifications in the precipitation techniques, this will facilitate external quality assessment and reduce between-method differences.
Calibration has been considered the most important issue in determining inaccuracy of HDL cholesterol assay. According to the MONICA Manual (standardization of lipid measurements, subsection 5) (6), each participating laboratory is responsible for its own analytical primary standards and/or secondary serum or plasma calibrators. Its is expected that during the surveys each laboratory has used the best and most appropriate pure substances and reagents. It must be stressed that the WHO-RLRC distributed during the pre-standardization period a set of standards and fresh serum samples to laboratories for testing linearity over the HDL cholesterol working range.
On the other hand, it has been stated (2,12) that all cholesterol analytical systems should be calibrated with fresh patient specimens. This procedure, if adopted, may have produced discrepancies with the results obtained by using secondary serum/plasma calibrators. Those discrepancies are said to be due to matrix effect, which has been extensively discussed in Section 5.4.2 of the quality assessment of total cholesterol measurements (1). It was recommended that any discrepancies between fresh samples and WHO-RLRC standards encountered by the participating laboratories be reported for assessment.
HDL cholesterol measurement was optional at the beginning of the MONICA Study. Hence, some RUAs either did not assess HDL cholesterol, or did so in a limited subsample. It is therefore necessary to assess the availability of HDL cholesterol data for all RUs.
Table 5 indicates that in the initial survey, available data for >=95% of the survey respondents is found for 30 RUAs, while 25 RUAs had HDL cholesterol data for less than 95% of survey respondents. Six RUAs (GER-ERFa, GER-KMSa, GER-RDMc, HUN-BUDa, HUN-PECa and ROM-BUCa) had the availability of data for HDL cholesterol under 50%. For GER-KMSa and GER-RDMc, the reason was that some RUs did not measure HDL cholesterol, thus decreasing the percentage of available data in the corresponding RUA (RUs 15, 16 and 18 in GER-KMSa and RUs 22 and 26 in GER-RDMc). As previously stated, GER-ERFa did not perform HDL cholesterol measurements in the first survey and so the percentage is zero. In the middle survey, available data for >=95% of the survey respondents is found for 34 RUAs, while 10 RUAs had HDL cholesterol data for less than 95% of the survey respondents. In the final survey, available data for >=95% of survey respondents is found for 31 RUAs, while 12 RUAs had HDL cholesterol data for less than 95% of the survey respondents. Most RUAs increased the percentage of the sample with HDL cholesterol values from the initial to the final survey; decreases of at least 10 percentage points in the availability of HDL cholesterol values were found for BEL-CHAa (61% initial survey - 51% final survey), CAN-HALa (91% initial survey - 75% final survey) and YUG-NOSa (98% initial survey - 81% final survey). The low availability of data for BEL-CHAa and BEL-GHEa is due to the fact that the survey procedure was conducted in two steps: in the first step the participants responded to the questionnaires and in the second step (conducted later) they had their blood drawn. Since a significant proportion of attendees in the first step did not attend the second in all surveys, this explains the low availability of data for those two RUAs.
The survey core data records submitted to the MDC had three data items related to HDL cholesterol (13):
When these data were received in the MDC, the data were checked routinely for the following constraints:
(Note the following interpretations: DHDL 99MMYY = 28MMYY and DHDL 9999YY = 3112YY).
All violations of these constraints were reported to the MCC for their correction or elucidation. Data values outside the constraint limits were acceptable, but the MCC had to check that the values were not unusual owing to data errors. The MCCs were asked to correct values only if they were incorrect.
The results are reported in Appendix 1. There are unresolved constraint violations for ITA-LATa and ROM-BUCa (initial), for BEL-CHAa, GER-BERa, GER-COTa, GER-HACa, GER-KMSa and ITA-BRIa (middle) and for FRA-STRa, FRA-TOUa, GER-ERFa, ITA-FRIb and RUS-NOCa (final).
HDL cholesterol measurement devices are usually calibrated using different concentration points, through which a regression line is drawn. The regression equation will then be used to estimate HDL cholesterol values according to the readings of the device. According to MONICA Manual (6), results obtained in mmol/l should be calculated and reported to two decimals and results in mg/dl should be calculated and reported to one decimal. Usually, one should expect the last digit of the HDL cholesterol value to follow a random fluctuation. Systematic deviations from randomness would then indicate that a rounding procedure was performed before reporting the data.
Table 6 shows that in the initial survey terminal digit preferences were observed in nine RUAs (GER-BERa, GER-EGEb, GER-HACa, GER-KMSa, GER-RDMc, HUN-BUDa, ROM-BUCa, SWE-NSWa and SWI-TICa) whereas a preference towards even digits was noted in three other RUAs (ITA-FRIb, POL-TARa and SWE-GOTa). In the middle survey, terminal digit preferences were observed in seven RUAs (AUS-NEWa, AUS-PERa, AUS-PERb, FRA-TOUa, GER-COTa, HUN-BUDa and SWE-NSWa). In the final survey, terminal digit preferences were observed in six RUAs (AUS-PERa, AUS-PERb, NEZ-AUCa, SWE-GOTa, SWI-TICa and SWI-VAFa) and a preference towards even digits was noted in one RUA (NEZ-AUCa). Most digit preferences were towards numbers 0 and 5, although two RUAs in the middle survey (FRA-TOUa and HUN-BUDa) showed a preference towards other digits.
Since most of the data were rounded during the assessment of the HDL cholesterol levels by the MCCs, it is not possible to correct them. The RUAs were contacted to assess the type of rounding, but not all of them responded (see Section 10). If the rounding was adequate, it will not cause bias. The reason for the preference for even numbers is not known.
The external quality control was performed to evaluate the analytic stage of the measurement procedure. Comparability of the results achieved by different laboratories in "cross sectional" designs and by different surveys in "longitudinal" designs should be evaluated. In particular, different combinations of Reporting Units (RU) may be used for cross-sectional and trend analyses if all RUs of the population were not included in all three or in two surveys.
Before starting the measurement of the samples, each participating laboratory was expected to analyse at least one set provided by the WHO-RLRC with known concentrations of HDL cholesterol. This self-evaluation period enabled the participating laboratory to check whether their analyses were within the limits of acceptability for every control pool before starting measurements. After this "open" period, a "blind" EQC system was set to evaluate laboratory performance during the entire survey analysis period. During the survey analysis period, the WHO-RLRC provided each participating laboratory with control sets containing either one, two or three pool(s) with different HDL cholesterol ranges (designated as "low", "middle" and "high" ranges). Each set contained about 20 samples, sufficient for about 2 months' measurements, and the participating laboratories had to process the samples according to specified instructions. Laboratories were requested to complete analysis of individual sets and report results within 2 months, at the latest (which unfortunately did not work in a number of cases). Failure to do so would mean that a further set would not be received, since only the receipt of results was a signal to the WHO-RLRC to indicate that a further set should be sent.
For assessing EQA some pools have been excluded because of evidence of matrix effects as well as problems in the reconstruction of the pools. This was evidenced when determinations were performed for the confirmatory values which were performed at a later stage.
The WHO-RLRC provided regular information to participating laboratories on the results of each control set (means, biases, between-run, within-run and overall standard deviations). According to the MONICA Manual, for any control pool, calculated bias (based on the EQC set mean and on the reference value) should be no greater than 7.5%. At the same time standard deviation should be smaller than 6.5%. Reference values were obtained by WHO-RLRC using the Lopes-Virella and the Boehringer Mannheim GmbH kit No. 543003 phosphotungstate-Mg++ methods. The kit method was shown to agree with the proposed post-ultracentrifugation heparin-Mn++ CDC reference method and appeared to give the same results as the ultracentrifugation and the heparin-Mn++ precipitation method used by some MCCs.
For the purposes of the quality assessment in this report, we define:
The APs and RSs for each survey are listed in Table 7.
Coverage relates to the proportion of survey AP which was covered by external quality control analysis. Ideally, it should be calculated as the number of HDL cholesterol analyses performed during the external quality control analysis divided by the number of all survey HDL cholesterol analyses. In practice, this score proved very difficult to calculate, and a more pragmatic index was built using the following proxies: average time of AP per relevant set and maximum gap. AVERAGE TIME (in months) of AP per RS is the ratio between AP and RS. Since the lengths of the analysis periods vary a lot, and the control sets are not always evenly distributed throughout the period, the AVERAGE TIME of AP per RS should be examined by including the additional index of the maximum gap.
Maximum-Gap (MAX-GAP) has been defined as the maximum length of time (in months) from a survey sample analysis to the nearest RS. In occasional situations the MAX-GAP may become extremely wide even if the actual coverage is adequate. This happens, for instance, when there is an isolated short part of the AP not covered by RSs, and there are EQC sets considered irrelevant because they were performed more than one month before the start of the isolated AP. Such situations are rare, however, and may be easily detected.
AVERAGE TIME of AP per RS and the MAX-GAP, a COVERAGE SCORE was calculated according to the following simple algorithm:
| COVERAGE SCORE = | 2+ | if AVERAGE TIME of AP per RS<= 6 months and MAX-GAP <= 4 months; |
| 2 | if AVERAGE TIME of AP per RS <= 9 months and MAX-GAP <= 10 months, but COVERAGE SCORE is not 2+; | |
| 1 | if AVERAGE TIME of AP per RS> 9 months or MAX-GAP is > 10 months, but not both; | |
| 0 | if AVERAGE TIME of AP per RS> 9 months and MAX-GAP is > 10 months or there are no relevant sets. |
COVERAGE SCORE 2+ should be considered an optimal score. If a population scores 0, the EQC results should be considered cautiously when evaluating EQA, because its AP was not sufficiently covered by RSs.
Table 8 gives the results of the COVERAGE SCOREs for the three MONICA surveys. In the initial survey, 26 RUAs achieved a COVERAGE SCORE of 2+, 11 RUAs a coverage score of 2, eight RUAs a score of 1 and nine RUAs a score of 0. In the middle survey, 36 RUAs achieved a COVERAGE SCORE of 2+, two RUAs a score of 2, one RUA a score of 1 and five RUAs a score of 0. In the final survey, 28 RUAs achieved a COVERAGE SCORE of 2+, five RUAs a score of 2, two RUAs a score of 1 and seven RUAs a score of 0. The summary of the COVERAGE SCOREs is given in Table A. Overall, there was n improvement in coverage from the first to the middle and final surveys. The relatively lower percentage of RUAs achieving a coverage score of 2+ in the final survey is due to the high frequency of RUAs with zero scores due to missing data.
| Coverage score |
Number of RUAs | Proportion of RUAs (%) | ||||
|---|---|---|---|---|---|---|
| Ini | Mid | Fin | Ini | Mid | Fin | |
| 2+ | 26 | 36 | 28 | 48 | 82 | 66 |
| 2 | 11 | 2 | 5 | 20 | 5 | 12 |
| 1 | 8 | 1 | 2 | 15 | 2 | 5 |
| 0 | 9 | 5 | 7 | 17 | 11 | 17 |
| Total | 54 | 44 | 42 | 100 | 100 | 100 |
The limit specified for the coefficient of variation of total and HDL cholesterol in the MONICA Lipid Standardization Manual (6) was derived from the following limit for the standard deviation:
where SD is the standard deviation of the pool and RV is the reference value (the break point 2.586 mmol/l is presumably arbitrary, and is equivalent to 100 mg/dl). This limit is shown as the dashed line ("Old limit") in Figure 1. For the HDL cholesterol the old limit was unreasonably strict. Therefore, we will apply here a looser limit for the low reference values, specified as
SD =< 0.1×RV0.4
The revised limit is shown as the solid line in Figure 1. The new limit was chosen such that it was close to the old limit for total cholesterol and gave a similar proportion of out-of-limit values for total and HDL cholesterol.
Figure 1: Comparison between the old and the new limit for the standard deviation of HDL cholesterol quality pools. The results for the total and HDL cholesterol pools in the initial and middle surveys are also given.
Variance out % (VAR%) is defined as the proportion of pools with coefficient of variation out of limits, as defined above. A VARIANCE SCORE is then defined as:
The first cut-off (10%) allows for the consideration of some unusual results such as outliers, i.e. 1 outlier in 10 analysed pools of RSs. Moreover, RUAs which analysed many EQC pools are not penalised since the probability of getting an unusual result increases with the number of pools analysed.
BIAS% is defined as the proportion of pools with bias out of limits (i.e. exceeding ± 7.5%). A BIAS SCORE is defined as:
Similar considerations hold as for the VARIANCE SCORE. In particular, the first cut-off (15%) allows one pool out of limit in 7-13 analysed pools, or 2 pools out of limit in 14-19 pools, and so on.
CONSIST is the smallest difference of the maximal and minimal bias over the pools, after 15% of the pools at most were excluded. Even this exclusion allows a minor part of unusual results to be considered as outliers.
| CONSIST SCORE = | 2 | if CONSIST <= 7.5; |
| 1 | if 7.5 < CONSIST <= 15; | |
| 0 | if CONSIST > 15 or there is one pool only |
AVERAGE BIAS of a survey is the average bias (%) of all control pools of a survey after exclusion of 15% of the most extreme pool biases (i.e. the same exclusions as in the definition of CONSIST).
Tables 9 and 10 provide the results for each individual RUA. For the VARIANCE SCORE, in the initial survey, 33 RUAs achieved a score of 2, three RUAs a score of 1, ten RUAs a score of 0, while no score could not be calculated for eight RUAs. In the middle survey, 38 RUAs achieved a VARIANCE SCORE of 2, one RUA achieved a score of 1, one RUA a score of 0, while no score could be calculated for four RUAs . In the final survey, 28 RUAs achieved a VARIANCE SCORE of 2 and seven RUAs a score of 1. No RUAs got a score of 0 and no score could be calculated for seven RUAs. The summary of the VARIANCE SCOREs is given in Table B, indicating an improvement between the initial and the middle surveys, whereas no improvement was observed between the middle and the final surveys.
| VARIANCE SCORE |
Number of RUAs | Proportion of RUAs (%) | ||||
|---|---|---|---|---|---|---|
| Ini | Mid | Fin | Ini | Mid | Fin | |
| 2 | 33 | 38 | 28 | 61 | 86 | 67 |
| 1 | 3 | 1 | 7 | 6 | 2 | 17 |
| 0 | 10 | 1 | 0 | 19 | 2 | 0 |
| npc | 8 | 4 | 7 | 15 | 10 | 17 |
| Total | 54 | 44 | 42 | 100 | 100 | 100 |
npc: not possible to be calculated
For bias, in the initial survey 25 RUAs achieved a BIAS SCORE of 2, five RUAs a score of 1, 16 RUAs a score of 0, while no score could be calculated for eight RUAs. In the middle survey, 26 RUAs achieved a BIAS SCORE of 2, seven RUAs a score of 1 and seven RUAs a score of 0. In the final survey, 22 RUAs achieved a BIAS SCORE of 2, six RUAs a score of 1, seven RUAs a score of 0 and no score could be calculated for seven RUAs. The summary of the BIAS SCORE is given in Table C: no improvement was observed throughout the surveys, with less than 60% of the RUAs achieving a BIAS SCORE of 2.
| BIAS SCORE |
Number of RUAs | Proportion of RUAs (%) | ||||
|---|---|---|---|---|---|---|
| Ini | Mid | Fin | Ini | Mid | Fin | |
| 2 | 25 | 26 | 22 | 46 | 59 | 52 |
| 1 | 5 | 7 | 6 | 11 | 16 | 14 |
| 0 | 16 | 7 | 7 | 28 | 16 | 17 |
| npc | 8 | 4 | 7 | 15 | 9 | 17 |
| Total | 54 | 44 | 42 | 100 | 100 | 100 |
npc: not possible to be calculated
For consistency, in the initial survey 20 RUAs achieved a CONSIST SCORE of 2, 13 RUAs a score of 1, 13 RUAs a score of 0, while no score could be calculated for eight RUAs. In the middle survey, 17 RUAs achieved a CONSIST SCORE of 2, 19 RUAs a score of 1 and four RUAs a score of 0. In the final survey, 23 RUAs achieved a CONSIST SCORE of 2, nine RUAs a score of 1, three RUAs a score of 0 and no score could be calculated for seven RUAs. The summary of the CONSIST SCORE is provided in Table D, showing an improvement in the final survey relative to the initial and the middle surveys.
| CONSIST SCORE |
Number of RUAs | Proportion of RUAs (%) | ||||
|---|---|---|---|---|---|---|
| Ini | Mid | Fin | Ini | Mid | Fin | |
| 2 | 20 | 17 | 23 | 37 | 39 | 55 |
| 1 | 13 | 19 | 9 | 24 | 43 | 21 |
| 0 | 13 | 4 | 3 | 24 | 9 | 7 |
| npc | 8 | 4 | 7 | 15 | 9 | 17 |
| Total | 54 | 44 | 42 | 100 | 100 | 100 |
npc: not possible to be calculated
EQA SCORE is a quality assessment score of a single survey, defined according to the following table:
| VARIANCE SCORE |
BIAS SCORE |
CONSIST SCORE |
EQA SCORE |
|---|---|---|---|
| 2 | 2 | 2 | 2 |
| 2 | 2 | 1 | 2 |
| 2 | 2 | 0 | illogical |
| 2 | 1 | 2 | 1 |
| 2 | 1 | 1 | 1 |
| 2 | 1 | 0 | 0 |
| 2 | 0 | 2 | 0 |
| 2 | 0 | 1 | 0 |
| 2 | 0 | 0 | 0 |
| 1 | 2 | 2 | 2 |
| 1 | 2 | 1 | 1 |
| 1 | 2 | 0 | illogical |
| 1 | 1 | 2 | 1 |
| 1 | 1 | 1 | 1 |
| 1 | 1 | 0 | 0 |
| 1 | 0 | 2 | 0 |
| 1 | 0 | 1 | 0 |
| 1 | 0 | 0 | 0 |
| 0 | 2 | 2 | 1 |
| 0 | 2 | 1 | 0 |
| 0 | 2 | 0 | illogical |
| 0 | 1 | 2 | 0 |
| 0 | 1 | 1 | 0 |
| 0 | 1 | 0 | 0 |
| 0 | 0 | 2 | 0 |
| 0 | 0 | 1 | 0 |
| 0 | 0 | 0 | 0 |
However, if COVERAGE SCORE is zero, then EQA SCORE cannot be more than 1.
EQA SCORE is an overall score with three levels: 2 connotes a good performance, 1 an acceptable performance and 0 an unacceptable performance or the absence of data. RUAs which score 0 should be considered for exclusion from cross-sectional comparisons.
Table 10 gives the EQA SCOREs for each individual RUA. In the initial survey, 18 RUAs had an EQA SCORE of 2, 8 RUAs an EQA SCORE of 1, 20 RUAs an EQA SCORE of 0, while for eight RUAs the EQA SCORE could not be calculated. In the middle survey, 25 RUAs achieved an EQA SCORE of 2, seven RUAs achieved an EQA SCORE of 1, eight RUAs an EQA SCORE of 0 while for four RUAs the EQA SCORE could not be calculated. In the final survey, 20 RUAs achieved an EQA SCORE of 2, seven RUAs an EQA SCORE of 1, eight RUAs an EQA SCORE of 0 while for seven RUAs the EQA SCORE could not be calculated.
For the RUAs for which EQA SCORE could not be calculated because RSs were missing, other available information on the laboratory standardization was reviewed and summarised in Section 10. If there is evidence that the laboratory was likely to be well standardized, EQA SCORE 1 is given in brackets in Table 10. Otherwise, 0 is given in brackets.
The summary of the EQA SCOREs is given in Table E, indicating an increase in the percentage of RUAs which achieved a good EQA SCORE.
| EQA | Number of RUAs | Proportion of RUAs (%) | ||||
|---|---|---|---|---|---|---|
| Ini | Mid | Fin | Ini | Mid | Fin | |
| 2 | 18 | 25 | 20 | 33 | 57 | 48 |
| 1 | 8 | 7 | 7 | 15 | 16 | 17 |
| 0 | 20 | 8 | 8 | 37 | 18 | 19 |
| npc(1) | 3 | 0 | 3 | 6 | 0 | 7 |
| npc(0) | 5 | 4 | 4 | 9 | 9 | 10 |
| Total | 54 | 44 | 42 | 100 | 100 | 100 |
npc: not possible to be calculated
If the EQA SCOREs of a RUA are acceptable in all surveys then, in principle, the quality of the HDL cholesterol data is valid for estimating HDL cholesterol trends. In practice, there might be survey-specific AVERAGE BIASes within limits but in opposite directions, which may produce sizeable effects when assessing HDL cholesterol trends. Hence, to estimate changes of bias over time, two separate parameters have been developed: DELTA BIAS and TREND BIAS. DELTA BIAS has been defined as the absolute difference between the average biases of the two surveys, after exclusion of 15% of the most extreme pool biases (Note, however, that the exclusion was not done if it would have eliminated all pools for some survey). Delta bias may be used only to compare two survey data. TREND BIAS has been defined as the time trend of the biases over all APs, after exclusion of 15% of the most extreme pool biases, adjusted for a 10-year period. It is calculated as the regression coefficient of pool biases against EQA APs. Trend bias has the advantage over DELTA BIAS that it takes into consideration the time span between the two surveys and that it may be adjusted for different pool reference value levels.
By analogy with CONSIST for the individual surveys, POOLED CONSIST is defined for the pairs of surveys as the smallest difference between the maximal and minimal bias over all the pools of both surveys, after 15% of the extreme pools were excluded. A POOLED CONSIST SCORE is defined as:
| POOLED CONSIST SCORE = | 2 | if POOLED CONSIST <= 7.5; |
| 1 | if 7.5 < POOLED CONSIST <= 15; | |
| 0 | if POOLED CONSIST > 15. |
Table 11 presents EQA SCOREs and AVERAGE BIASes for each survey, as well as DELTA BIAS, POOLED CONSIST and POOLED CONSIST SCORE for the RUAs which performed at least two consecutive MONICA surveys and participated actively in the EQC program. The RUAs which have performed only one survey (BEL-LUXa, GER-RDMa, GER-RHNa, ISR-TELa, ITA-LATa, MLT-MLTa, ROM-BUCa, RUS-MOIb and RUS-NOCb) have been excluded from the table because trend analyses cannot be carried out. The results are summarised in Table F, and indicate that only a small proportion of all survey pairs reach a score of 2
| EQA | Number of Pairs | Proportion of Pairs (%) | ||||
|---|---|---|---|---|---|---|
| Mid-Ini | Fin-Mid | Fin-Ini | Mid-Ini | Fin-Mid | Fin-Ini | |
| 2 | 10 | 9 | 8 | 22 | 20 | 20 |
| 1 | 13 | 12 | 14 | 29 | 26 | 35 |
| 0 | 10 | 7 | 7 | 22 | 15 | 18 |
| npc | 8 | 11 | 9 | 18 | 24 | 23 |
| ir | 4 | 7 | 2 | 9 | 15 | 5 |
| Total | 45 | 46 | 40 | 100 | 100 | 100 |
ir: irrelevant; npc: not possible to be calculated
Table 12 shows the TREND BIAS per 10-year period, calculated considering the initial and final surveys or all three surveys, the related standard errors and the p-values for testing the null hypothesis that the trend is zero. As for Table 11, the nine RUAs which carried out just one survey have been excluded. For eight RUAs TREND BIAS cannot be calculated because the initial or the final survey was not carried out (AUS-PERb, GER-BERa, GER-BREb, GER-COTa, GER-HACa, GER-KMSa, HUN-BUDa and HUN-PECa). For eleven RUAs it was not possible to calculate TREND BIAS because, as reported in Table 7, RSs were not available (AUS-PERa, FIN-KUOa, FIN-NKAa, FIN-TULa, NEZ-AUCa, RUS-NOCa, RUS-NOIa, SWI-TICa, SWI-VAFa, USA-STAa). For four RUAs it was possible to calculate TREND BIAS for the initial and final surveys only, because the middle one was not carried out (CAN-HALa, FRA-LILa and FRA-STRa) or because of missing data (SWE-GOTa). Out of the 31 RUAs for which it was possible to calculate TREND BIAS, 8 had values significantly (p<0.05) different from zero (AUS-NEWa, BEL-CHAa, CHN-BEIa, CZE-CZEa, ITA-FRIb, LTU-KAUa, SWE-GOTa, and UNK-GLAa) and 11 RUAs had Ini-Fin or Ini-Mid-Fin absolute values which exceeded 5 (AUS-NEWa, CAN-HALa, CHN-BEIa, GER-EGEa, ICE-ICEa, ITA-BRIa, ITA-FRIb, LTU-KAUa, POL-TARa, UNK-BELa and UNK-GLAa). Most of the RUAs with TREND BIAS values significantly different from zero or whose value was over 5 had at least one EQA SCORE equal to zero (see Table 10). Considering [-5; +5] a secure range for TREND BIAS, it appears unreasonable to include in the trend analysis RUAs with the absolute value of TREND BIAS > 5 or at least one EQA SCORE of zero.
One RUA whose EQA SCOREs was different from zero (ITA-FRIb) required special consideration. This RUA assessed a limited number of RS during the initial survey, which may have produced large AVERAGE BIASes, which in turn produced large values of TREND BIAS. This RUA could be considered for trend analysis. Finally, in some RUAs which performed the three surveys there are discrepancies between TREND BIASes calculated using only the initial and the final surveys (Ini-Fin) and using all three surveys (Ini-Mid-Fin) (GER-EGEb, ICE-ICEb and POL-TARa); taking into account the middle survey decreases the magnitude of the TREND BIAS for GER-EGEb and ICE-ICEb, whereas it increases for POL-TARa). But since in those RUAs at least one of the surveys used for calculating TREND BIAS has an EQA SCORE of zero, those RUAs should not be considered for trend analysis.
To summarise the quality of HDL cholesterol data for trend analysis, HDL Cholesterol Overall Summary Score (HCOSS), consisting of HDL Cholesterol Pre-Analytic Summary Score (HCPASS) and HDL Cholesterol External Quality Assessment Summary Score (HCEQASS) was developed. The score gets a high value for the RUAs with evidence of good quality and a lower value for the RUAs where such evidence is inconsistent or lacking.
HCPASS will be defined according to four subscores, taking into account the four major pre-analytic sources of variability:
A Storage Before Isolation Score (SBIS) is defined as the combination of two sub-scores, SBIFRESHS and SBIFROSTS
| SBIFRESHS = | 2 | if isolation performed only in fresh samples in all surveys OR isolation performed only in frozen samples in all surveys; |
| 1 | if isolation performed in fresh and frozen samples; | |
| 0 | if data on isolation are not available. |
For frozen samples:
| SBIFROSTS = | 2 | if max.time is 14 days or less AND temperature is -20°C or less in all surveys; |
| 0 | if max.time > 14 days in at least one survey OR temperature > -20°C in at least one survey OR max.time or temperature is missing for at least one survey. |
If only fresh samples were used in all surveys, SBIFROSTS is defined as "2".
Then:
| SBIS = | 2 | if SBIFRESHS = 2 AND SBIFROSTS = 2; |
| 1 | if SBIS is not 2 or 0; | |
| 0 | if SBIFRESHS = 0 OR SBIFROSTS = 0 |
A Storage After Centrifuging Score (SACS) is defined using two sub-scores, RRS and DFS:
For samples stored using refrigeration or at room temperature or adopting other referred procedures:
| RRS = | 2 | if max.time is less than 5 days, in all surveys; |
| 1 | if max.time is between 5 and 10 days, in at least one survey; | |
| 0 | if max.time is more than 10 days or max.time is missing, in at least one survey. |
For frozen samples:
| DFS = | 2 | if temperature is between -60°C and -20°C and max.time at most 1 year,
in all surveys OR if temperature is -60°C or less, in all surveys; |
| 1 | when DFS is not 2 or 0; | |
| 0 | if temperature is at least -20°C and max.time is more than 1 year, in at
least one survey OR if temperature if more than -15°C, in at least one survey OR if temperature or max.time is missing. |
If samples were not frozen, DFS is defined as 2.
Then:
A Posture of Blood Drawing Score (PBDS) is defined as:
| PBDS = | 2 | if blood samples were drawn from sitting subjects, in all surveys OR blood samples were drawn from lying subjects, in all surveys; |
| 1 | if up to 60% of blood samples were drawn with subjects in different positions (i. e. sitting or lying) in different surveys | |
| 0 | if more than 60% of blood samples were drawn with subjects in different
positions (i.e. sitting or lying), in different surveys OR when information is missing in any survey. |
A Plasma-Serum Difference Score (PSDS)is defined as:
| PSDS = | 2 | if serum was used in all surveys OR plasma, using the same anticoagulant, was used in all surveys; |
| 1 | if serum was changed to Heparin-plasma or vice versa OR heparin was changed to EDTA or vice versa OR EDTA-plasma was changed to serum between the surveys; |
|
| 0 | if serum was changed to EDTA-plasma
between the surveys OR the information is missing in any survey. |
A Pre-Analytic Summary Score (HCPASS) is then defined as:
| HCPASS = | (SBIS+SACS+PBDS+PSDS-4)/2 | if SBIS>0 and SACS>0 and PBDS>0 and PSDS>0; |
| 0 | if SBIS=0 or SACS=0 or PBDS=0 or PSDS=0. |
The External Quality Assessment Summary Score (HCEQASS) summarises the quality of the analytic part of HDL cholesterol measurements. Its components are: COVERAGE SCORE, EQA SCORE, AVERAGE BIAS and TREND BIAS. It is calculated taking into account the initial and the final surveys, and the middle one when available.
| HCEQASS = | 2 | if EQA SCORE = 2 for all surveys AND |AVERAGE BIAS| =< 3.0% AND |TREND BIAS| =< 3.0% AND COVERAGE SCORE = 2 in all surveys; |
| 1 | if not 2 or 0; | |
| 0 | if |EQA SCORE| = 0 for at least
one survey OR |TREND BIAS| > 5%, |
where | | denotes the absolute value. In the cases where EQA SCORE was not possible to calculate, the score in brackets in Table 10 was used for calculating HCEQASS.
Finally, the HDL Cholesterol Overall Summary Score (HCOSS) is defined as:
The values of the summary score for trend analysis are presented in Table 13. The table includes only those RUAs with at least the initial and final survey data. For each RUA, scores have been calculated considering the final and the initial surveys. Separate scores have been provided when middle survey data are available. Each score has three levels: 2 indicates that MONICA requirements are fully met, 1 attests that MONICA requirements are not fully met, but no relevant deviation occurred; 0 states that major deviation from MONICA requirements occurred which affects trend analysis. Twenty seven RUAs presented major deviations from MONICA requirements in both trend analyses (Ini-Fin and Ini-Mid-Fin), 3 had at least one zero score for one of the trends (GER-BREa, ITA-FRIc and SWE-GOTa), 7 had a score of 1.25 for both trends, one had a score of 1.5, and 2 RUAs (SPA-CATa and YUG-NOVa) had scores 1.5 and 2. Among the 26 RUAs which had a HCOSS of zero, the most frequent cause was a zero value for the HCEQASS (25 RUAs), followed by a zero value for the HCPASS (12 RUAs).
HDL cholesterol was an optional biochemical measurement in the MONICA survey.
Nevertheless, it was measured in almost all RUAs. As for
total cholesterol, standardization of HDL cholesterol measurements was covered by an EQC which provides quantitative results.
The results of the EQC were summarised in an EQA SCORE, which used information on the coverage of the
EQC as well as the variation and bias of the individual EQC pools. If a laboratory shows a
good performance in the EQC, it is very likely that its measurements are reasonably
accurate. On the other hand, if the quality of a laboratory is considered unacceptable,
there is still the possibility that the quality is good, but for some reason the coverage
of the EQC was poor or the laboratory had problems in handling of the quality control
pools, but not of the actual survey samples.
Figures 2 and 3 present the absolute biases and coefficients of variation respectively for
all EQC pools analysed in all the participating
laboratories and after excluding RUAs with an EQA SCORE
of zero. For coefficients of variation, the results indicate a slight improvement, mainly
for RUAs which had scores different from zero. Those results are in agreement with those
of Table B. Conversely, for absolute biases, no improvement was
detected, as already indicated in Table C. Those findings would
indicate (a) that most RUAs did not manage to improve the quality of their HDL cholesterol
measurements or (b) that it is not possible to achieve a better standardization with the
current criteria.
Bias of a pool was considered to be out of limit if its absolute value was more than
7.5%. This limit appeared to be quite tight because only a minority of the RUAs achieved
an acceptable EQA SCORE in all three surveys. On the
other hand, considering the fact that the population mean of HDL cholesterol is usually
between 1 and 1.5 mmol/l and the between population standard error is about 0.1 mmol/l
(i.e. less than 10% of the population mean), the 7.5% bias limit is fairly large for
useful detection of differences in mean HDL cholesterol between populations or trends
within populations. This suggests that there is a contradiction between the accuracy that
is reasonable to achieve in such population surveys and the requirements of meaningful
assessment of the population mean values.
Nevertheless, it appears that the limit applied by the CDC
in their external quality control was even wider (10%). (Note, however, that for the EQA SCORE in this report, 7.5% limit was applied also
for the RUAs which were stantardised by CDC.) For comparison, the EQA SCORE using the 10% limit for bias was also
calculated, but it made a negligible difference to the results.
In the quality assessment of total cholesterol measurements (1),
criteria for the correction of the total cholesterol data for the bias detected in the
external quality control were defined. For HDL cholesterol, correction criteria could not
be applied because of the many potential sources of bias in the the analysis of the EQC
sets.
In summary, the EQC results indicate that standardization
of HDL cholesterol measurements was difficult to achieve and did not improve significantly
during the MONICA Project. This can be partly due to (a) the different isolation and
measurement procedures used by the participating laboratories, (b) the changes in HDL
isolation procedures, (c) the changes in participating laboratories during the study
period, (d) the use of lyophilised control pools, which increase the probability of
undetectable matrix effects, and (e) the inability of some of the laboratories to maintain
a sufficiently high quality of the procedures applied. Whether or not a much better
standardization of HDL cholesterol measurements is feasible in such a multinational
setting, remains an open question. The observed trend towards the acceptance of a limited
number of HDL isolation and measurement procedures by most participating laboratories is
likely to improve the comparability of data between RUAs in future studies.
The findings of the quality assessment should be considered carefully when deciding on the inclusion of the RUAs in data analyses. Different types of analyses will have different quality requirements, but here are general recommendations for analyses which compare the levels of or trends in HDL cholesterol between the RUAs:
The following list includes only the RUAs with specific findings or exceptional background information relevant for the use of the data, and the RUAs for which additional clarification or correction of data are expected.
AUS-NEW
AUS-PER
BEL-CHA
BEL-GHE
BEL-LUX
CAN-HAL
CHN-BEI
CZE-CZE
DEN-GLO
FIN-KUO, FIN-NKA and FIN-TUL
FRA-LIL
FRA-STR
FRA-TOU
GER-AUR and GER-AUU
GER-BER
GER-COT
GER-EGE
GER-ERF
GER-HAC
GER-KMS
GER-RDM
GER-RHN
HUN-BUD
HUN-PEC
ICE-ICE
ISR-TEL
ITA-BRI
ITA-FRI
ITA-LAT
LTU-KAU
MLT-MLT
NEZ-AUC
POL-TAR
POL-WAR
ROM-BUC
RUS-NOC and RUS-NOI
SWE-GOT
SWE-NSW
SWI-TIC
SWI-VAF
UNK-BEL
UNK-GLA
USA-STA