Study design: what’s in a name?

in European Journal of Endocrinology
View More View Less
  • 1 Departments of Clinical Epidemiology, Leiden University Medical Center, Leiden, the Netherlands
  • 2 Endocrinology, Leiden University Medical Center, Leiden, the Netherlands
  • 3 Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands

Correspondence should be addressed to O M Dekkers; Email: o.m.dekkers@lumc.nl

The name of the study should properly reflect the actual conduct and analysis of the study. This short paper provides guidance on how to properly name the study design. The first distinction is between a trial (intervention given to patients to study its effect) and an observational study. For observational studies, it should further be decided whether it is cross-sectional or whether follow-up time is taken into account (cohort or case–control study). The distinction prospective-retrospective has two disadvantages: prospective is often seen as marker of higher quality, which is not necessarily true; there is no unifying definition that makes a proper distinction between retrospective and prospective possible.

Abstract

The name of the study should properly reflect the actual conduct and analysis of the study. This short paper provides guidance on how to properly name the study design. The first distinction is between a trial (intervention given to patients to study its effect) and an observational study. For observational studies, it should further be decided whether it is cross-sectional or whether follow-up time is taken into account (cohort or case–control study). The distinction prospective-retrospective has two disadvantages: prospective is often seen as marker of higher quality, which is not necessarily true; there is no unifying definition that makes a proper distinction between retrospective and prospective possible.

For new-borns, parents can choose, within certain limits, whatever name they consider appropriate and nice. This is, however, not the case when naming the design of a research project, as here the name should properly reflect the actual conduct and analysis of the study. And even though reporting guidelines like STROBE (1) provide guidance, design mislabelling still occurs. In the present paper, we propose four sequential questions (see also Fig. 1), the answers to which will guide the proper naming of the design.

Figure 1
Figure 1

Flow-chart to determine the design of a clinical study.

Citation: European Journal of Endocrinology 183, 6; 10.1530/EJE-20-0873

Is it a trial or an observational study?

The first question is whether the study assesses the effect of an intervention. If not, it is not a trial. If an intervention is studied, the next question is, whether the intervention was given to patients with the aim to study its effect or whether the study merely reflects the assessment of actual care. In the first case the study should be labelled as trial, in the second case the study is observational. For example, the effect of levothyroxine in subclinical hypothyroidism can be studied by randomizing patients to levothyroxine or placebo (trial), or using data from routine care (observational study). The difference can also be viewed from an ethical point of view: in a trial, patients should consent to be treated, in an observational study the patient should consent to have their data used for research purposes.

In a strict sense, a trial does not necessarily includes a control group, and also randomization is not a distinguishing feature of a trial; so called single arm trials lack both a control group and randomization. An example of a single arm trial would be a study of Cushing’s disease with a new and fancy drug, where cortisol levels are compared before and after 12 weeks treatment.

Although we generally tend to think of interventions as drugs, or surgical procedures, the range of potential interventions is much broader and also diagnostic strategies can be studied as interventions. For example, in MEN 1 syndrome, one could compare mortality and quality of life between family members who are tested for the gene mutation and family members who are not tested. In principle, such testing could also be studied in a randomized trial design.

Does the study take follow-up time into account?

The next question for classification of the study design is whether the study assesses the relation between exposure (e.g. potential risk factors) and outcome (e.g. development of disease) at one point in time, or whether that relation is studied longitudinally. For example, growth hormone (GH) levels can be related to glucose levels measured at the same time, or can be related to glucose levels (or diabetes status) after a specified follow-up period. Studies that assess a relation at one moment in time are called cross-sectional. If follow-up time is taken into account, the study is longitudinal (being either a cohort or a case–control study, see below). Clinical trials are always longitudinal.

It is relevant to distinguish the timing of the actual measurement of exposure and outcome, from the question how exposure relates to the outcome in terms of follow-up time. Consider the question whether smoking is a risk factor for adrenal incidentalomas. One could ask all incidentaloma patients, at time of diagnosis, about past smoking behaviour. Although measurements of exposure and outcome are made at the same time, the study takes follow-up time into account, as it aims to measure past smoking behaviour (before the development of the incidentaloma). A similar reasoning applies to variables that can reasonably be assumed stable over time (genes, sex); however, if GH levels (exposure) are related to glucose levels measured at the same time, the study is better labelled cross-sectional, since GH levels cannot be assumed to be stable over time.

it is important to note that it is the actual analysis that determines the answer to the question above. It can be the case that the relationship between GH and glucose levels are analysed based on baseline data from a randomized controlled trial. In this case, the original design of data collection may suggest that the study is longitudinal, yet since all baseline measurements are made at the same time, the study that answers this particular relation is in fact cross-sectional, despite the fact that the data come from a trial. Similarly, if laboratory data used for analysis are collected at one point in time from all Cushing patients in a hospital, this should be classified a cross-sectional study, even though researchers may be inclined to call their group of study patients a cohort. Diagnostic test accuracy studies can be seen as a specific type of cross-sectional studies.

Are individuals sampled into the study based on exposure or based on outcome?

GH levels, or acromegaly, are often studied as potential risk factors for colon cancer. Researchers have two options when designing their study: they can compare patients with the exposure (high GH levels) to patients unexposed (normal GH levels) and assess whether the risk of developing colon cancer is higher in exposed patients. They can also start by sampling patients with the outcome (colon cancer) and compare their exposure levels (for example GH levels) to exposure levels of subjects without the outcome (i.e. without colon cancer). In the first situation, comparing outcome risk between groups of exposed and unexposed subjects, the design is called a cohort study. In the second situation, comparing levels of exposure between patients with and without the outcome, the design is a case–control study (see (2) for details).

Especially the term 'case–control study' is frequently used incorrectly. Often, so called case–control studies are in fact cohort studies; whereas also true case–control studies go without proper naming. The reason for this ‘case–control confusion’ (3) stems likely from the fact that clinicians think of cases as persons with a disease. In a cohort study assessing the potential increased risk for infections in Cushing’s disease, for example, compared to people without Cushing’s disease, the fact that patients with Cushing’s disease are considered ‘clinical cases’ does not classify the study as a case–control study. In fact, it classifies as a cohort study.

The defining feature of a cohort study is thus that individuals are followed over time and the risk for (developing) the outcome is estimated for a group of exposed subjects only, or is estimated for a group of exposed as well as a group of unexposed individuals. This means that a control group is not a necessary feature of a cohort study (4). Assessing the risk of acromegaly recurrence after surgery classifies as cohort study, even when no control group is included.

Prospective or retrospective does it matter?

Cohort studies are often labelled as either retrospective or prospective. However, this retrospective–prospective distinction has two disadvantages: prospective is often seen as marker of higher quality, which is not necessarily true; but more fundamentally, there is no unifying definition that makes a proper distinction between retrospective and prospective possible (1).

Often, the implicit definition of a retrospective design is that the study idea and conception of the study come after the data have been collected. The prototype of such retrospective study would for example be a cohort study assessing the association between GH levels and diabetes occurrence where researchers have to look in historic patient records to extract GH data. Along this line, also an analysis based on data from a randomized trial in which GH levels were measured and later related to diabetes occurrence, would need to be classified as retrospective if the idea for this additional analysis was not prespecified in the protocol. However, the data quality will likely differ between the two approaches: non-standardized GH measurements with missing data in the former vs standardized GH measurements and protocolized follow-up in the latter example. This shows that retrospective is not necessarily a marker for low quality data collection. In countries where almost all medical data is routinely collected in large databases (for e.g. in Denmark (5)) the distinction retrospective–prospective is also non-informative.

Better than to use the retrospective–prospective distinction, is to describe in detail how the study was designed, how the data were collected, and what potential weaknesses of the study are. Such a fully transparent approach is the best guarantee that the reader can judge the quality of the data and the conduct of the study.

Declaration of interest

O M D is a Deputy Editor for European Journal of Endocrinology. He was not involved in the review or editorial process for this paper, on which he is listed as an author.

Funding

This work was supported by grants from the Netherlands Organization for Scientific Research (ZonMW-Vidi project 917.16.430) and the LUMC.

References

  • 1

    Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, Poole C, Schlesselman JJ & Egger M Strengthening the reporting of observational studies in epidemiology (STROBE): explanation and elaboration PLoS Medicine 2007 4 e297. (https://doi.org/10.1371/journal.pmed.0040297)

    • Search Google Scholar
    • Export Citation
  • 2

    Vandenbroucke JP & Pearce N Case-control studies: basic concepts International Journal of Epidemiology 2012 41 1480148 9. (https://doi.org/10.1093/ije/dys147)

    • Search Google Scholar
    • Export Citation
  • 3

    Grimes DA ‘Case-control’ confusion: mislabeled reports in obstetrics and gynecology journals. Obstetrics and Gynecology 2009 114 1284128 6. (https://doi.org/10.1097/AOG.0b013e3181c03421)

    • Search Google Scholar
    • Export Citation
  • 4

    Dekkers OM, Egger M, Altman DG & Vandenbroucke JP Distinguishing case series from cohort studies. Annals of Internal Medicine 2012 156 3740. (https://doi.org/10.7326/0003-4819-156-1-201201030-00006)

    • Search Google Scholar
    • Export Citation
  • 5

    Frank L Epidemiology. When an entire country is a cohort. Science 2000 287 2398239 9. (https://doi.org/10.1126/science.287.5462.2398)

If the inline PDF is not rendering correctly, you can download the PDF file here.

 

     European Society of Endocrinology

Sept 2018 onwards Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 101 101 101
PDF Downloads 39 39 39
  • 1

    Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, Poole C, Schlesselman JJ & Egger M Strengthening the reporting of observational studies in epidemiology (STROBE): explanation and elaboration PLoS Medicine 2007 4 e297. (https://doi.org/10.1371/journal.pmed.0040297)

    • Search Google Scholar
    • Export Citation
  • 2

    Vandenbroucke JP & Pearce N Case-control studies: basic concepts International Journal of Epidemiology 2012 41 1480148 9. (https://doi.org/10.1093/ije/dys147)

    • Search Google Scholar
    • Export Citation
  • 3

    Grimes DA ‘Case-control’ confusion: mislabeled reports in obstetrics and gynecology journals. Obstetrics and Gynecology 2009 114 1284128 6. (https://doi.org/10.1097/AOG.0b013e3181c03421)

    • Search Google Scholar
    • Export Citation
  • 4

    Dekkers OM, Egger M, Altman DG & Vandenbroucke JP Distinguishing case series from cohort studies. Annals of Internal Medicine 2012 156 3740. (https://doi.org/10.7326/0003-4819-156-1-201201030-00006)

    • Search Google Scholar
    • Export Citation
  • 5

    Frank L Epidemiology. When an entire country is a cohort. Science 2000 287 2398239 9. (https://doi.org/10.1126/science.287.5462.2398)