5. DESIGN ELEMENTS AND DATA SOURCES FOR CLINICAL STUDIES: ICH E8(R1) Guideline

Study objectives impact the choice of study design and data sources, which in turn impact the strength of a study to support regulatory decisions and clinical practice. As discussed in Section 4, there are a wide variety of study objectives in drug development. Similarly, there is a wide range of study designs and data sources to address these objectives. Sections 5.1 through 5.6 discuss key elements that may be used to define the study design, and Section 5.7 discusses the various data sources that may be used for the study.

Clear objectives will help to specify the study design, and conversely, the process of specifying the design may help to further clarify the objectives. At the design stage, the objectives may need to be modified if substantial practical considerations and limitations or other risks to critical to quality factors are identified. The study objectives are further refined through specification of estimands. Estimands, discussed in ICH E9(R1) Addendum: Statistical Principles for Clinical Trials, provide a precise description of the treatment effects reflecting the clinical questions posed by the study objectives. The estimand summarises at a population level what the outcomes would be in the same patients under the different treatment conditions being compared.

An important distinction between studies is whether the allocation of individuals to the study drug(s) is controlled by the study procedures or allocation to the drug is not controlled but exposure to the drug(s) is observed in the study. In this document, the former case is referred to as an interventional study and the latter case is referred to as an observational study.

Interventional studies, and in particular randomised studies, play a central role in drug development, as they can better control biases. The designs of randomised studies range from simple parallel group designs to more complex variants. For example, adaptive design studies allow prospectively planned modifications to the study, such as changes in the population studied or changes in doses of the drug studied over the course of the study, based on accumulating data. Master protocol studies allow for the investigation of multiple drugs or multiple conditions under a shared framework. Platform studies allow for multiple drugs to be investigated in a continuous manner, with different drugs entering the study at different times and leaving the study based on pre-specified decision rules.
Studies without randomisation (whether interventional or observational) can play a role as well in certain settings when randomisation is not feasible. Observational studies are often conducted post-approval but can be of utility as complementary sources of evidence during development and across the life cycle of a drug.

Along with the breadth of study designs, there are multiple sources of data that studies may employ. Traditionally, studies have used study-specific data collection processes. Data such as that obtained from electronic medical records or digital health technologies may be leveraged to increase the efficiency of studies or generalisability of study results.
This section presents important elements that define the design of a clinical study including population, treatment, control group, response variable, methods to reduce bias, statistical analysis, and data sources. It is intended to assist in identifying the critical to quality factors necessary to achieve the study objectives, while also enabling flexibility in study design and promoting efficiency in study conduct. Although the focus is on interventional studies, the discussion is intended to apply to both interventional and observational studies. The elements outlined here are expected to be relevant to study types and data sources that are used in clinical studies now and that may be developed in the future.

5.1 Study Population

The population to be studied should be chosen to support the study objectives and is defined through the inclusion and exclusion criteria for the study. The degree to which a study succeeds in enrolling the desired population will impact the ability of the study to meet its objectives.

The study population may be narrowly defined to reduce the risk to study participants or to maximise the sensitivity of the study for detecting a certain effect. Conversely, it may be broadly defined to more closely represent the diverse populations for which the drug is intended. In general, studies conducted early in a development programme, when little is known about the safety of the drug, are more homogeneous in study population definitions. Studies conducted in the later phases of drug development or post-approval are often more heterogeneous in study population definitions. Such studies should involve participants who are representative of the diverse populations which will receive the intervention in clinical practice. Available knowledge about participant characteristics that may predict disease outcomes or effects of the intervention can be used to further define the study population.

The number of participants (sample size) in a study should be large enough to provide a reliable answer to the questions addressed (see ICH E9). This number is usually determined by the primary objective of the study. If the sample size is determined on some other basis, then this should be made clear and justified. For example, a sample size determined to address safety questions or meet important secondary objectives may need larger numbers of participants than needed for addressing the primary efficacy question (see ICH E1). If study objectives include obtaining information on certain subgroups, then efforts should be made to ensure adequate representation of these subgroups.

5.2 Treatment Description

The treatment(s), including controls, under study should be described explicitly and specifically. These might be individual treatments (including different doses or regimens), combinations of treatments, or no treatments, and can include specification of background treatments. The definition of treatments should align with the objectives of the study (ICH E9(R1)). For example, if the objective of the study is to understand the effect of the treatment in clinical practice, the study may specify that the background treatment, if any, is up to the discretion of the participants and healthcare providers. If the objectives are to understand the effect of the drug when added to a specific background treatment, the background treatment should be defined explicitly and specifically for all groups including controls.

5.3 Choice of Control Group

The major purpose of a control group is to separate the effect of the treatment(s) from the effects of other factors such as natural course of the disease, other medical care received, or observer or patient expectations (E10 Choice of Control Group in Clinical Trials). The treatment effect of interest may be the effect relative to not receiving the drug or the effect relative to receiving other therapies. Comparisons may be made with placebo, no treatment, standard of care, other treatments, or different doses of the drug under investigation.

The source of control group data may be internal or external to the study. The intent of using an internal control group is to help ensure that the only differences between treatment groups are due to the treatment they receive and not due to differences in the selection of participants, the timing and measurement of study outcomes, or other differences. A special case of an internal control group is when each participant serves as their own internal control by receiving the drug and control at different points of time. With use of an external control group, individuals are selected from an external source, and the individuals may have been treated at an earlier time (historical control group) or during the same time but in another setting than participants in the study.

Important limitations of the use of external controls are discussed in ICH E10. Particular care is needed to minimise the likelihood of erroneous inference. The use of an external control requires that the disease course is well known and predictable. External control individuals may differ from study participants with respect to demographic and background characteristics (e.g., medical history, concurrent diseases). In addition, external control individuals may differ from participants in the study with respect to concurrent care and the measurement of study outcomes and other data elements. Because the use of internal controls generally mitigates the potential for bias better than external controls, particularly in conjunction with randomisation, the suitability of the use and choice of external control should be carefully considered and justified. Section 5.5 discusses the sources of bias which can arise in observational studies and is relevant to the use of external controls.

Participant level data may not be available for some choices of external control groups. Summary measures may be available to form the basis of comparisons with treated participants to estimate drug effects and test hypotheses about those effects. There is, however, less ability to control for differences in characteristics between study individuals in the external control group and study participants in the internal treatment groups in making these comparisons or examining the quality and completeness of individual data elements. Additionally, there may not be the ability to examine subgroups or modify the response variable to be consistent with the response variable used in the study.

5.4 Response Variables

A response variable is an attribute of interest that may be affected by the drug. The response variable may relate to pharmacokinetics, pharmacodynamics, efficacy, or safety of the drug, or to the use of the drug including, for example, in adherence to risk minimisation measures post- approval. Study endpoints are the response variables that are chosen to assess drug effects.

The primary endpoint should be capable of providing clinically relevant and convincing evidence related to the primary objective of the study (ICH E9). Secondary endpoints are either supportive measurements related to the primary objective or measurements of effects related to the secondary objectives. Exploratory endpoints are used to further explain or to support study findings or to explore new hypotheses for later research. The choice of endpoints should be meaningful for the intended population and may also take into account the views of patients. The definition of each study endpoint should be specific and include how and at what time points in a participant’s treatment course of the drug and follow-up it is ascertained.

Knowledge of the drug, along with the clinical context and purpose of a given study affect what response variables should be collected. For example, a proof-of-concept study of relatively short duration may employ a pharmacodynamic outcome rather than the outcome of primary interest (ICH E9). A larger study of longer duration could then be used to confirm a clinically meaningful effect on the outcome of primary interest. In other cases, such as a study where the safety profile of the drug is well characterised, the extent of safety data collection may be tailored to the objectives of the study.

5.5 Methods to Reduce Bias

The study design should address potential sources of bias that can undermine the reliability of results. Although different types of studies are subject to different sources of bias, this section addresses some common sources. ICH E9 discusses principles for controlling and reducing bias mainly in the context of interventional studies.

In studies with internal control groups, randomisation is used to ensure comparability of treatment groups, thereby minimising the possibility of bias in treatment assignment.

Randomisation at the start of the study addresses differences between the groups at the time of randomisation but does not prevent bias due to differences arising during the study. Events after randomisation (particularly intercurrent events (ICH E9(R1)) may affect the validity and interpretation of comparisons between treatment groups. Examples include treatment discontinuation or use of rescue medications. There may also be differences in the follow-up patterns between the groups due to participants in one group discontinuing the study at different rates, because of, for example, adverse events or perceived lack of efficacy. Careful consideration of the potential for intercurrent events to occur during the study and their impact will help with the identification of critical to quality factors, such as reducing study discontinuation, continuing data collection following treatment discontinuation, and retrieving data after study discontinuation, if appropriate. It is important when defining the treatment effect (estimand) to account for the occurrence of intercurrent events.

Concealing the treatment assignments (blinding) limits the occurrence of conscious or unconscious bias in the conduct and interpretation of a clinical study that may affect the course of treatment, monitoring, endpoint ascertainment, and participants’ responses. In a single-blind study the investigator is aware of the treatment, but the participant is not. When the investigators who are involved in the treatment or clinical evaluation of the participants are also unaware of the treatment assignments, the study is referred to as double-blind. In an open- label study, the consequences of the lack of blinding may be reduced through the use of pre- specified decision rules for aspects of study conduct, such as recruitment, treatment assignment, participant management, safety reporting, and response variable ascertainment. Blinding for staff at the study sites or sponsor should be implemented where feasible.

Knowledge of interim results (whether individual or treatment group level) has the potential to introduce bias or influence the conduct of the study and interpretation of study results. Specific considerations related to information flow and confidentiality are therefore necessary.

Observational studies introduce unique challenges to the assessment and control of bias. These include ensuring that the individuals have the condition under study and ensuring comparability between treatment groups, in prognostic factors associated with the choice of therapies, in the ascertainment of response variables, and in post-baseline concomitant patient care. These challenges may also exist with the use of external controls in an interventional study. Methods exist that may mitigate some of these challenges and should be considered during the design phase.

5.6 Statistical Analysis

The statistical analysis of a study encompasses important elements necessary to achieving the study objectives. The specification and documentation of the statistical analysis are important for ensuring the integrity of the study findings. The principal features of the statistical analysis should be planned during the design of the study and should be clearly specified in a protocol written before the study begins (ICH E9). Full details of the planned statistical analysis should be specified and documented before knowledge of the study results that may reveal the drug effects, which may be accomplished using a separate statistical analysis plan. The protocol should define the estimand(s) following the framework established in ICH E9(R1).

Statistical analyses of primary and secondary endpoints that address key study objectives with respect to both efficacy and safety should be described in the protocol, including any interim analyses and/or planned design adaptations. Other statistical aspects of the study that should be described in the protocol include the analytical methods for any planned estimation and tests of hypotheses about the drug effect and a justification of the sample size.

The statistical analysis should include pre-specified sensitivity analyses for assessing the impact of the assumptions made for the primary and important secondary analyses on the results of the study (E9(R1)). For example, if the analysis relies on a particular assumption about the reasons for missing data, sensitivity analyses should be planned to assess the impact of that assumption on the study results. In the case of observational studies, sensitivity analyses might, for example, consider additional potential confounders.

For double-blind studies, the statistical analysis should be finalised before treatment assignments are revealed. Therefore, if a study includes one or more interim analyses, the planned statistical analysis should not be changed after an interim analysis that involves unblinding. For open-label and single-blind studies, details pertaining to the primary and important secondary analyses would ideally be finalised before the first participant is randomised or allocated to study intervention.

Pre-specification of the analysis approach is particularly important for studies that make use of existing data sources rather than primary data collection (Section 5.7), not only for the statistical analysis planned for the study but also for any feasibility analysis to assess the applicability of the existing data. For example, for a single-arm interventional study with an external control, the specifics of the external control should be defined prior to the conduct of the interventional aspect of the study. Pre-specification of the analysis should be in place so that any review of the existing data sources prior to the design of the study does not threaten the study integrity.

The statistical analysis should be carried out in accordance with the prospectively defined analysis plan, and all deviations from the plan should be indicated in the study report (E3 Clinical Study Reports).

5.7 Study Data

Study data comprise all information generated, collected, or used in the context of the study ranging from existing source data to study-specific assessments. The study data should contain the necessary information to conduct the statistical analysis specified in the protocol and statistical analysis plan, as well as to monitor for participant safety, protocol adherence, and data integrity.

Study data can be broadly classified into two types: (1) data generated specifically for the present study (primary data collection) and (2) data obtained from sources external to the present study (secondary data use). Data generated for the study may be collected via case report forms, laboratory measurements, electronic patient reported outcomes, or mobile health tools. Examples of external sources of data include historical clinical studies, national death databases, disease and drug registries, claims data, and medical and administrative records from routine medical practice. A study may make use of both types of data.

For all data sources, procedures to ensure the protection of personal data of the individuals being studied should be implemented. The study protocol, and if applicable the informed consent, should explicitly address the protection of personal data. Regulations related to protection of individuals’ data need to be followed. When considering data from external sources, it is important to ascertain whether the regulatory authorities accept the use of such data for purposes other than the original intent.
Study data should be of sufficient quality to address the objectives of the study and, in interventional studies, to monitor participant safety. Data quality attributes include consistency (uniformity of ascertainment over time), accuracy (correctness of collection, transmission, and processing), and completeness (lack of missing information). These aspects should be proactively considered during study planning by identifying the factors, critical to the quality of the study, associated with data sourcing, collection, and processing.

The use of standards for data recording and coding (or recoding) is important to support data reliability, facilitate correct analysis and interpretation of results, and promote data sharing. Internationally accepted data standards exist for many sources of study data and should be used where applicable.

With primary data collection, the methods and standards established for use at the point of capture and the subsequent processing provide an opportunity to prospectively ensure the quality of the data.

With secondary data use, the relevance of the available data should be considered and clearly described in the study protocol. For example, when using existing electronic health record data to ascertain the study endpoint rather than through primary data collection, information in the health record about outcomes may need to be converted to the study endpoint.

In some cases, secondary data use may not be sufficient for all aspects of the study and may need to be supplemented by primary data collection. The quality of data collected for a different purpose should be evaluated when re-used in the context of the present study. Careful quality control processes may have been applied during their acquisition; where used, those processes were not necessarily designed with the objectives of the present study in mind.

There are several additional considerations with secondary data use. For example, methods to conceal the treatment should be considered when selecting and prior to analysing data from external sources. As another example, absence of affirmative information on a condition or event does not necessarily mean the condition or event is not present. There may also be a delay between the occurrence of events and their appearance in existing data sources. To the extent possible, uncertainties and potential sources of bias should be addressed at the study design stage, during data analysis, and in the interpretation of the study results.