Louise Parmenter, PhD, Quintiles

Fig. 1. Quality, time and cost elements of major research approaches.

		Observational research—evaluating outcomes in patients in a naturalistic rather than an experimental setting–has the potential to answer many research questions. But in order to do so effectively, researchers must select “the right research approach for the right question.” Doing so requires thorough evaluation of feasibility of delivery, specialized operational approaches with an understanding of pitfalls to avoid, and an upfront plan to communicate the results to stakeholders for whom the results are relevant.

		The starting point for selecting “right research/right question” is to refine the question into something specific that can be transformed into a viable study. The research question represents the uncertainty that the sponsor wants to resolve by performing the study. Sometimes, the research question is confused with the research interest, which is a broader concept. To refine the research question requires understanding the strategic objectives of any research, and reviewing the current body of literature and any ongoing research to ensure that the question is relevant, specific and novel.

		The right research approach is determined by practical considerations such as time and budget, ethical considerations and the strength of evidence required to meet healthcare stakeholder needs. Various study design options exist, with varying strengths and limitations that can be evaluated against these considerations. For example, a randomized controlled trial (RCT), while providing the highest strength of evidence, might cost too much and take too long. There may be ethical questions concerning randomization. A cross-sectional study conducted in a database might be quick and cheap, but might not provide sufficient certainty of evidence for decision-making. All of these factors should be weighed against each other to determine the right research approach.

		The feasibility of a study is a key ingredient in its ability to generate the right quality evidence in the right time at the right cost. There are several research approaches to choose from (Fig. 1). The quality of data generated via each research approach varies, as does the time and cost to generate that evidence. In general, the time needed to gather data prospectively is longer than approaches using real-world data sources, and the cost of research can vary from less than $100,000 to tens of millions of dollars. RCTs and prospective observational cohort studies provide the most certainty of evidence for decision-makers, but take longest and cost most. In contrast, research using routine data sources—while relatively quick and cost-effective–may not be considered sufficient by some decision-makers.

		The various types of research approaches—such as prospective observational cohort studies and patient registries, chart reviews, database studies, and hybrid designs–have different features, advantages and disadvantages.

	For example, features of prospective observational cohort studies include the fact that they are non-randomized and non-interventional, and they monitor cohorts over time. Patient registries may focus on disease, product or exposure. Advantages include the fact that prospective observational cohort studies and patient registries can:

			Examine longer-term outcomes in populations typically excluded from trials

			Examine risks for uncommon harms & factors that modify risk

			Obtain more representative data on a range of outcomes

			Be more cost-effective than randomized trial designs

			Assess actual use (including off-label) to identify potential new indications.

		Disadvantages of prospective observational cohort studies and patient registries include the fact that they are prone to bias and confounding. Optimal design, conduct and analysis are critical for producing strong evidence.

		Chart reviews and database studies use existing clinical or administrative data present in patient paper charts or electronic databases. Advantages of using existing data sources include the ability to assess benefits and harms across an extremely large population, and the fact that this is a cost- and time-efficient approach compared to prospective, longitudinal research. Disadvantages are:

			Underlying information is not collected in a systematic way

			It is difficult to interpret missing data which may introduce bias

			Non-randomization of patients to treatment introduces the possibility of confounding

			Data abstraction from paper records is resource-intensive

			Complete medical and clinical histories may not be available

			Administrative databases and electronic health records are not uniformly available across all countries

			Privacy issues may create the need to aggregate data.

Fig. 2. Advantages and disadvantages of hybrid designs.

		Hybrid designs can capitalize on the speed, cost-efficiency and strengths of different methods, using a combination of primary and secondary data to shorten time and lower costs (Fig. 2). With this approach, sub-studies—focusing on health economic data or patient reported outcomes–are conducted alongside larger prospective studies, providing added value.

	For all types of study design, strategies to ensure a feasible study include:

 Much of the data cited as difficult to provide to decision-makers around the time of launch can be captured during development, or at least planned for during Phase 3.

Multifunctional collaboration within companies:

 This allows the right scientific, commercial and operational input into study designs.

 This guides research questions, which in turn determine data needs. To gather required data demands an understanding of where these may exist in routine data sources or whether they must be gathered prospectively.

		Importantly, observational research requires a different operational approach from that used in experimental clinical research. It is nonetheless common for companies to use clinical development teams to conduct observational research. Observational research requires specialized people, processes and technology in four major areas:

 should be informed by stakeholder needs in terms of endpoint definition, design and execution.

Data sources, data tools and data elements

 should be selected with the needs of observational approaches in mind. Data need to be available and accessible. For a truly observational study, data should be captured as part of routine care and should not be requested purely for the purpose of the study. If a database is being used, it should be accessible and any necessary permissions obtained for data use. Most technologies for data capture in clinical research were designed with clinical trials in mind. Companies should consider technologies compatible with usual care. Selection of data elements is another area requiring a different approach from that for clinical trials. Potential for confounding and bias—which can invalidate the results of observational studies–must be understood. Failure to capture data elements critical to a full analysis of a study may lead to false conclusions; epidemiologists should be employed to advise teams on the selection of the right data elements.

Site and patient recruitment and retention

 are key areas requiring specialized approaches. For many observational studies, it is important that sites are representative of the target population of sites. The starting point to achieve a representative sample of sites is to understand the target population profile. For example, to understand primary versus secondary care treatment, rural versus urban distribution, and for product registries, prescribing trends. Randomizing a site target list may appear an attractive choice but can lead to an insufficient site list once site interest has been determined. More appropriate is to determine site interest from the universe of sites and to back this list into the ideal site sample. Ensuring a representative sample of patients can also prove challenging when site enrollment may be uneven. Strategies such as patient caps might be employed but should be carefully balanced against time and cost considerations. Recruitment and retention strategies may be employed on observational studies but options are more limited than for clinical trials due to the need to avoid the introduction of bias.

Statistical analysis, interpretation and reporting

 are also important. Specialized methodological approaches are critical to ensure appropriate use of analytical techniques and interpretation of that analysis.

		In practice, observational research requires the full integration of strategy, science and operations throughout the lifecycle of every study. Regulatory and commercial teams are needed to provide global and local insights into evidentiary needs and the tools for evidence dissemination. Epidemiologists and outcomes researchers are involved in writing the study protocol; reviewing data management and biostatistical deliverables; leading site and patient sampling; reviewing data for missing elements, bias and confounding; and interpreting, analyzing and reporting data throughout the study. Epidemiological input is needed to all aspects of operational delivery to minimize bias and confounding. Dedicated observational research operational teams work on specialized observational study delivery processes, and leveraging fit-for-purpose data capture technology ensure high-quality delivery.

Articles

Observational Research: Choosing the right research approach for the right question

Fig. 1: Review of first 24 Months of Pharmacovigilance Risk Assessment Committee Oversight

that high-quality observational research is needed to meet real-world evidence needs. In late 2014, Quintiles polled 135 biopharma customers to ask what they considered to be the most challenging aspect of providing evidence to health technology assessment (HTA) decision-makers. There were three response choices: understanding the diversity of stakeholder needs, varying real-world data access and availability, and generating quality evidence at the right time at the right cost. Most customers chose generating quality evidence at the right time at the right cost to be their greatest challenge.

		However, while the need for quality is broadly acknowledged, its definition in the context of observational research is not always clear. This article offers a framework designed to deliver consistent quality in observational research based on five key features:

			Study design that provides data relevant to all key healthcare system stakeholders

			Feasible based on the availability of time, budget and data

			Specialized in operational delivery, weaving together strategy, science and operations

			Results are communicated to stakeholders before, during and after study completion.

		Why “quality” not “quantity”? There’s tremendous excitement in healthcare over the growing availability of “Big Data” including administrative claims and electronic medical records. In parallel, observational research leveraging secondary and primary data sources is becoming more widely used by healthcare decision-makers. As such, the expectation of what observational research can deliver is huge, and it has never been more important to deliver high-quality observational research to meet the demand for real-world evidence. However, the challenges inherent in observational research remain: selection bias, information bias, confounding, and unreliable or missing data. Delivering studies based on large numbers of people using poor-quality data may, at best, waste healthcare spend and, at worst, lead to poor healthcare decisionmaking.

Fig. 2: Key evidence requirements of payers

		Research by biopharma companies must meet the information needs of all healthcare stakeholders, including regulators, payers, physicians and patients.

		Regulators—with a primary focus on the detection of safety signals and ensuring long-term effectiveness–require well-designed, scientifically sound studies. A Quintiles review of decisions made by the European Medicines Agency’s Pharmacovigilance Risk Assessment Committee (PRAC; Fig. 1) indicates that many Post-Authorisation Safety Study (PASS) protocols have been rejected due to issues such as: inadequate study design to meet objectives, sample size, statistical approach, missing timelines, enrollment strategy, representativeness, bias and confounding, and inappropriate design. In the US, big data has been a mainstay of drug safety using programs such as the MiniSentinel, which relies largely on administrative health insurance claims data and some electronic medical record information, but also has the capacity for supplementary prospective data collection. Further, FDA has shown its interest in prospective observational data as evidenced by granting line extensions based on registry data. For example, a recent line extension within the same extension was granted for bevacizumab using data from one clinical trial and two registry-based studies.

		Payers bring a different perspective, one focused on determining value and coverage, monitoring usage within specific criteria, and ensuring cost effectiveness. HTA bodies have a common aim of determining the value of new technologies; however, there is heterogeneity among the agencies in how they operate and how different types of information influence decisionmaking. The balance between central and local decision-makers varies by market, as does the primary notion of value. This is demonstrated by a survey conducted by the European network for Health Technology Assessment, EUnetTHA, which found that all payers want to know whether a drug works and is safe, and many also examine value (Fig. 2). Real-world evidence is often required either as part of longer term post-market safety/efficacy monitoring or for conditional coverage and reimbursement re-evaluation. In addition, real-world evidence is needed for targeting differentiation and uptake at regional payer level or with providers who are increasingly involved in financial management and care pathways/efficiency. In the US, there is also a longstanding interest in quantitative evidence generation to support new trends that are starting to examine not just whether a particular treatment or diagnostic works, but how much of a difference it makes. For example, the National Oncologic PET Registry was used to evaluate the impact of PET scanning on cancer diagnosis and how much it affected physician decisionmaking. [1]

Fig. 3: Payer demands for benefit over standard of care [2]

		Payers also want to see superiority over the existing standard of care. A recent analysis of HTA reports in non-small cell lung cancer (NSCLC) identified a strong preference for payers to demand benefit over existing standard of care in the primary endpoint (Fig. 3).

		Publicly available data from UK National Institute for Care Excellence (NICE) oncology assessments indicate that payers are looking for hard endpoints, such as overall survival versus progression-free survival for oncology products. Payers also want to see evidence in the right patient population, with differences between the trial population and the requested reimbursement indication being the most frequently reported critique. Comparator selection is also critical for this stakeholder group.

		In terms of evidence demands, physicians wish to obtain locally relevant evidence, advance science, improve patient care, ensure continued reimbursement and generate publications. Patients want to understand their treatment options, the associated risks and benefits, impacts on quality of life, convenience of administration, and affordability.

		These diverse information needs should be reflected in biopharma companies’ research strategy, as they work to build a drug’s safety profile, evaluate efficacy to improve patient outcomes, prove value, secure reimbursement, enhance understanding of unmet patient needs, explore new indications, generate publications, and inform the product’s brand strategy.

		Due to their observational nature, epidemiological studies in the past were widely regarded as not raising any significant ethical issues and were commonly carried out without approval of an ethical review committee. This is no longer the case and the International Ethical Guidelines for Epidemiological Studies, prepared in 2009, call out the need to submit observational studies to ethical committees for review. [3] Prepared by the Council for International Organizations of Medical Sciences (CIOMS) in collaboration with the World Health Organization (WHO), the guidelines cover both observational and interventional research. They were designed for use, particularly by low-resource countries, in defining national policies on the ethics of biomedical research (and particularly in clinical trials of pharmaceuticals).

		Observational studies usually require scientific and ethical review, although some observational studies, such as those utilizing publicly available or anonymous data, may not be subject to prior ethical review and approval under local regulations. If in doubt about whether a study needs ethical review, sponsors are encouraged to consult an ethical review committee or to submit their studies for review. Any observational study that is not submitted to an ethical review committee should receive administrative confirmation by a competent authority that the study is exempt from review. Also, physicians participating in observational studies should receive incentives commensurate with fair market value for the work performed, and payments should not represent an inducement to prescribe.

		While observational research normally does not pose a risk of physical harm to individuals, this is not always the case. For example, in some nonexperimental studies, researchers may take blood or tissue samples; and even when an observational study involves only questionnaires or examination of medical records, subjects may be at risk of physical or psychological harm. Informed consent is needed when the research poses a risk for patients, and is normally required if the study involves data containing personal identifiers.

		When populations and communities with limited resources are involved, any research should be responsive to the health needs and priorities of the participating population or community. In addition, any intervention, product developed or knowledge generated, must be made reasonably available for the benefit of these participants. Study sponsors should be aware that if a patient is switched from an affordable standard of care to an unaffordable new treatment in an observational study, the research may be characterized as exploitative and, therefore, unethical.

		There is an ethical obligation to disseminate any study findings that are of potential scientific or public health importance. This should take place in a timely fashion through publications in the scientific literature and presentations at scientific conferences, workshops or symposia. It should be noted that scientific journals generally require that papers submitted for publication have received prior review by an ethical review committee. Authorship of study reports should follow the guidelines established by the International Committee of Medical Journal Editors, [4] and any potential conflicts of interest, financial and nonfinancial, should be disclosed. Agreement to adhere to these guidelines should be included in the study protocol.

			Louise Parmenter, PhD is Global Head of Operations, Epidemiology & Outcomes Research, at Quintiles.

					Larson SM. Practice-Based Evidence of the Beneficial Impact of Positron Emission Tomography in Clinical Oncology. JCO May 1, 2008 vol. 26 no. 13 2083-2084; 

http://jco.ascopubs.org/content/26/13/2083.full

http://www.ufrgs.br/bioetica/cioms2008.pdf

The right studies at the right time in the right way to meet real-world evidence needs