CLINICAL TRIALS - Why Are Metrics Important in Starting Clinical Trials?

INTRODUCTION

This question may seem counter intuitive, as we are exposed almost daily to the dire performance of clinical trials and their spiraling costs resulting from incurred delays. According to a recent study by KPMG,¹ within the pharmaceutical industry, the return on R&D expenditure has fallen from an industry average of approximately 20% 20 years ago, to 10% now, with the average cost of developing a drug rising during that period at a rate 7.4% higher than inflation, with the increasing costs of conducting clinical trials responsible for most of this increase.² It is estimated that it now costs upward of $2 billion dollars to bring a new drug to market.³

And perhaps most distributing is the fact that cycle time associated with starting clinical trials, ie, steps involved in study startup (SSU), such as, the selection of investigate sites at which to conduct the study, and activation of the site to receive first subject, have not changed in more than 2 decades.⁴ According to the Tufts Center for the Study of Drug Development (CSDD), 37% of sites selected for clinical trial studies under-enroll, and 11% fail to enroll a single subject. Eventually, 89% of studies meet enrollment goals, but often at the expense of sponsors faced with doubling the original timeline due to poor enrollment.⁵ At a time when it takes an estimated 8 months to move from pre-visit through to site initiation,⁶ with the associated cost of initiating one site ranging from $20,000 to $30,000.⁷ Overall, poor site selection, the inability of sites to predict the rate of enrollment, and the subsequent need for study rescue may increase cost of trials by 20% or more.⁸

Against this backdrop, it is clear that metrics are indeed critical to efforts to rein in clinical trials that are either poorly initiated or have incurred unforeseen events, which place the original timelines and/or budgets at risk of overages. They also drive competitive performance among those organizations performing trials.

LEVERAGING BUSINESS INTELLIGENCE IN CLINICAL TRIALS

Metrics provide the foundation for business intelligence (BI), affording clinical research teams an opportunity to intervene before the effects of a risk have been occurred. Risk mitigation is therefore optimal, using systems that can provide timely, preferably real-time data on trial bottlenecks, which indicate red flags to be reviewed and addressed or at least tracked carefully throughout the trial.

BI has become an increasingly popular topic in clinical trials as clinical project managers⁹ are expected to make smarter decisions on intelligence derived from clinical trial data, and sponsors/contract research organizations (CROs) are looking for ways to incorporate BI into the eClinical systems they are using to empower oversight — turning raw trial data into actionable information.

By 2020¹⁰ 72% of clinical trials are anticipated to be outsourced, up from just 23% in 2012. With this in mind, technology that can provide sponsors with real-time insights into clinical operations is essential. This technology should also provide CROs with automated alerts for workflows and sponsors with multiple reporting options, including on-demand static reports, snap-shot reports with status data that can be manipulated for further analysis, and full access.

Having technology that can automate or assist in the timely monitoring of trials is a significant improvement over the current “status quo” of manual methods, such as spreadsheets, which are cumbersome and erroneous, not to mention only provide a dated snap-shot of trial performance.

BUT HOW DO METRICS DRIVE PERFORMANCE COMPETITIVENESS?

Benchmarking of trial data allows clinical research teams to gauge their performance and progress against internal data, as well as externally run trials. It allows them to see at a glance if they are on par with past trials the organization has run of a similar size, geographic footprint, therapeutic area, indication, etc. If not, why not? But equally as important, and maybe arguably more so, is how is the clinical research team performing against other organizations? This is particularly important in the case of a CRO vying for a pharma outsourced study contract or for pharma’s needing to justify the continued outsourcing relationship. A review of benchmarking data may indicate red flags not otherwise raised during the monitoring of the trial, and may be country specific. But benchmarking is not without its challenges.

“Benchmarks should be generated from standardized, well-conceived data elements and performance metrics,” said Linda Sullivan, Co-founder and President at the industry group Metrics Champion Consortium. “Additionally, the data needs to have sufficient metadata associated with it so you can make meaningful comparisons and correlate benchmark results with best practice outcomes.”

From an internal perspective, organizations can capture cycle-time metrics on whichever artifacts they deem important to measure, and as long as these metrics have clear definitions and are measured consistently between trials, then these measurements become internal benchmarks, upon which future trials can be gauged. But for external purposes, allowing organizations to gauge performance against one another, clear, consistent, and concise industry-wide standards are required. This ensures a true “apples to apples” comparison that has the added benefit of improving trial data quality, because data that might not have been previously recorded, such as certain start or stop dates, is now required. Negative cycle times or cycle times that are outliers should be reviewed to ensure accurate data entry.

With standards in place that can be milestones need to be utilized. Global milestones are important because they recognize that the nomenclature of artifact naming conventions is not consist across organizations, or even countries, and nor will it ever be. For example, these are dependent on an organization’s SOPs in which the events Activated, IP Release, and Site Initiated could be synonymous. Nevertheless, what is important is that these cycle-time metrics can be accurately measured and mapped to an industry defined standard.

Applying industry standards and global milestones for clinical and operational data, the goal of external benchmarking in clinical trials is achievable. But is this the end of the story? No, in reality it is just the beginning…

MOVING BEYOND OPERATIONAL & REGULATORY METRICS

Benchmarking allows for gamification, which could be extended beyond country or CRO (if the trial is outsourced to multiple vendors) to be based on role assignment, with associated financial incentives. Some might view this option as unethical or raise questions of quality, of individuals potentially gaming the system to reap the rewards and accolades of exceeding the threshold of industry performance for their position. But nevertheless, it is a logical progression, and many of these arguments don’t carry much weight in a system with numerous checks and balances. Moreover, it allows for greater transparency in the process of conducting clinical trials and would allow management the opportunity to highlight those Clinical Research Associates (CRAs) and others that are star performers.

Gamification in the pharmaceutical industry has been used to improve relationships with patients by using games to encourage disease management, with Sanofi¹¹, Boehringer Ingleheim, and Eli Lilly¹², developing apps. In the context of clinical trials, gamification presents an excellent opportunity to improve performance and reduce costs. There are a number of areas that hold promise, including patient recruitment, patient and key clinical staff retention, disease research, investigator and site training, and improving site performance.

Sponsors and CROs are looking to innovative methods to improve site performance, such as rewarding sites with badges as they pass certain predetermined milestones (eg, 10 patients screened, all training completed) or using leaderboards to show sites how they are performing relative to their peers. Principal investigators are motivated by watching their site on the leaderboard to see how they rank on key metrics, such as patient enrollment and data query resolution. Meanwhile, exposing clinical operations teams to metrics, leaderboards, and other activities as they undertake their daily work can potentially improve both performance and quality. And inspiring friendly competition can motivate global site performance areas, such as activation, patient enrollment, and more.

Take the case of T.J. Sharpe. Faced with the prospect of a Stage IV Melanoma diagnosis back in 2012, he vowed to never give up, determined to see his two young children grow up with a father. Working with his oncology team, T.J. identified the best possible treatment option, which at the time came in the form of a promising immunotherapy clinical trial nearly 4 hours away from his home. After packing up his family and relocating to Tampa, he learned shortly after his arrival that the trial was delayed due to a pending signature on a clinical trial agreement (CTA), a contract. Without the luxury of time on his side, T.J.’s new oncologist suggested that he may have to consider “plan B.” Determined not to let a document sitting on someone’s desk get in the way of a potentially life-saving treatment for him or for other patients in the study, T.J. got to work. After a long process, which involved contacting the trial sponsor, finding the right person, and telling his story, the study team resolved the startup hurdle in a timely enough manner for T.J. to receive treatment. After a long journey, which included participation in a second clinical trial, today, T.J. is in remission and healthier than ever. T.J.’s story is regrettably all too common – incentives could reduce these unnecessary delays.

Benchmarking would also allow for efficient resource allocation. A review of subpar performance may indicate that this is simply due to staffing issues, affording executives the option of either allocating more staff to critical steps in the progress, called crashing the schedule¹³ in project management terminology or opting to incur the subsequent financial ramifications from a delayed launch to the market. Ultimately, sponsors stand to lose up to $8 million daily due to a trial delaying a product’s development and launch.¹⁴

Lastly, benchmarking is the precursor to predicative analytics or forecasting, enabling clinical research teams to estimate future outcomes based on their current state of progress. This is critical to risk mitigation and a preemptive weapon in the fight against the dreaded rescue study.

INDUSTRY-DRIVEN ADOPTION

CROs, often seen as the bastions of innovation in clinical trials, are leading this charge into the BI foray. Top CROs have been aggressively acquiring data sources to leverage in data mining. In 2013, PPD acquired Acurian¹⁵ to gain analytics-driven feasibility capabilities, LabCorp acquired Covance¹⁶ for collective data resources to drive greater R&D productivity, and Quintiles merged with IMS Health¹⁷ last year to improve clinical trial execution using patient data.

Informatics is the new frontier in their innovative efforts, as they look to gain insights in operational data to drive improvements via targeted enrollment efforts. A theme that was central to the recent Disrupting Clinical Operations from the CRO Perspective¹⁸ presentation at DPharm 2016 by top level CRO executives. What is the common thread? We now operate in a data-driven environment. Follow T.J.’s story at www.philly.com/patient1/.

REFERENCES

1. Radical changes needed in pharmaceutical industry to improve patient outcomes, KPMG LLP UK Press Release, August 4, 2014.
2. Rapidly rising clinical trial costs worry researchers. Canadian Medical Association, February 3, 2009. Website accessed: http://www.cmaj.ca/content/180/3/277.full.pdf+html.
3. Examination of Clinical Trial Costs and Barriers for Drug Development. US Department of Health and Human Services. July 25, 2014. Available at: https://aspe.hhs.gov/sites/default/files/pdf/77166/rpt_erg.pdf.
4. Getz K, Lamberti ML. 89% of trials meet enrollment, but timelines slip, half of sites under-enroll. Impact Report. Tufts Center for the Study of Drug Development. Jan/Feb2013;15(1).
5. Getz K, Lamberti ML. 89% of trials meet enrollment, but timelines slip, half of sites under-enroll. Impact Report. Tufts Center for the Study of Drug Development. Jan/Feb2013;15(1).
6. Kramer JM. Schulman KA. Transforming the economics of clinical trials. Institute of Medicine. April 13, 2012.
7. START Study Tufts CSDD-goBalto, 2012. Ken Getz’s presentation entitled: Uncovering the drivers of R&D costs.
8. Unclogging the Patient Recruitment Bottleneck, PharmaVoice, Feb. 2011. Website visited: http://www.pharmavoice.com/article/2182/.
9. Business Intelligence In Clinical Trial Management, Clinical Leader, May 12, 2015 http://www.clinicalleader.com/doc/business-intelligencein-clinical-trial-management-0001.
10. Research and Markets: The New 2015 Trends of Global Clinical Development Outsourcing Market, January 20, 2015. Website visited: http://www.businesswire.com/news/home/20150130005621/en/Research-Markets-2015-Trends-Global-Clinical-Development#.Vh50c-xVhHx.
11. Sanofi launches diabetes gaming app for children, PMLive, September 5, 2014. Website visited: http://www.pmlive.com/pharma_news/sanofi_launches_diabetes_gaming_app_for_children_596954.
12. Boehringer Ingelheim and Eli Lilly & Company launch type 2 diabetes educational game, “Complications Combat”, May 24, 2013. Website visited: https://www.boehringer-ingelheim.com/pressrelease/boehringer-ingelheim-and-eli-lilly-company-launchtype-2-diabetes-educational-game.
13. Project Management Learning Series: Fast Tracking vs Crashing Simplilearn August 24, 2015. Website visited: http://www.simplilearn.com/fast-tracking-vs-crashing-article.
14. Drug Companies Lose Millions Due to Clinical Trial Inefficiencies, Cutting Edge Information, March 9, 2005. Website visited: http://www.prnewswire.com/news-releases/drug-companies-lose-millions-due-to-clinical-trial-inefficiencies-54192787.html.
15. PPD Acquires Acurian, Industry Leader in Patient Recruitment and Retention, August 27, 2013. Website visited: http://www.ppdi.com/News-And-Events/News/2013/PPD-Acquires-Acurian.aspx.
16. LabCorp to Acquire Covance for $6.1B, November 3, 2014. Website visited: https://www.genomeweb.com/archive/labcorp-acquirecovance-61b.
17. IMS Health and Quintiles to Merge, Quintiles IMS to Become Industry-Leading Information and Technology-Enabled Healthcare Service Provider, May 3, 2016. Website visited: http://www.quintiles.com/news/2016/05/ims-healthand-quintiles-to-merge.
18. Disrupting Clinical Operations from the CRO Perspective, DPharm 2016. Website visited: http://theconferenceforum.org/conferences/disruptive-innovations-us/2016-agenda/.

To view this issue and all back issues online, please visit www.drug-dev.com.

Craig Morgan is a technology and life sciences management professional with more than 15 years of experience in the application of informatics and bioinformatics to drug discovery. He currently heads up the Marketing and Brand Development functions at goBalto, working with sponsors, CROs, and sites to reduce cycle times and improve collaboration and oversight in clinical trials.