Global Evidence Requirements to Launch a Digital Health Technology

Published November 12, 2021

Executive Summary

• Given the growing number of digital health technologies available, there is a wide range of frameworks that outline key requirements to support decision-makers with the value assessment of digital health solutions
• Understanding how requirements differ based on the country and organization that develops the framework is key to prepare the successful global launch of a digital health technology
• Clinical evidence that demonstrates an improvement in the state of health or quality of life should be taken into consideration to demonstrate the medical benefit of novel digital health technologies

Trinity’s Take: Having analyzed existing guidelines and evaluation frameworks for digital health technologies, including mobile health (mHealth) applications, several key evidence requirements and areas have been established and should be considered in the clinical assessment of digital health services.

Digital Health Technology Findings

• Some organizations, such as the American Psychiatric Association (APA), introduced an evaluation model that provides a rating system for APA members, patients, and other providers who are considering the use of a mental health app
• Other institutions such as NICE developed an evidence standards framework for digital health technologies to demonstrate their value in the UK’s health care system by describing the standards for evidence that should be available or developed
• Similarly, digital health applications that are prescribed and reimbursed in Germany must be listed in the DiGa Directory, which was developed by the Federal Institute for Drugs and Medical Devices (BfArM) and is intended to outline evidence requirements that digital health applications must fulfil
• Trinity reviewed the selected tools and frameworks that have been developed by different organizations across markets and identified key case studies and evidence requirements that are commonly used to assess the clinical benefit of digital health solutions

Evidence Standards to Demonstrate Clinical Benefit

• While APA acknowledges that few apps will have randomized double blinded placebo-controlled studies that demonstrate the app’s clinical benefits, APA’s evaluation model provides general questions (e.g., is there evidence of specific benefit from academic institutions, end user feedback, or research studies?) to guide the decision to use an app
• NICE’s framework provides more detail on the evidence for effectiveness standards; a high quality observational or quasi-experimental study that describes the effect of the technology on a group of representative users, and allows some comparison with outcomes without the use of the technology is the minimum requirement to demonstrate that the digital health technology can prevent and manage a disease
• Additionally, the DiGa Directory specifies that the patient-relevant effects of the digital health app (e.g., reduction of the duration of a disease, prolongation of survival, or improvement in QoL) should be demonstrated, preferably in a comparative study which shows that the use of digital health app is better than not using it
• Other non-clinical aspects, such as data protection, usability, accessibility, interoperability, and data integration are commonly considered in the evaluation of digital health technologies to determine appropriateness for use
  

Existing Assessments of Digital Health Technologies

• APA provides several different sample app evaluations to illustrate how to use the model; in one of the recent evaluations, an effectiveness study of BetterHelp, a digital psychotherapy platform that provides access to licensed mental health professionals, was listed as clinical evidence supporting the use of the app
  • Study findings demonstrate that depression symptom severity was significantly reduced after the use of BetterHelp, as measured by the Patient Health Questionnaire (PHQ-9)
• NICE has recently completed the first assessment using the evidence standards framework; the Zio XT Service, which is used for detecting abnormal heart rhythms, is recommended as an option while further data is collected to address evidence gaps about the benefit for patients and the NHS
  • The recommendation was based on 4 comparative studies that compared 14‑day Zio XT with a 24‑hour Holter monitor; overall, studies showed an increased diagnostic yield with Zio XT over total wear time
  • However, the lack of evidence showing that an increased diagnostic yield with Zio XT improves clinical outcomes was highlighted by the committee; in addition, uncertainties related to the resource use and long-term consequences of using Zio XT were mentioned in NICE’s recommendation
• In Germany, five mobile applications have been assessed and incorporated into the DiGa Directory between 2020 and 2021; an additional 10 apps have been temporarily incorporated given that the medical benefit or patient-relevant effect of the app have not been conclusively demonstrated
  • Technologies such as Invitro, which reduces the anxiety symptoms of agoraphobia and panic disorder, have been temporarily incorporated to the DiGa Directory based on a comparative study vs. non-treatment; however, outcomes of such study have not yet been published and therefore the application may be removed from the list based on the results of the study
• Technologies that have been assessed and incorporated into the DiGa Directory are nationally reimbursed in Germany; however, APA’s evaluation model is intended to only provide guidance for physicians, patients, and other providers that are considering the use of a digital health app
  • Similarly, NICE’s framework is intended to help NHS commissioners to make more informed decisions by providing the different levels of evidence that should be presented for the assessment of the technology

Global Trends in Digital Health Technologies 

• While the selected evaluation frameworks have been developed by different types of organizations across countries, common aspects, such as the prolongation of survival, reduction of the duration of a disease or the improvement of the state of health, are highlighted as key criteria to be considered in the assessment of digital health technologies
• Initial assessments of digital health technologies using these frameworks are currently being published and most manufacturers who submit for evaluation by the organizations are conducting comparative studies that compare the clinical effect of the technology with the outcomes without the use of the technology
• Assessments conducted thus far highlight key uncertainties associated with the use of digital health solutions; as such, further data evaluating the long-term impact of using the technology and the positive therapeutic effect on health and care systems, as well as clinical evidence that demonstrates the impact of the technology on clinical outcomes are likely to be required for the evaluation of future digital health technologies

Written by Anna Alonso and Maximilian Hunt