Share this article.

e-Mental health implementation requires more robust studies

ArticleDetailDownload PDF

During the COVID-19 pandemic, mental health conditions are being exacerbated by new stressors like social isolation. Digital mental health services (e-mental health) offer a way of providing accessible and affordable support. However, with the swift adoption of a multitude of e-mental health applications and platforms that have emerged during the pandemic, little is known about their immediate, medium or long term clinical value. To investigate this, Dr Louise Ellis at the Australian Institute of Health Innovation at Macquarie University has been researching the processes by which these e-mental health programs are being implemented in real-world practice and identify shortfalls in current implementation studies.

Even before the onset of the COVID-19 pandemic, the increasing prevalence of mental health conditions worldwide had been characterised as a crisis. Now, with the added stressors of social isolation, risk of unemployment and educational dislocation exacerbating existing mental health conditions and increasing individuals’ anxiety over their health and wellbeing, the mental health crisis is more pronounced than ever.

In Australia, early estimates suggest that the population prevalence of poor mental health has more than doubled from 20% to close to 45% over the past year. Similarly, a recent international survey conducted in May 2020 found that over 50% of survey respondents self-reported as having poor mental health, with reduced physical activity due to mandatory isolation often cited as a contributing factor. This result shows a marked increase from the previously estimated prevalence of poor mental health, suggesting that these numbers have more than doubled during the pandemic. Although COVID-19 vaccines are expected to alleviate some immediate stressors, much of the additional mental health burden is anticipated to persist for many years to come. Best-case scenario models predict a 13.7% increase in suicides in Australia between 2020-25 alone.

Although mental illness can be treated and managed with appropriate care, millions are still missing out. In 2020, the Australian Government recommended using supported digital treatments as a convenient, clinically effective, low-cost way to amplify reach.

e-Mental health
The potential of e-mental health to treat and manage mental ill-health has been widely recognised in Australia and globally. e-Mental health is defined as ‘mental health services and information delivered or enhanced through the internet and related technologies.’ The growing field of e-mental health covers much more than digital mental health information, ranging from telehealth and online support groups, to digital assessment and treatment programs, and even therapeutic gaming.

A survey conducted in May 2020 found that over 50% of respondents self-reported as having poor mental health.

The potential benefits of the use of e-mental health programs have been greatly reinforced by the need for mental health services to adapt as a result of social distancing measures, and has propelled telehealth and online services into widespread use in developed countries in favour of face-to-face therapies. However, with the onset of COVID-19, there has generally been a hasty and ‘ad hoc’ roll out of various online programs and services, without any coordinated system or plan, and often without links back into traditional mental health services; with experts now calling for a ‘thorough evaluation’.

Further, emerging evidence suggests the benefits of e-mental health previously established in controlled clinical trials are not being realised in real-world settings, with implementation challenges reported among both clinicians and patients. There are several possible reasons behind this. In the case of patients, they may be sceptical of the legitimacy of e-mental health programs, whereas clinicians may lack the training to effectively deploy these services or feel resistant to a change in treatment methods.

Technology has the potential to revolutionise mental health care. Nattakorn_Maneerat /

To further investigate the disparity, Dr Louise Ellis has been researching the processes by which these e-mental health programs are being implemented in real world clinical settings.

Dr Ellis specialises in implementation science, the study of methods used to promote the systematic uptake of research findings to improve healthcare services. She and her team began their work prior to COVID-19 by conducting a scoping review of implementation research of e-mental health programs for anxiety and depression to assess the quality of the research tools and implementation processes used as well as the implementation outcomes the research evaluated.

Scoping the research

A search of four e-databases for research conducted between January 2000 and January 2019 produced 33 implementation studies of e-mental health programs. Of these, 28 were from the last five years of the search period. This vast increase in the number of studies conducted in recent years reflects the growing interest in this area of research. However, key flaws in several of these studies highlight the need for more robust implementation studies that are supported by strong theory.

The potential of e-mental health to treat and manage mental ill-health is widely recognised.

Of the 33 studies, only 10 used an implementation framework. A framework is a model or theory used to guide the implementation and evaluation of a new program. The use of a framework aids the assessment of potential facilitators of, and barriers to, implementation, and helps to develop context-specific strategies and evaluate the success of the implementation. There are several, widely accepted frameworks available, but the number of studies not using frameworks, and thus lacking a strong theoretical underpinning, risks slowing both the advance of e-mental health services and implementation science as a discipline. Whilst the number of studies that used frameworks was low, more than half of the ones that did were conducted in the final two years of the search period, suggesting that the recognition of their importance is increasing.

Over half of the studies (n=21) used surveys to study implementation outcomes. However, only 9 of these used an existing survey, with most choosing to use a bespoke survey that did not contain previously validated questions. This decision is likely to have stemmed from the lack of available instruments for the assessment of e-mental health programs but implies that knowledge of implementation strategies overall is limited. This was also evident in the lack of consistency in the use of terminology and application of concepts between studies, which made drawing comparisons difficult.

More than half of the studies focussed on early-stage implementation outcomes, e.g., the acceptability, appropriateness and feasibility of the programs, but later-stage outcomes like the sustainability of using the program as a treatment were seldom researched. This is likely due to the need for long-term follow up, but it is still critical to assess these outcomes to get a full picture of the success of the implementation.

Countries of included studies.

Notably, only two of the studies assessed the cost of implementing the programs, despite this being one of the main interests of decision-makers. Without assessing cost, it is difficult to calculate the cost-effectiveness of e-mental health programs, which may act as a barrier to their implementation. Dr Ellis noted that implementation cost should be included in future studies as a priority to support the scaling up of e-mental health programs in real-life clinical settings.

Overall, more research is urgently needed to address the research-to-practice gap in the field of e-mental health. Whilst the number of implementation studies being conducted seems to be increasing, it is vital that future research utilises proper frameworks, and that both reliable and practical measures for assessing the implementation of e-mental health programs are developed.

Key flaws in several studies highlight the need for more robust implementation studies that are supported by strong theory.

e-Mental health in action

In addition to investigating the implementation of e-mental health platforms, Dr Ellis and her team have been exploring the potential of existing applications and digital platforms to act as, or in tandem with, e-mental health services.

Location-based augmented reality (AR) games like Pokémon GO, although widely used for entertainment purposes, have been shown to have a beneficial impact on the mental health of their players, particularly during the periods of social isolation brought on by the pandemic.


Across over 2000 people that completed a self-reported survey, approximately three-quarters of participants reported that playing these games improved their mental health during the pandemic. Respondents said that they used these games to maintain a social connection virtually, lower stress and alleviate the effects of any existing mental health conditions. Additionally, increased gameplay was strongly associated with increased exercise, another factor in improving mental health.

The results of this survey suggest that future research could explore the potential of AR games as digital behavioural interventions that could be used to maintain mental wellbeing in the wider population

The opportunities that e-mental health services present in treating and managing mental health conditions have been recognised for over two decades. With face-to-face therapies still impossible in most countries, e-mental health also holds the potential for a safe and effective way to offer support to those living with mental health conditions during periods of mandatory social isolation.

The recent proliferation and rapid uptake of e-mental health options in the face of the global COVID-19 pandemic presents a unique opportunity to translate the efficacy of digital programs seen in clinical trials into effective, affordable and accessible adjuncts or alternatives to face-to-face care. There is much to learn about how e-mental health can be best incorporated into real-world settings so that e-mental health can reach its full potential.

What would a well-designed implementation study comprise?

Implementation science offers a multidisciplinary perspective and systemic approach for the design and evaluation of implementation studies. The integration of e-mental health into real-world settings requires consideration of the interdependencies between technology, clinicians, patients and local conditions. That is, understanding variations in the availability of resources, levels of expertise and commitment to using e-mental health is critical for modelling impacts across different setting. Also critical is the regular adoption of implementation frameworks, as well as the development and use of validated measures of implementation outcomes.



  • Ellis, L. et al. (2020). Implementation of e-mental health for depression and anxiety: A critical scoping review. J Community Psychol. 48, pp, 904-920.
  • Ellis, L. et al. (2020). COVID-19 as ‘Game Changer’ for the Physical Activity and Mental Well-Being of Augmented Reality Game Players During the Pandemic: Mixed Methods Survey Study. J Med Internet Res. 22(12): e25117.

Research Objectives

Louise Ellis’s research focuses on mental health services and systems, implementation science and complexity science approaches.


The work has been supported research grants from the National Health and Medical Research Council (APP9100002, APP1134459, APP1176620) and the Medical Research Future Fund (APP1178554).


Jeffrey Braithwaite, Professor and Director [email protected]


Dr Louise Ellis is an Implementation Scientist at the Australian Institute of Health Innovation at Macquarie University. Her specific research areas of expertise include: mental health services and systems, implementation science and complexity science approaches.

Dr Louise Ellis

75 Talavera Rd, North Ryde
NSW 2113 Australia

E: [email protected]
T: +61 422 030 396 T: +61 2 9850 2484

Creative Commons Licence

(CC BY-NC-ND 4.0) This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Creative Commons License

What does this mean?
Share: You can copy and redistribute the material in any medium or format
Related posts.