Dr David Schelly and Dr Alisha Ohl of Clarkson University investigated the efficacy of a popular intervention to prevent depression in elderly people, The Well Elderly Intervention. Building upon previous investigations and by using advanced statistical analyses, the authors found that the effectiveness of the Well Elderly Intervention was not as straightforward as originally thought. The authors outline several areas for methodological improvements in the field.
The Well Elderly Intervention is a behavioural intervention aimed at older adults, over the age of 60. It focuses on building a healthy lifestyle to prevent chronic health conditions that are more common in later life, such as depression.
The Well Elderly Intervention was developed in the United States during the 1990s and has been found to be more beneficial to physical and mental health outcomes than other approaches. Accordingly, the Well Elderly Intervention is now being adopted in other countries such as Scandinavia and the UK. Notably, the Well Elderly studies led the UK’s National Institute for Health and Care Excellence to add a guideline for occupational therapy services in community settings.
The Well Elderly II study was conducted in 2012 to build upon the original research by testing the effectiveness of the intervention among a more ethnically diverse elderly population.
The Gold-Standard
To establish the effectiveness or efficacy of an intervention, researchers often use randomized controlled trials (RCTs). RCTs are the gold standard of study design. As Dr David Schelly, Assistant Professor in the Department of Occupational Therapy at Clarkson University explains: “At their best, RCTs allow us to establish a causal connection between a given treatment and outcome.” The random allocation of individuals into different groups (e.g., treatment and control) evens out inherent differences in the groups, making them equal. Once the groups are equal, we can conclude that any post-test differences between the groups are associated with the treatment.
The best way to analyse RCT data, and the only way to preserve the benefits of randomization, is using “intent-to-treat” (ITT) analyses. ITT involves including analyses of all individuals who were randomly assigned to each group at the start, regardless of whether they dropped out, changed groups, or received different levels of treatment to others in their group.
However, if groups are not equal in the level of treatment received, it can be difficult to reach conclusions about the effectiveness of the intervention. This highlights an issue with using only ITT analyses to determine the effectiveness of an intervention, especially in relation to preventative interventions.
Preventative interventions such as the Well Elderly are aimed at healthy people. However, research has found that healthy individuals are much more likely than unhealthy individuals to drop out of treatment or skip treatment sessions. In the Well Elderly II, for example, 30.6% of those who were randomized into the treatment group never attended an individual treatment session. This creates substantial imbalance in the groups, whereby people with more severe symptoms at the start will receive a higher “dosage” by attending more treatment sessions and therefore may have better outcomes.
Therefore, it may be useful to conduct follow-up analyses whereby individuals are grouped as they were treated (“as-treated analyses”). Unlike ITT, as-treated analyses focus on treatment actually received. While the approach necessarily abandons the benefits of RCT data, where intergroup differences are washed away during randomization, it also provides a snapshot of treatment efficacy, or whether a treatment actually works.
” Healthier people are much more likely to drop out of treatment or skip treatment sessions. “
The current research
The Well Elderly II study reported that the invention led to significant improvements in several health outcomes, including general health, mental health, life satisfaction and depression. However, because the Well Elderly Intervention is a preventative treatment programme which enrols many healthy individuals, there is a high rate of non-adherence to the intervention.
The authors of the Well Elderly II study used ITT analysis, but they did not follow-up with an as-treated analysis. Given that there is a high rate of non-adherence to the intervention by healthy people, their results might be different when investigating individual outcomes based on their treatment (as-treated) rather than original group allocation (ITT).
To investigate this, Dr David Schelly and Dr Alisha Ohl of Clarkson University conducted an as-treated analysis of the Well-Elderly II study data. They wanted to understand what dose of therapy, and in what form, is needed to experience improvements in symptoms of depression.
Using advanced statistical methods in a large sample of 322 participants, they investigated three modes of treatment in the Well-Elderly II study, including individual therapy sessions, group therapy sessions, and community outings, to consider the effects of treatment on depression outcomes.
Findings
Drs Schelly and Ohl found that the benefits of the Well Elderly II study might not be as straightforward as originally reported. Overall, there was a small decrease in depression symptoms after treatment, which concurs with the original research. However, when looking at the unique influence of each treatment programme along with potentially confounding factors, no mode of treatment had a direct effect on depression outcomes.
More detailed exploration of the data showed that individual sessions and community outings had small indirect effects on depression by increasing “activity frequency,” which then slightly reduced depression scores. There was no indirect effect of group sessions on depression.
A “selection effect” was present for individual treatment, where individuals with higher depression scores at baseline were the most likely to participate. One reason for this may be that individual sessions attract people with more depressive symptoms, whereas healthier individuals choose not to attend the sessions. This selection effect has important implications because the programme is meant to target healthy individuals.
As Well Elderly was designed as a preventative treatment, its target participants would have lower depression scores at the start, making improvements in scores more difficult. Furthermore, the results of the study suggest that for individuals with high depression scores, community outings are as effective as individual sessions, but require much less professional training to staff. Individual and group sessions with occupational therapists are expensive, and there is no evidence from the Well Elderly II data that they are effective at preventing health declines in older adults. The authors note, however, that the Well-Elderly II RCT study design may not be the most appropriate for examining age-related health declines, because it did not collect prospective longitudinal data.
Conclusion
Dr Schelly and Dr Ohl’s findings highlight the importance of investigating an intervention using “as-treated” analyses, as well as ITT analyses. As-treated analyses are especially important when there are high levels of drop out and non-adherence to the treatment programme. In an upcoming publication Dr Schelly also outlines the potential difficulties in interpreting intervention efficacy when a small number of individuals experience large improvements, but the majority of individuals appears very similar to the control group. When the large improvers are in the extremes at baseline (e.g., high depression scores), their improvement may be in part because of regression to the mean.
Further research by the team also recommends that researchers avoid using significant p-values as indicators of intervention effectiveness. The researchers show that statistical significance can often have little real world relevance, and that measures of clinical relevance are more appropriate for assessing intervention effectiveness. This highlights the importance of using the most appropriate methods of statistical analysis and interpretation to determine the true effectiveness of an intervention. While the gold standard is to use ITT to analyse RCT data, it is often appropriate to conduct follow-up analyses to assess treatment effectiveness.
It is still possible that “Lifestyle Redesign” prevents age-related declines, but Well Elderly studies have traditionally relied on a crossover design, where the control group receives treatment after the treatment group’s treatment has ended. Thus, there is no way to continue comparing the two groups over time. Of course, it is difficult to recruit participants if they have a 50% chance of receiving no treatment (as opposed to receiving it later). Future research should randomise participants into two treatments: one-on-one occupational therapy treatment in one group, and group sessions and community outings in the other, with objective measures of health repeated over many years.
References
- National Institute for Health and Care Excellence. Public Health Guideline. October 2008. Mental Wellbeing in Over 65s: Occupational Therapy and Physical Activity Interventions. https://www.nice.org.uk/guidance/ph16/resources/mental-wellbeing-in-over-65s-occupational-therapy-and-physical-activity-interventions-pdf-1996179900613
- Ohl A, & Schelly D. (2017). Beyond p-values: A case for clinical relevance. British Journal of Occupational Therapy, 80(12), 752-755). https://doi.org/10.1177/0308022617735048
- Schelly D, & Ohl A. (2019). Examining Clinical Meaningfulness in Randomized Controlled Trials: Revisiting the Well Elderly II. Am J Occup Therap, 73(1). doi:10.5014/ajot.2019.030874
- Schelly D, Ohl A, & Nadres R. (2020). Dosage and Efficacy in Behavioral Interventions With Community Dwelling Older Adults: Lifestyle Redesign Revisited. J Appl Gerontol, 1-9. https://doi.org/10.1177/0733464820911335
Research Objectives
The research of David Schelly and Alisha Ohl centers on evidence-based practice and health behaviour.
Collaborators
Ramona Nadres
Bio
Dr Schelly received a PhD in Sociology from the University of Wisconsin in 2016 and is currently an Assistant Professor of Occupational Therapy at Clarkson University.
Dr Ohl received a PhD in Occupational Therapy from New York University in 2012 and is the Chair and Program Director of the Occupational Therapy Program at Clarkson University.
Contact
Department of Occupational Therapy
Lewis School of Health Sciences
Clarkson University
8 Clarkson Ave.
Potsdam
New York 13699
United States
