Managing dementia through a multisensory smart phone application to support ageing in place

ArticleQ&ADetailDownload PDF
Alzheimer’s disease (AD) affects more than 40 million people worldwide, and a large part of its daily management depends on the voluntary efforts of a patient’s friends and relatives. Current research indicates that patients may die sooner if their caregivers are mentally stressed. This further includes stress-related health deterioration in the caregivers. Through the use of non-invasive sensor devices, Dr Tanvi Banerjee, Assistant Professor at the Department of Computer Science and Engineering at Wright State University, analyses both the physical attributes and behavioural patterns of AD patients and their caregivers, with the overall objective of supporting the quality of life of both parties.

Alzheimer’s disease (AD) is the sixth leading cause of death in the United States (US), where it is directly responsible for more than 500,000 deaths each year. Worldwide, approximately 44 million people live with AD, and with an ageing population, this number is expected to rise significantly over the next three decades. The management of AD and other forms of dementia is challenging and expensive. US statistics from 2013 revealed that 15.5 million relatives and friends provided a staggering 17.7 billion hours of unpaid care to patients with Alzheimer’s and other forms of dementia, illustrating the critical role of caregivers in the management of these diseases.

Caregiver burnout
It is often the spouses of dementia patients who act as caregivers, and care is frequently provided around the clock. It is not surprising that the vast responsibilities entailed in caregiving, such as assistance with daily living activities, medication management, housework, transportation, and meal preparation, to name a few, become overwhelming for many caregivers, resulting in so-called “caregiver burnout”. This, according to Dr Banerjee, “…has become a big problem. It can result in the patient ending up in an alternative care facility or a nursing home, not to mention severe physical and cognitive deterioration in the caregiver’s health.”

Dr Banerjee presenting the overarching concept behind the project using the DIKW framework.

Additionally, a number of recent studies have found that AD patients may actually die sooner if their caregivers are mentally stressed. These findings highlight the urgent need to not only monitor dementia symptoms in the patient, but to also evaluate the caregivers’ stress levels on an on-going basis.

Fitness devices and gaming technology
The steady flow of fitness devices entering the market creates great potential for the management of chronic diseases in children and adults. Using such devices to monitor physiological and behavioural patterns in dementia patients might allow the prediction of episodic symptoms before they become distressing or disruptive. Harnessing the potential of these devices to detect early signs of stress in caregivers would also provide an opportunity for intervention and support before the burnout stage is reached. However, in order to realise their potential, these devices, which are primarily designed with fitness assessment in mind, need to be validated for the target population – AD patients and their caregivers – in controlled environments.

A caregiver may normally be able to reach a certain level of the game in three minutes … if it takes much longer, it may indicate stress as a result of caregiving responsibilities Quote_brain

In an innovative and interdisciplinary project within Dr Banerjee’s Data Science for Healthcare laboratory, her research group and collaborators from the Departments of Geriatrics and Social Work at Wright State University combine computer science and clinical knowledge about AD to investigate the potential of wearable sensing devices and gaming applications to monitor chronic conditions in dementia patients and to detect caretaker stress.

Collaborators Dr William Romine and Dr Tanvi Banerjee discuss the statistical analysis for the NIH-funded Dementia project.

Data-Information-Knowledge-Wisdom pyramid
Dr Banerjee’s research is built upon a framework called the DIKW pyramid (an acronym for Data-Information-Knowledge-Wisdom).

Vast amounts of Data are obtained directly from mobile devices e.g., games/apps such as Sudoku or word scramble, and from wearable sensors. However, this data is of little value unless the relevant actionable Information is extracted from it, to allow meaningful real-world changes to take place. In the case of AD patients, relevant information might include physical parameters such as heart rate variability, breathing rate, and lung volume. For caregivers, this could include gameplay information such as the time it takes to unscramble a word. After extraction of the information, relevant domain information or Knowledge is applied. In the case of the patient, knowledge might encompass the common dementia behaviours exhibited by patients e.g., wandering, aggression, apathy, or a combination of these. For caregivers, prior knowledge could include information such as whether or not the caregiver maintains a job while juggling caregiving duties.

The Knowledge, combined with the Information from the Data, can then be used to address pertinent and time-sensitive health-related questions – Wisdom – such as: What are the current dementia symptoms on a given day? And, how are the caregiver’s stress levels at a given time?

The system block diagram showing the caregiver and person with dementia as well as the modalities measured in the NIH-funded Dementia project.

Project to date
This project began in 2014 and received funding from the National Institutes of Health (NIH) in 2016. Dr Banerjee’s group and collaborators recently published the results of a study set up to validate a commercial fitness tracker, the Hexoskin vest, with 31 adult participants 60 years of age and above. The physiological parameters estimated in this study include heart rate, breathing rate, lung volume, step count and the participants’ activity levels. The study revealed that while some of the parameters correlated strongly to clinically accepted gold standard values (i.e., values measured using conventional medical devices), others did not e.g., lung volume and activity level. These findings highlight a need for caution when interpreting the data obtained from wearable technology. However, the strongly correlated parameters show that wearable technology nonetheless provides useful information, warranting further investigation as a tool for health monitoring in elderly individuals.

The goal is to enable both the patient and caregiver to be able to stay in their home and maintain a high quality of life for a longer period of time Quote_brain

The project has also assessed Caregiver Assessment using Serious Gaming Technology (CAST), a gaming app for caregivers, which is designed specifically to detect stress levels. The app is currently under development and includes two games and a stress questionnaire. In a preliminary study published in 2017, the researchers presented a prototype of the CAST app to interested caregivers. The app was given to ten individuals participating in a shortened version of the stress questionnaire and a brief interaction with one of the app’s games. Overall, participants demonstrated interest in the app, and suggested ways in which the games could be improved to support more straightforward, routine use. This feedback will contribute to further refinements of the app’s user interface, and the app will undergo further investigation for its use in monitoring caregiver stress.

Changing the face of healthcare
The DIKW pyramid represents a holistic approach to analysing the whole person, rather than just treating a disease or its symptoms, while simultaneously providing important feedback to the caregiver about his or her own health. This approach also enables patient empowerment, giving the patient the possibility to gain greater control over decisions and actions affecting their health.

The innovation in Dr Banerjee’s work lies in its interdisciplinary outlook; different disciplines are working together to create practical solutions that could change the face of healthcare in the coming decades by protecting the welfare of caregivers and supporting ageing in place. This will enable elderly people to stay in their community and avoid institutional care while maintaining their quality of life.

How did you end up working with both computer science and health care?
I was fortunate to be a part of an interdisciplinary team during my doctoral programme at the University of Missouri where I worked with the Centre for Eldercare and Rehabilitation Technology (CERT). The two leads at CERT, Dr Marjorie Skubic (Electrical and Computer Engineering) and Dr Marilyn Rantz (Nursing) were strong female role models right from the beginning of my career. Currently, I am extensively collaborating with two faculty from healthcare, Dr Jennifer Hughes (Social Work) and Dr Larry Lawhorne (Geriatrics). I think having strong clinical collaborators played a big role in my decision to work in healthcare. These clinical collaborators are absolutely essential in helping my lab’s research move forward and make a positive impact.

What has been the biggest challenge in your research to date?
I think the biggest challenge has been convincing participants to try out our system which is still at its initial stage, and showing its use to actual caregivers. Most participants are not familiar with research and find it hard to understand that some of the results may take time to achieve. Here again, having strong clinical collaborators really helps the process as the caregivers find it much easier to relate to geriatrics clinicians and social workers as compared to computer scientists who may not be able to relate directly to the challenges of being a primary caregiver.

How far away are we from seeing these devices as a standard tool in the management of dementia?
I think we certainly have many more years to go before these are clinically approved as standard tools, although we do have a gaming prototype ready for testing at this point. We are currently working on adding other games to the system such as card games and word matching. We are also still in the process of analysing the wearable data we have collected from caregivers over a longitudinal period to see how the sensor data changes over time. That said, it is an exciting time for technology and healthcare; we are making good headway in building non-invasive smart home systems that can help older adults live in their homes longer.

To what other disease areas could you apply your research?
My research is applicable to many chronic health conditions. Currently, we are working in collaboration with Dr Nirmish Shah from the Duke School of Medicine to analyse pain in sickle cell disease patients using the same concept: an app to assess the pain, and a wearable to continuously measure the changes in physiology over time. The overall idea remains the same but in this case, the cohort has different demographics i.e., the patients are younger with the majority being of African American ethnicity. I foresee that this research may be applicable in monitoring and managing a variety of conditions including fibromyalgia, obesity, and diabetes, to name a few, where disease management would play a critical role in improving the patients’ quality of life.

Research Objectives
Professor Banerjee combines her expertise in Computer Science with her colleagues’ clinical knowledge of Alzheimer’s in order to analyse data from patients with dementia and their caregivers.

NIH Grant No. 1K01LM012439


  • Dr Jennifer Hughes, Wright State University (WSU)
  • Dr Larry Lawhorne, WSU
  • Jerry Mallicoat, Alzheimer’s Association, Miami Valley Chapter
  • Dr Matthew Peterson, WSU
  • Dr William Romine, WSU
  • Dr Noah Schroeder, WSU
  • Dr Nirmish Shah, Duke University
  • Dr Amit Sheth, Knoesis, WSU

Dr Banerjee is an Assistant Professor in the Department of Computer Science and Engineering, and the Department of Geriatrics (joint appointment) at Wright State University. Her current research interests include development of explanatory models to understand healthcare conditions such as dementia and caregiver stress measurement to support ageing in place.

Tanvi Banerjee, Ph.D.
Assistant Professor
Joshi Research Center 389
3640 Colonel Glenn Hwy
Dayton, OH 45435-0001

T: +1 (937) 775 5283
In the news:
Press from Alzheimer’s Association:

Related posts.


Leave a Comment

Your email address will not be published. Required fields are marked *

Share this article.

Share on facebook
Share on twitter
Share on whatsapp
Share on linkedin
Share on reddit
Share on email