Pay attention to contextual, implementation, and mechanisms of impact factors when evaluating technological interventions
Guidance
When evaluating the benefits of technological interventions for people with dementia and their carers it is recommended to conduct a process evaluation to understand the possible influence of contextual, implementationA set of planned, intentional activities that aim to put into practice evidence-informed policies and practices in real-world services (see www.implementation.eu) and mechanisms of impact factors that may have influenced the intervention outcomes. This will also provide useful information on the conditions for successful implementationA set of planned, intentional activities that aim to put into practice evidence-informed policies and practices in real-world services (see www.implementation.eu) of the intervention.
Explanation and example
In our randomised controlled exploratory pilot trial into the FindMyApps programme, a tablet-based selection tool and training to help people with dementia to find apps for better self-management and meaningful activities, we conducted a process evaluation based on the British Medical Research Council’s (MRC) guidance for process evaluation of complex interventions (Moore et al., 2015).
This framework highlights the possible influence that contextual, implementationA set of planned, intentional activities that aim to put into practice evidence-informed policies and practices in real-world services (see www.implementation.eu) and mechanisms of impact factors may have on intervention outcomes. The process evaluation in the FindMyApps study provided very relevant information. For instance, with regard to contextual factors we found that it is important that the person with dementia has someone who is easy to approach and who can help them in case of practical problems, and that a helpdesk is in place for more complicated questions and technical problems.
With regard to implementationA set of planned, intentional activities that aim to put into practice evidence-informed policies and practices in real-world services (see www.implementation.eu), it proved important to check if and how much a participant had experience in working with technological devices, and to adapt their training accordingly. Additionally, it proved necessary to personalise the approach to a participants’ awareness of their deficits. This was largely because some people with dementia had a more accurate understanding of their abilities and limitations with respect to their deficits than others. With regard to mechanisms of impact, we found that users who regularly practiced and who’s caregivers helped them by means of the errorless learning method learned to use FindMyApps easier than users who practiced less and who’s caregivers were less active in guiding them by using errorless learning.
This information is not only relevant for the outcome evaluation, but also to get insight into conditions for successful implementationA set of planned, intentional activities that aim to put into practice evidence-informed policies and practices in real-world services (see www.implementation.eu) of FindMyApps.
Themes
Dementia FindMyApps Meaningful activities MRC guidance for process evaluation of complex interventions Process evaluation Self-management Tablet intervention Technology
Target groups
Researchers developing technology Technology developersType of evidence
Process Evaluation of the FindMyApps program (Beentjes et al., submitted)
References
Beentjes, K.M., . Kerkhof Y.J.F., Neal D.P., Ettema, T.I., Koppelle, M.A., Meiland, F.J.M., Graff, M., Dröes, R.M. PhD (2020). Process evaluation of the FindMyApps program trial among people with dementia or MCI and their caregivers based on the MRC guidance. Gerontechnology, 20(0), 1-15. https://doi.org/10.4017/gt.2020.20.1.406.11
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., … Baird, J. (2015). Process evaluation of complex interventions: Medical Research Council guidance. BMJ (Online), 350. https://doi.org/10.1136/bmj.h1258