When a test is more than just a test… Finding unexpected properties that add to the value proposition
Summary
P1vital developed a set of digital tests to predict whether someone’s responding to antidepressants within 10 days of starting them (it usually takes several weeks to find out). They wanted to demonstrate how well it worked using a clinical trial, but the results were inconclusive. Fortunately they’d also asked MindTech to explore the user experience during the trial. We discovered the tests had unexpected effects, giving participants a better healthcare experience. The problem? These effects were happening in the control arm too! Our research shed light on why the trial didn’t show the tests to be effective, and helped to redefine the product as more holistic than previously thought, because of the unexpected effects we identified. This helped their business case going to market.
The ‘Test’ and our role in its trial
P1vital’s digital test measures the speed of people recognising different facial emotions. It predicts whether someone’s responding to their anti-depressants by detecting changes in speed before and after starting medication.
It was tested in a trial across France, Germany, Netherlands, Spain and UK. MindTech was asked to explore patients’ and doctors’ experience of using these tests as part of care (only the patients’ results are presented here).
How we did it
I led data collection, analysis and reporting, in a team of 4:
● Interviews in UK and Germany
● Surveys in UK, France, Germany, Netherlands, Spain
The research took place alongside the clinical trial, in the native language of each country. For analysis of the interviews, translation services were used.
User interviews in the UK and Germany showed the in-depth experience of having the Test as part of care, and we were able to compare experiences across both countries. Topics included confidence in the tests, convenience and ease of use, what it was like to have as part of their care:
● Total patients interviewed: UK = 22; Germany = 20
Surveys gave insight into people’s experience across all five countries. This enabled statistical comparison between countries and an overview of important factors e.g. convenience, ease of use and whether they’d use it again:
● Total number of surveys completed =778
I led the qualitative analysis using a framework approach, alongside the German researcher and with input from a translator. Quantitative data was analysed by a statistician.
Whilst final findings were presented in a report (and publication), at a practical level, regular reporting meetings took place with P1vital to summarise qualitative findings as they emerged. This allowed for in-depth discussion and a chance to iteratively shape subsequent data collection if new questions arose. We used immediate contact (via email) for any unexpected queries or issues.
Finding unexpected effects, and more…
We found several effects that weren’t expected, all of which could have had an impact on the trial results. We discovered that by taking the Test, patients were getting feedback about their wellness and could draw information from their answers to questions. They also felt more supported by having the Test, but some thought the results were being more closely monitored than they were which was an unexpected safety issue:
“I liked that there was that additional support and I kind of felt that if I did suddenly decline really badly, somebody would be watching and they would be like ‘ok we need to do something else now’, so I actually found that very helpful”
We flagged this immediately to the product team. The Test also made some patients realise how unwell they were, especially for those who had suicidal thoughts and feelings. This was another potential safety issue flagged with the product team.
Study results enhanced the business case for use of the Test within the NHS, and the Test (now called i-Spero) is now used in primary care in some parts of the UK. The results also helped make a case for a different trial that takes into account the unanticipated properties we identified.
Overall, our impact was:
● Discovering unanticipated effects helped to explain the trial results
● The extra properties we identified added to the business case for its use. They enhanced the value proposition and redefined it as more holistic than had been thought, because of these extra effects:
o It made patients feel supported
o It helped them have more confidence in their treatment
o It had a positive all-round effect on their experience of care
● We identified a small number of safety issues allowing them to be addressed
If you’d like to read more:
Study publication: https://bit.ly/3ZH3kFN
P1vital website: www.p1vital.com and product website: Solutions | i-Spero