The buzzword these days in the mobile health community is "evidence" and whether mHealth is actually having a global impact on outcomes. Last week, two new reports from the mHealth Alliance--the latest in as many weeks--highlighted the lack of quality evidence and methodologically rigorous research for mobile health. According to the reports, the evidence base supporting mHealth is "still relatively nascent" and in need of better ways to measure its impact worldwide.
It stands to reason that you can't improve what you can't measure. Relative to mHealth, if you can't measure progress, then how do you know you are really making progress?
In his 2013 annual letter, Bill Gates rightfully calls for "innovations in measurement" that he says are critical to advancing the global health agenda. No one can deny the impact that the Bill & Melinda Gates Foundation has had in developing countries, especially in eradicating diseases like polio. However, even Gates recognizes that his foundation and the larger healthcare community 'need better measurement tools to determine which approaches work and which do not."
In his letter, Gates highlights strong examples over the past year of how measurement is making a difference. Specifically, he points to efforts in the country of Nigeria and "how the digital revolution allows us to improve the use of measurement" in the foundation's polio vaccination campaign. "Thanks to cell phones, satellites, and cheap sensors, we can gather and organize data with increasing speed and accuracy," Gates argues.
Still, two recent reviews have found little evidence that mobile phone technology is having a great impact on managing disease in developing countries. The problem is a lack of rigorous studies in low- and middle-income environments. As the reviews point out, most existing evidence on the success of mHealth comes from trials performed in the developed world and is of poor quality.
Only three out of 75 trials that aimed to assess whether mobile technology interventions for healthcare consumers could change health behavior or improve disease management were conducted in developing countries. And, none of the 42 trials of interventions designed to support communication among healthcare providers or between health services and patients were done in the developing world.
Likewise, a separate review found very few formal outcome evaluations of mHealth in developing countries and, despite "vast documentation of project process evaluations," there were few studies demonstrating an impact on clinical outcomes.
As the global health community sets its agenda for 2013, organizations need to demand more rigorous research and methods of measurement in evaluating how mHealth is impacting health programs worldwide, and not just in developed countries. Evidence matters for mHealth just as other medical fields.
A good place to start would be randomized controlled trials. As Dr. Francis Collins, director of the National Institutes for Health, pointed out at the 2012 mHealth Summit, only about 20 randomized clinical trials involving mHealth tools or services have been conducted in the United States since 2008 under NIH auspices. And, as Collins revealed, more than half of them have failed to document clear evidence of improved outcomes. Let's hope 2013 is the year of evidence for mHealth. - Greg (@Slabodkin)