Photo credit: Getty/pandpstock001

Despite doctors’ continued concerns about the usability of their electronic health records, comparing those experiences remains a highly difficult proposition, say researchers with MedStar Health and the Georgetown University School of Medicine.

In an article published in the Journal of the American Medical Informatics Association, the researchers, led by Raj Ratwani of MedStar’s National Center for Human Factors in Healthcare, say the current federal regulatory framework and implementation process doesn’t allow for “meaningful comparison” of EHR design processes, certification testing results and usability testing by providers.

Regarding the design processes, the authors say while two EHR vendors may attest to following a specific standard, their paths for getting to that point might not be the same. For instance, one vendor might choose to conduct several iterations of early-user testing with a number of clinicians, while another might only conduct such testing once using nonmedical users.

For comparing vendor usability tests, they say, while recommended testing scenarios exist, there are no required scenarios, meaning each vendor creates its own. While one testing scenario could be very complicated, another could be simple and straightforward, the authors point out.

Meanwhile, for comparing usability of products post-implementation, Ratwani and colleagues say that no objective test-based assessments of implemented EHRs exist. While survey-based comparisons of clinician feedback and perceptions of usability from IT leaders are often used, the latter, in particular, underrepresent actual usability challenges. Additionally, due to customization with different clinical systems at different sites, actual use from site to site is often dramatically different, the authors say.

The authors say efforts proposed by the Office of the National Coordinator for Health IT to improve post-market surveillance of EHR products could support improved comparison of vendor products, but note that cooperation between the agency, vendors and providers is needed.

“Until free market conditions can push vendors to aggressively compete on usability, ONC certification requirements will remain the primary lever for promoting usability, even though vendor adherence to these requirements remains subpar,” they say.

A study published last September by Ratwani in the Journal of the American Medical Association notes that many vendors did not follow “basic federal” certification requirements around usability, yet still had products certified by for Meaningful Use. To that end, MedStar and the AMA developed their own usability framework.