Microsoft, Nuance developing ambient and AI technology to tackle doctors' documentation headaches

Microsoft and Nuance Communications are developing technology to "listen" to physician-patient conversations and automatically document in an electronic health record. (Microsoft/Nuance)

Tech giant Microsoft is teaming up with Nuance Communications to use technology to solve a big pain point for doctors—too much time spent on documenting and administrative tasks.

The two companies are collaborating to use ambient technology combined with artificial intelligence, automation and cloud computing to create an exam room experience where the clinical documentation "writes itself," the companies said in a press release.

Physician burnout continues to be a significant problem in healthcare. A recent study shows that primary care doctors now spend two hours on administrative tasks for every hour they’re involved in direct patient care. Physicians reported one to two hours of after-hours work each night, mostly related to administrative tasks.

Webinar This Week

Optimizing Healthcare Operational Excellence to Drive Care Transformation

Join us in this webinar to learn how organizations have leveraged modern technology to enable transformative innovation and continuous improvement across their operations resulting in overall cost savings, process optimization, and clinical improvements.

Joe Petro, chief technology officer at Nuance, said the two companies saw an opportunity to leverage their respective technical strengths to tackle the problem of physician burnout.

RELATED: Epic, Nuance team up on voice assistance technology to help lighten doctor documentation burden

The collaboration brings together Microsoft's Azure cloud platform and AI capabilities with Nuance's expertise with clinical documentation and clinical speech recognition to develop ambient clinical intelligence (ACI). The goal is to use technology to reduce clinician burnout by allowing doctors to focus more on patients and less on electronic health records (EHRs) and administrative tasks. 

The technology is designed to "listen" to physician-patient conversations, with the patient's consent, during a doctor's visit. ACI then synthesizes the conversation, integrates the data with contextual information from the EHR and updates the patient’s medical record. The technology also provides workflow, task and knowledge automation.

"We want to provide a frictionless and engaging experience between clinicians, the providers, and patients in the exam room," Greg Moore, M.D., Ph.D., corporate vice president of healthcare strategy and alliances at Microsoft, told FierceHealthcare. "Often the computer is a barrier to interaction and even eye contact between a patient and physician, leading to an unsatisfactory experience for both."

"We're confident that we can deliver a solution that will bring joy back to the practice of medicine for providers," he said.

Nuance Communications developed the Dragon Medical platform that uses voice recognition and conversational AI to help physicians document in EHRs. That platform is used by more than 500,000 clinicians. 

RELATED: Physicians spend half of their time face-to-face with patients, half on the computer, study finds

The Microsoft-Nuance collaboration is part of a growing trend of healthcare providers and technology vendors migrating to cloud platforms offered by big tech giants, Jeffrey Becker, senior analyst for healthcare strategy at Forrester Research, told FierceHealthcare. Nuance can tap into Microsoft's deep bench of AI expertise for product innovation, he said.

As part of the agreement, Nuance will migrate the majority of its current on-premise internal infrastructure and hosted products to Microsoft Azure. Moving to a cloud-hosted infrastructure will give Nuance a market advantage as well, according to Becker.

"It gives the client flexibility and reduces the capital expenditure associated with the initial decision to purchase," he said.

Developing a "smart" exam room

Nuance has made significant advances with ambient sensing and conversational AI solutions, including capabilities like wake-up words, voice biometrics, signal enhancement, document summarization, natural language understanding, clinical intelligence and text-to-speech.

The new platform will leverage those tools as well as Microsoft's Project EmpowerMD, which is a virtual AI medical assistant.

The two companies are using their technologies to tackle the holy grail problem of converting conversational AI into a highly formatted medical document, Petro said. "That’s where the magic comes in; that's where the end-to-end neural networks come into play. We’re looking forward to joining forces between our research teams and Microsoft's teams to move the needle on that problem."

RELATED: Providence St. Joseph Health, Microsoft form strategic alliance to leverage cloud, AI technology

The joint technology platform will roll out in early 2020 starting with five physician specialties—orthopedics, podiatry, ENT (ear, nose and throat), ophthalmology and dermatology. Nebraska Medicine and Rush Medical are development partners, and more partnerships are expected, the companies said.

Moore anticipates the solution has the potential to advance beyond documentation to provide real-time clinical decision support. If a patient is talking to a doctor about medication, the system could flag an allergy or help recommend a more effective treatment plan, he said.

“We have an incredible opportunity to use advances in cloud and AI technology to transform healthcare delivery. Together with Nuance, we will apply the power of Azure and Azure AI to this challenge, with the aim of improving the day-to-day life of frontline healthcare workers everywhere—so they can provide better care," Microsoft CEO Satya Nadella said in a statement.

Suggested Articles

David Feinberg, M.D., head of Google Health, posted a blog post and video Tuesday to directly address growing concerns about the Ascension data deal.

Clinicians are the primary end users of digital health, and a clinical champion can make all the difference in whether a solution is adopted.

Year by year, resistance to extending Medicaid to more low-income Americans in conservative states has given way.