Microsoft, Nuance developing ambient and AI technology to tackle doctors' documentation headaches

Microsoft and Nuance Communications are developing technology to "listen" to physician-patient conversations and automatically document in an electronic health record. (Microsoft/Nuance)

Tech giant Microsoft is teaming up with Nuance Communications to use technology to solve a big pain point for doctors—too much time spent on documenting and administrative tasks.

The two companies are collaborating to use ambient technology combined with artificial intelligence, automation and cloud computing to create an exam room experience where the clinical documentation "writes itself," the companies said in a press release.

Physician burnout continues to be a significant problem in healthcare. A recent study shows that primary care doctors now spend two hours on administrative tasks for every hour they’re involved in direct patient care. Physicians reported one to two hours of after-hours work each night, mostly related to administrative tasks.

Product Spotlight

Top-Rated Mobile App for Health Insurance Members

Zipari’s Mobile App is the smarter, easier, and better way for payers to engage members on the go and directly in the palm of their hands. Members can find the right doctors, receive notifications, send messages, view claims, track spending, talk to a nurse, download ID card, and more. It’s ready to install and launch in a few months.

Joe Petro, chief technology officer at Nuance, said the two companies saw an opportunity to leverage their respective technical strengths to tackle the problem of physician burnout.

RELATED: Epic, Nuance team up on voice assistance technology to help lighten doctor documentation burden

The collaboration brings together Microsoft's Azure cloud platform and AI capabilities with Nuance's expertise with clinical documentation and clinical speech recognition to develop ambient clinical intelligence (ACI). The goal is to use technology to reduce clinician burnout by allowing doctors to focus more on patients and less on electronic health records (EHRs) and administrative tasks. 

The technology is designed to "listen" to physician-patient conversations, with the patient's consent, during a doctor's visit. ACI then synthesizes the conversation, integrates the data with contextual information from the EHR and updates the patient’s medical record. The technology also provides workflow, task and knowledge automation.

"We want to provide a frictionless and engaging experience between clinicians, the providers, and patients in the exam room," Greg Moore, M.D., Ph.D., corporate vice president of healthcare strategy and alliances at Microsoft, told FierceHealthcare. "Often the computer is a barrier to interaction and even eye contact between a patient and physician, leading to an unsatisfactory experience for both."

"We're confident that we can deliver a solution that will bring joy back to the practice of medicine for providers," he said.

Nuance Communications developed the Dragon Medical platform that uses voice recognition and conversational AI to help physicians document in EHRs. That platform is used by more than 500,000 clinicians. 

RELATED: Physicians spend half of their time face-to-face with patients, half on the computer, study finds

The Microsoft-Nuance collaboration is part of a growing trend of healthcare providers and technology vendors migrating to cloud platforms offered by big tech giants, Jeffrey Becker, senior analyst for healthcare strategy at Forrester Research, told FierceHealthcare. Nuance can tap into Microsoft's deep bench of AI expertise for product innovation, he said.

As part of the agreement, Nuance will migrate the majority of its current on-premise internal infrastructure and hosted products to Microsoft Azure. Moving to a cloud-hosted infrastructure will give Nuance a market advantage as well, according to Becker.

"It gives the client flexibility and reduces the capital expenditure associated with the initial decision to purchase," he said.

Developing a "smart" exam room

Nuance has made significant advances with ambient sensing and conversational AI solutions, including capabilities like wake-up words, voice biometrics, signal enhancement, document summarization, natural language understanding, clinical intelligence and text-to-speech.

The new platform will leverage those tools as well as Microsoft's Project EmpowerMD, which is a virtual AI medical assistant.

The two companies are using their technologies to tackle the holy grail problem of converting conversational AI into a highly formatted medical document, Petro said. "That’s where the magic comes in; that's where the end-to-end neural networks come into play. We’re looking forward to joining forces between our research teams and Microsoft's teams to move the needle on that problem."

RELATED: Providence St. Joseph Health, Microsoft form strategic alliance to leverage cloud, AI technology

The joint technology platform will roll out in early 2020 starting with five physician specialties—orthopedics, podiatry, ENT (ear, nose and throat), ophthalmology and dermatology. Nebraska Medicine and Rush Medical are development partners, and more partnerships are expected, the companies said.

Moore anticipates the solution has the potential to advance beyond documentation to provide real-time clinical decision support. If a patient is talking to a doctor about medication, the system could flag an allergy or help recommend a more effective treatment plan, he said.

“We have an incredible opportunity to use advances in cloud and AI technology to transform healthcare delivery. Together with Nuance, we will apply the power of Azure and Azure AI to this challenge, with the aim of improving the day-to-day life of frontline healthcare workers everywhere—so they can provide better care," Microsoft CEO Satya Nadella said in a statement.

Suggested Articles

Employment growth in the healthcare industry cooled off in July as the sector added fewer jobs than in June as COVID-19 continues to spread.

A new study found that more than half of doctors don't believe drug-resistant superbugs are a major concern for their practice.

Blue Shield of California is teaming up with Cricket Health to offer coordinated care to members with late-stage and end-stage renal disease.