Microsoft has rolled out a new AI assistant for healthcare professionals that it bills as an all-in-one technology that combines voice dictation, ambient listening and generative AI.
The new tool, Dragon Copilot, combines the natural language speech recognition capabilities of Dragon Medical One with the ambient listening of DAX Copilot, which it launched about a year ago.
The tech giant acquired Nuance Communications, which makes speech recognition and conversational AI software, for nearly $20 billion in 2021 and has been rapidly building out its voice-enabled clinical documentation and workflow tools for clinicians.
Joe Petro, corporate vice president of Microsoft Health and Life Sciences Solutions and Platforms, said Dragon Copilot marks the first unified voice AI experience in the healthcare market.
The upgraded voice-enabled AI assistant enables clinicians to dictate medical notes during patient visits, search for medical information and automate tasks like conversational orders and referral letters all in one application.
Kenneth Harper, general manager of the Dragon product portfolio at Microsoft, has worked in the AI and speech technology space for more than two decades and touted the new voice-enabled Dragon Copilot "by far the most exciting advancement that we've made to date."
Dragon One has been used by more than 600,000 clinicians to document billions of patient records, according to Microsoft executives, while DAX Copilot has documented more than 3 million ambient patient conversations across 600 healthcare organizations just in the past month.
The new AI assistant combines speech capabilities and ambient AI along with "fine-tuned generative AI powering many new capabilities and innovations all running in a single workspace, all backed by healthcare adapted safeguards," Harper said during a briefing with reporters.
Typically, clinicians have to toggle between different applications that focus on specific tasks, he noted. "They also weren't able to use natural language AI prompting to surface information or easily change and customize their documentation," he said. "By leaning into the power of Microsoft's enterprise capabilities and bringing these applications together into a single user interface, patients, clinicians and healthcare organizations will realize increased efficiency and improved clinician wellbeing as well as patient experiences," Harper noted.
Physicians and clinicians report high levels of burnout due in large part to tedious and time-consuming administrative workloads. Many healthcare organizations are turning to AI to automate or streamline some of these tasks and ease the workload.
David Rhew, M.D., Microsoft's chief medical officer, said clinicians often spend time after work hours doing patient charting, answering emails, filling out insurance information and prior authorizations. "Nearly 50% of clinicians today are burned out, and many are leaving the practice. This is a problem. In fact, Mercer projects the deficit of over 100,000 healthcare workers in the U.S. alone by 2028 and on top of that, with the increasing aging population, there's going to be an even greater demand for clinicians," he said during the press briefing.
A growing list of companies have developed ambient AI scribes and voice dictation tools for healthcare professionals.
Microsoft executives claim Dragon Copilot's key point of differentiation is that it combines these capabilities and streamlines the clinical workflow all in one application.
"Dragon Copilot is making it much easier for clinicians to choose between dictation input versus ambient input for conversational capture, and do that all through a single user experience, a single workspace that's available across mobile, web and desktop," Harper said. "You can think about your clinician throughout the course of their clinic, they can pick and choose the right modality at the moment that's necessary across the entire care continuum."
"The seamless user experience is something that's really important and what's different from how our products have been designed historically," Harper added.
Clinicians can also give Dragon Copilot instructions using natural language. "It lets clinicians more easily change and modify documentation simply by speaking in natural language. The system understands what changes to make based on those natural language instructions. As part of this, you can also access medically certified content sources. If you want to get an answer from a source like the CDC for clinical guidelines, Dragon Copilot allows you to do that by simply asking a question," Harper said.
He added, "If you're looking at documentation and you just want to say, 'Add some more details to the HPI (history of present illness) about what the patient said, about feelings, about nourishment,' you can speak in your natural language, this is the change I want to make, and Dragon Copilot can actually make that change on your behalf. That gives clinicians yet another tool to leverage to help them with that administrative work," he said.
Dragon Copilot will be generally available in the U.S. and Canada in May, followed by the U.K., Germany, France and the Netherlands.
Clinician burnout in the U.S. dropped from 53% in 2023 to 48% in 2024, according to data from the American Medical Association, and Rhew credits some of that improvement to technology like AI.
Clinicians using DAX Copilot reported they saved five minutes per patient encounter. Seventy percent of clinicians reporting reduced feelings of burnout and fatigue and 62% of clinicians stating they are less likely to leave their organization, according to responses from nearly 900 clinicians back in July.
“With Dragon Copilot, we’re not just enhancing how we work in the EHR — we’re tapping into a Microsoft-powered ecosystem where AI assistance extends across our organization, delivering a consistent and intelligent experience everywhere we work,” said R. Hal Baker, M.D., senior vice president and chief digital and chief information officer at WellSpan Health. “It’s this ability to enhance the patient experience while streamlining clinician workflows that makes Dragon Copilot such a game-changer.”
The newest evolution of Dragon represents a "significant step forward" in alleviating the strain on clinicians, said Glen Kearns, executive vice president and chief information officer at The Ottawa Hospital. “We are thrilled to be one of the first customers in Canada to use Microsoft’s ambient and generative AI technology. The newest evolution of Dragon Copilot could help alleviate documentation burden for our clinical teams.”