Smart speakers are known for playing your favorite music, telling you the weather or reminding you when to get up and take a walk. They also could be an important tool in cardiology.
Researchers at the University of Washington have developed a form of contentless monitoring, allowing smart speakers to identify and monitor individual heartbeats using sonar technology. Machine learning (ML) makes this technology possible. The UW researchers published the findings on March 9 in the journal Communications Biology.
Co-senior author Shyam Gollakota, a UW associate professor in the Paul G. Allen School of Computer Science & Engineering, says it’s not detecting heartbeats that is a surprisingly smart speaker skill but the fact that it can detect irregular heartbeats.
“Regular heartbeats are easy enough to detect even if the signal is small because you can look for a periodic pattern in the data,” Gollakota said in a statement. “But irregular heartbeats are really challenging because there is no such pattern. I wasn’t sure that it would be possible to detect them, so I was pleasantly surprised that our algorithms could identify irregular heartbeats during tests with cardiac patients.”
The algorithms can detect motion of the chest wall and separate signals from breathing motions and ambient noise, according to the study.
RELATED: Google expands into health tracking using smartphone cameras to monitor heart, respiratory rates
“Specifically, the speaker emits an inaudible high-frequency sound that gets reflected back by the user's chest and received by the microphone array,” lead author Anran Wang, a doctoral student in UW’s Allen School, told Fierce Healthcare. “We designed a self-supervised learning algorithm to analyze the signal and obtain the tiny skin vibration caused by the heartbeat.”
Some study participants wore a Polar H10 Sensor System that outputs heart rate and intervals of R waves, called RR intervals. Others wore a CorSense monitor on their fingertip. They transmitted data to their smartphone using Bluetooth. Researchers compared results from the smart speaker to that of an ECG monitor. They found the difference in data to be not “medically relevant,” according to The Washington Post.
If a smart speaker could offer an inexpensive test for people, it could help diagnose cardiac arrhythmias, which cause strokes, according to co-senior author Arun Sridhar, M.D., assistant professor of cardiology at the UW School of Medicine.
“Availability of a low-cost test that can be performed frequently and at the convenience of home can be a game-changer for certain patients in terms of early diagnosis and management,” Sridhar said in a statement.
Researchers used ML to take signals coming from a smart speaker’s multiple microphones to spot an "elusive heartbeat signal,” according to the researchers. Just as a smart speaker like Alexa can spot a person’s voice even if a TV is playing or several people are in the room, the smart speaker can also pick out a heartbeat among other sounds.
“The self-supervised learning algorithm learns the spatial locations of the heartbeat by their characteristics meanwhile minimizing other interferences, such as environmental noise and respiratory motion,” Wang said. “It then focuses on the heartbeats and derives the beat-to-beat intervals.”
RELATED: Doctor Alexa will see you now: Is Amazon primed to come to your rescue?
The news follows a proof-of-concept tool UW researchers introduced last year that uses a smart speaker like Google Home or Alexa or a smartphone to detect if someone is gasping for air, which is known as an agonal breathing event. The app can then call 911.
Smart speakers as health tracking tools
In the future, medical professionals could use smart speakers to monitor heart rhythms as part of telehealth systems and to track people that require a contactless test due to skin sensitivity. The technology is also valuable for people in quarantine like during a health crisis such as COVID-19.
“Doctors and nurses can set up screening procedures using smart speakers in an empty room without worrying about spreading the diseases,” Wang said.
Telehealth and remote monitoring have grown significantly during the COVID health crisis, and companies such as Tyto Care have received new investments to help them ramp up remote monitoring systems.
RELATED: Smart devices can detect cardiac arrest by monitoring breathing: study
“Now our techniques enable heart rhythms detection using smart speakers, which is a step forward to achieve telemedicine,” Wang said. “It can potentially benefit millions of people who live far from medical services or do not have the necessary resources to check their health conditions to detect potential cardiac diseases at home.”
Smart speakers like Alexa have received attention from the healthcare industry before. Last year health insurer Anthem launched an enhanced version of its Alexa skill across 13 commercial insurance markets. The skill lets customers get in touch with an Anthem customer service agent or order prescription refills. People with any Alexa device or the Alexa mobile app can use the skill.
Tech giant Google also is leveraging its smart devices as health monitoring tools. The company just announced its updated Nest Hub smart home device has a sleep tracking feature. The smart display uses radar-based sleep tracking in addition to an algorithm for cough and snore detection.
Going forward, research with smart speakers will draw on work that allows the sensors to detect sleep apnea and cardiac arrest.
“We believe more and more medical applications can be developed in the future,” Wang said.