Google expands into health tracking using smartphone cameras to monitor heart, respiratory rates

Google Health developed a new feature to measure respiratory rate. Users place their head and upper torso in view of their phone’s front-facing camera and breathe normally to capture that measurement. (Google Health)

Google is stepping up its health-tracking game with a new tool to help consumers monitor key vital signs.

And while Google just closed a deal acquiring Fitbit, it says this new product doesn't need a wearable. 

Instead, Google Health, the tech giant’s health and wellness division, has added new features to its Google Fit app that enables users to take their pulse just by using their smart phone's camera.

“We’re working on ways to unlock the potential of everyday smart devices. We’re exploring how to leverage sensors that are becoming more ubiquitous to support health and wellness,” said Shwetak Patel, director of health technologies at Google Health, during a briefing with reporters Wednesday.

“There is a paradigm shift in health measurement. Our team of researchers, engineers, and clinicians are exploring how everyday devices and inexpensive sensors can give people the information and insights they need to take control of their health," he said.

The new features will be available in the Google Fit app for Pixel phones in the next month, with plans to expand to more Android devices, the company said.

RELATED: Apple steps up its focus on health with new blood oxygen sensor, population health initiative in Singapore

The measurements aren’t meant for medical diagnosis or to evaluate medical conditions but can be useful for people using the Google Fit app to track and improve day-to-day wellness, Patel said. The health tracking tools do not provide any feedback based on users’ measurements.

Google Health developed a way to measure respiratory rate using a smartphone camera and a computer vision technique called optical flow to detect subtle movements in the chest. Users place their head and upper torso in view of their phone’s front-facing camera and breathe normally to capture that measurement, the company said.

To detect heart rate, Google Health uses smartphone cameras to detect subtle color changes in the fingertip that happen when freshly oxygenated blood flows from the heart through the body, a process called photoplethysmography (PPG). Users place their fingers on the rear-facing camera lens to measure heart rate.

Google also is exploring the ability to measure heart rate from color changes in the face.

With the aim of making the features work for as many users as possible, Google said it completed initial clinical validation with people of different health status, in different ambient lighting, different skin tones, as well different heart rate ranges, such as users sitting at rest and users elevating their heart rate by briefly exercising.

RELATED: Google closes $2.1B acquisition of Fitbit as Justice Department probe continues

The company plans to share the results of these studies in a preprint publication in the coming weeks, and is seeking publication in a peer-reviewed journal, company officials said.

For the respiratory rate tracking feature, researchers examined its accuracy among healthy individuals as well as those with respiratory conditions that might impact measurement. Google Health’s algorithm is accurate within 1 breath per minute on average on both groups, said Jiening Zhan, technical lead at Google Health.

The heart rate measurement algorithm is accurate within 2% on average, Zhan said.

“We’re exploring a variety of sensors to bring to bear for health and wellness. The smartphone is a good starting point and we’re exploring other devices in our product lines,” Patel said.

Last month, Google closed its deal to buy fitness tracking giant Fitbit for $2.1 billion. It’s not clear how Google’s new health-monitoring features will fit into Fitbit’s business.

RELATED: Google moves into health research with mobile app, starting with respiratory illnesses like COVID-19

“We’re excited to collaborate with Fitbit and work on health and wellness. It’s still very early to figure out how we can align our efforts,” Patel said, noting that the acquisition closed just a few weeks ago.

The consumer-facing health monitoring market is crowded with smartwatches and wearables that can monitor steps taken, heart rate, sleep cycles, and blood oxygen levels. Compared to a Fitbit that can continuously track heart rate, Google’s health-tracking features require users to actively measure their vitals using the camera.

Patel said smartphones are more ubiquitous than wearables and the accessibility of phone cameras will help more people to monitor their health.

“Relatively few people in the U.S. actually have wearables. We know a lot of people in disadvantaged populations don’t have wearables and aren’t benefitting from health tracking. Equity is an important part of Google’s mission,” he said.

Currently, Google is focused on using the features for general health and wellness but there is the potential to use the technology as part of a medical device, which would require approval from the Food and Drug Administration.

“Work in the technology and research community suggests the technology might work for sick patients but there is not enough testing and validation. We’re definitely working on those use cases and are exploring that,” said Ming Jack Po, product management at Google Health, during the briefing with reporters.