Google innovates personal healthcare using advanced generative AI

Google AI showcased new healthcare product innovations and research Tuesday – including AI-powered visual searches, diagnostics, and even a personalized AI-generated LLM health coach for FitBit.

The announcement is all part of the its annual Google Health Check-Up event, which is geared towards how the company is applying AI to the healthcare industry.

Google said the AI powered products and generative models are designed to help users better understand their own health data.

Google also said it is currently researching ways to create a version of the Gemini model – fine-tuned for the medical domain – which will be able to use advanced reasoning, understand a high volume of context, and process multiple modalities.

Using an LLM with advanced reasoning will enable medical professionals to sift and make sense of large quantities of text, images, audio, and video and find relevant information quickly, for example, buried in a patient’s medical history.

The company Check-Up featured three leading edge AI projects it has been working on this past year.

Visuals for Google medical searches

The first progress update will use AI to incorporate visual answers for users who search Google for any health-related topics.

“Some health questions are difficult to describe in words alone,” Google explained, “Our products and tools help you find answers to your health questions.”

Users can take a picture of an ailment on their body, for example a skin rash, and now search it with AI-powered Google Lens to find out more.

Images, text, and video will also be used to present information in search results to help users more easily understand health conditions and symptoms, such as neck pain, migraines, or kidney stones, Google said.

The feature will be rolling out worldwide in the next few months, including on mobile platforms.

FitBit's new Personal Health LLM

Next, the Google-owned Fitbit activity tracker gets its own generative AI upgrade to provide more tailored and personalized health insights based on a users unique needs and preferences.

Google announced its research department is working with FitBit to create the first Personal Health Large Language Model with AI features that can be used across all platforms.

The new FitBit Personal Health LLM – trained by health and wellness experts, doctors, and certified coaches – will be fine-tuned to deliver personalized coaching capabilities, actionable messages, and guidance individually based on a person's individual health and fitness goals.

The approach will help users get a fuller perspective on their personal health, ask questions, and create personalized charts of their own data, according to Google.

“Fitbit Labs, introduced last year, will give Premium users early access to some of the experimental AI features to test out and give feedback on,” Google said.

The FitBit labs feature will be available a limited number of Android users with the mobile app later this year.

Google FitBit personal health LLM
An example of Fitbit Labs features that will be available later this year to a limited number of Android users who are enrolled in the Fitbit Labs program in the Fitbit mobile app. Image by Google.

Last year, Google successfully released two commercial healthcare-specific LLMs, the Med-PaLM 2 and MedLM. The foundation models are currently available through the Google Cloud's Vertex AI platform and are being used by healthcare organizations worldwide to “build solutions for a range of uses — including streamlining nurse handoffs and supporting clinicians’ documentation,” the company said Tuesday.

YouTube expands health content

Google’s final AI powered project will involve bringing “authoritative, health-related content” to more audiences, with a focus on Spanish-language videos.

The company plans to offer a new AI-powered dubbing tool – called Aloud – that creators can use to streamline video translation and dubbing processes for free.

“The tool has allowed institutions, such as Mass General Brigham, which is expanding the program, to dub first-aid videos from English into Spanish, providing potentially life-saving information to more people.”

Additionally Google says it will offer Stanford Medicine Continuing Medical Education YouTube classes in Spanish for free to help underserved communities.

In other Google news Tuesday, the company announced some changes to FitBit, which it acquired in 2021. Users will now have to log into FitBit through their Google accounts, and as of March 18th, the activity tracker will now be known as the ‘Google FitBit.’