Google Introduces Med-Gemini, A Set Of Powerful Multimodal Medical Models

As the capabilities of artificial intelligence (AI) and generative artificial intelligence (GenAI) continue to grow and improve, the technology expands its reach and impacts even more industries.

As a result, AI companies are striving to update and enhance their products and services to increase the scope of benefits and advantages offered by their AI technology, while also looking to stay ahead of the competition.

Google, one of the leading companies in the AI sector, is looking to do the same in the healthcare industry, by building on its Med-PaLM foundation models.

Researchers from different companies, all owned by Alphabet Inc., came together to build a new, highly capable and powerful set of generative AI models to enhance its offerings in the healthcare industry further.

So, what did Google researchers reveal? Let’s explore!
 

What Did Google Announce?

 

• Through a published research paper, researchers from Google Research, Google DeepMind, Google Cloud and the Alphabet-owned Verily introduced Med-Gemini, a family of highly capable, multimodal medical models built upon Google’s Gemini.

• As per the paper, the team used Google’s Gemini 1.0 and 1.5, which is the company’s new generation of highly capable multimodal models, to build Med-Gemini.

• This also means that Med-Gemini can process information from text, images, audio and videos with specialized fine-tuning.

• The model can even scour the web to offer more advanced clinical reasoning, more factually accurate and reliable answers.

• However, this feature may need to be updated in future generative AI models, as the team haven’t restricted the model’s search to more authoritative medical sources or to analyze the relevancy and accuracy of search results.

• The capabilities and methods used included advanced reasoning via self-training and web search integration, multimodal understanding via fine-tuning and customized encoders and long-context processing with chain-of-reasoning.

• “Med-Gemini inherits Gemini’s foundational capabilities in language and conversations, multimodal understanding, and long-context reasoning,” reads the paper, adding, “For language-based tasks, we enhance the models’ ability to use web search through self-training and introduce an inference time uncertainty-guided search strategy within an agent framework.”

• Med-Gemini achieves state-of-the-art (SoTA) performances on 10 out of 14 medical benchmarks, spanning text, multimodal and long-context applications.

• Furthermore, the researchers find that Med-Gemini surpasses the GPT-4 model family on every directly comparable benchmark.

• It also beat Google’s prior best SoTA by Med-PaLM 2 by a significant margin of 4.6%, hitting a state-of-the-art (SoTA) performance of 91.1% accuracy on MedQA (USMLE - United States Medical License Exams).

• Ahead of this, Med-Gemini also does better than GPT-4V by an average relative margin of 44.5% on 7 multimodal benchmarks, which includes the NEJM (New England Journal of Medicine) Image Challenges and MMMU (health & medicine).

• As per the researchers, Med-Gemini's performance shows promise in real-world use cases by “surpassing human experts on tasks such as medical text summarization and referral letter generation’, as well as for multimodal medical dialogue, medical research and education.

 

What Did The Researchers Say?

 

• Through the research paper, the researchers said, “We introduce Med-Gemini, a family of highly capable, multimodal medical models built upon Gemini. We enhance our models’ clinical reasoning capabilities through self-training and web search integration, while improving multimodal performance via fine-tuning and customized encoders.”

• “Moving beyond benchmarks, we also demonstrate the real-world potential of Med-Gemini through quantitative evaluation on medical summarization, referral letter generation, and medical simplification tasks where our models outperform human experts, in addition to qualitative examples of multimodal medical dialogue.”

• “The advances of Med-Gemini have great promise, but it remains crucial to carefully consider the nuances of the medical field, acknowledge the role of AI systems as assistive tools for expert clinicians, and conduct rigorous validation before real-world deployments at scale.”

Do you think this move by Google will help it capture a stronger position in the healthcare industry? Do you think Google’s competitors should make similar moves?

Let us know in the comments below!