Microsoft Research announces CoExplorer and MAIRA-2, groundbreaking AI advancements in healthcare and productivity

Microsoft Research has announced two major developments in its Research Focus for the week of June 24, 2024. These innovations, MAIRA-2 and CoExplorer, showcase Microsoft’s commitment to pushing the boundaries of AI technology to improve medical diagnostics and enhance remote collaboration.

MAIRA-2: Revolutionizing radiology report generation

The first breakthrough, MAIRA-2 (Multimodal AI for Radiology Assistance), represents a significant advancement in the field of medical imaging and diagnosis. This large multimodal model combines a radiology-specific image encoder with a large language model (LLM) to generate detailed and accurate radiology reports from chest x-rays.

MAIRA-2 addresses several key challenges in automated radiology reporting:

Improved image understanding: The model’s radiology-specific image encoder is trained to recognize and interpret subtle details in medical images, particularly chest x-rays.

1. Comprehensive Input Processing: Unlike previous models, MAIRA-2 can process multiple inputs simultaneously, including:
   1. • Current frontal chest x-ray
      • Current lateral chest x-ray
      • Prior frontal chest x-ray
      • Prior radiology report
      • Indication, Technique, and Comparison sections of the current report
2. Reduced AI hallucinations: By incorporating more contextual information, MAIRA-2 significantly reduces the likelihood of generating false or irrelevant findings, a common problem in AI-generated medical reports.
3. Grounded report generation: MAIRA-2 introduces the novel concept of “grounded” reporting, where the model can localize individual findings on the image. This feature enhances the transparency and interpretability of AI-generated reports.
4. State-of-the-art performance: The researchers report that MAIRA-2 has established a new benchmark for findings generation on the MIMIC-CXR dataset, a widely used benchmark in medical AI research.

To evaluate the performance of MAIRA-2, the research team developed a novel framework called RadFact. This framework leverages the reasoning capabilities of LLMs to assess the factuality of generated sentences and the correctness of spatial localizations in the reports.

Dr. Fernando Pérez-García, a senior research machine learning engineer at Microsoft Research Health Futures, commented on the significance of MAIRA-2: “This model represents a major step forward in our ability to assist radiologists with AI. By providing more accurate and contextually relevant reports, we can help improve diagnostic accuracy and efficiency in healthcare settings.”

CoExplorer: Reimagining remote collaboration with GenAI

The second innovation, CoExplorer, addresses the challenges of planning and executing effective video meetings in our increasingly remote work environment. This adaptive meeting prototype harnesses the power of generative AI to create a more intuitive and productive meeting experience.

Key features of CoExplorer include:

1. Automated meeting phase generation: Using the meeting invitation text, CoExplorer can generate a detailed list of likely meeting phases, helping participants prepare and stay on track.
2. Collective feedback integration: The system generates a “meeting focus tool” that acts as a conversation seed, capturing topics that attendees want to discuss during the meeting.
3. Dynamic phase adaptation: CoExplorer can update the meeting phase list based on attendee feedback, ensuring the meeting structure remains relevant and flexible.
4. Intelligent window management: The system detects phase changes during the meeting and automatically adjusts the window layout, displaying appropriate resources and tools for each phase.
5. 2D and VR versions: CoExplorer is available in both traditional 2D interface (CoExplorer2D) and virtual reality (CoExplorerVR) versions, catering to different remote work setups.

The research team, led by Gun Woo (Warren) Park, Payod Panda, Lev Tankelevitch, and Sean Rintel, conducted a user study with 26 participants from a global technology company to evaluate CoExplorer’s potential. The findings suggest that the system can help meetings stay on track and reduce workload for participants.

However, the study also highlighted some concerns regarding user agency, trust in AI-generated content, and potential disruption to traditional meeting norms. The researchers emphasize the importance of striking a balance between automation and user control in future iterations of the technology.

Sean Rintel, a senior researcher at Microsoft, explained the vision behind CoExplorer: “We’re exploring how generative AI can transform the entire meeting lifecycle, from planning to execution. Our goal is to reduce the cognitive load on participants, allowing them to focus on meaningful discussions and decision-making rather than administrative tasks.”

Future directions

As Microsoft continues to push the boundaries of AI technology, these developments in MAIRA-2 and CoExplorer demonstrate the company’s commitment to creating practical, impactful AI solutions for real-world challenges in healthcare and workplace productivity.

With ongoing refinement and user feedback, these technologies have the potential to revolutionize how we approach medical imaging analysis and remote collaboration in the coming years. As always, the key to successful implementation will lie in balancing the power of AI with human expertise and judgment.