Introduction

The integration of artificial intelligence (AI) into medical diagnostics has revolutionized various fields, including obstetric ultrasound. AI-powered chatbots like ChatGPT-3.5, ChatGPT-4.0, and Microsoft Copilot are being evaluated for their potential to assist healthcare professionals and patients. However, while these tools offer promising capabilities, they also present certain limitations that must be acknowledged.

Evaluating AI Performance in Obstetric Ultrasound

A recent study assessed the performance of ChatGPT-3.5, ChatGPT-4.0, and Microsoft Copilot in answering obstetric ultrasound-related questions and analyzing ultrasound reports. The findings revealed:

  • Accuracy in Answering Questions: ChatGPT-3.5 and ChatGPT-4.0 achieved an accuracy of 95.0%, outperforming Microsoft Copilot, which scored 80.0%.
  • Consistency: ChatGPT-3.5 and ChatGPT-4.0 demonstrated consistency rates of 90.0% and 85.0%, respectively, compared to Copilot's 75.0%.
  • Report Analysis: In analyzing 110 obstetric ultrasound reports, ChatGPT-3.5 and ChatGPT-4.0 showed superior accuracy over Copilot, with all three models exhibiting high consistency and the ability to provide recommendations.

These results suggest that large language models (LLMs) have the potential to enhance clinical workflows by improving patient education and communication regarding obstetric ultrasound findings. However, the study also emphasized the necessity of physician supervision due to instances of inconsistent and inaccurate responses, as well as cybersecurity concerns.

Understanding AI Hallucinations in Medical Contexts

A significant challenge in deploying AI in healthcare is the phenomenon of "hallucinations," where AI models generate plausible but incorrect or nonsensical information. In medical settings, such inaccuracies can have serious implications. For instance, a case reported in JAMA Otolaryngology–Head & Neck Surgery highlighted an AI chatbot providing fictitious data, underscoring the need for vigilance when integrating AI into clinical practice.

Enhancing AI Reliability in Healthcare

To mitigate the risks associated with AI hallucinations, researchers have developed frameworks like HALO (Hallucination Analysis and Learning Optimization). HALO aims to improve the accuracy and reliability of medical question-answering systems by detecting and reducing hallucinations. By generating multiple query variations and retrieving relevant information from external knowledge bases, HALO enriches the context provided to LLMs, thereby reducing the likelihood of hallucinations.

Implications for Clinical Practice

The integration of AI into obstetric ultrasound and broader medical diagnostics offers several benefits:

  • Improved Patient Education: AI can provide patients with accessible explanations of ultrasound findings, enhancing their understanding and engagement in their care.
  • Support for Clinicians: AI tools can assist healthcare providers by offering preliminary analyses and recommendations, potentially reducing workload and supporting decision-making processes.

However, these advantages come with important considerations:

  • Need for Supervision: AI-generated information must be reviewed by qualified healthcare professionals to ensure accuracy and appropriateness.
  • Addressing Hallucinations: Continuous monitoring and refinement of AI models are necessary to minimize the occurrence of hallucinations and ensure patient safety.

Conclusion

AI-powered chatbots like ChatGPT and Microsoft Copilot hold significant promise in enhancing obstetric ultrasound diagnostics and patient education. While they demonstrate high accuracy and consistency, the potential for inaccuracies and hallucinations necessitates careful oversight. Ongoing research and development are crucial to harness the benefits of AI in healthcare while mitigating associated risks.