Despite the convenience and benefits of using Artificial Intelligence (AI) for health advice, there are significant risks that health information seekers should be aware of. These include the potential for misinformation, breaches of confidentiality, and biased healthcare recommendations. Here’s what benefits brokers need to know.
Why Are More People Turning to AI for Health Advice?
With healthcare access sometimes being slow and/or costly, AI systems like ChatGPT can seem like an easy solution. Many people appreciate that chatbots provide responses 24/7, without the discomfort of sharing personal health details with a human provider. According to the latest KFF Health Misinformation Tracking Poll, 1 in 6 adults use AI chatbots at least once a month to find health information and advice, rising to one-quarter of adults under age 30 (25%).
However, there’s a growing concern over the quality and safety of health information offered by these AI tools, and health advice seekers should be cautious when using them.
Top Four Risks of AI in Healthcare
- Quick Access Isn't Always Thorough Enough: While AI chatbots can respond instantly to health-related questions, they often lack the depth and accuracy required for comprehensive medical guidance. AI tools rely on data patterns rather than personalized medical histories or diagnostic tests, which means they can miss critical nuances in a patient's condition. For example, an AI tool may suggest general treatments for symptoms but overlook underlying issues that would be caught through thorough exams, lab work, or blood tests.
- AI Can't Replace Human Expertise: Chatbots are programmed with vast amounts of data, but they lack the nuanced understanding that healthcare professionals bring to the table. Unlike doctors, AI cannot conduct physical exams, consider subtle patient cues, or adjust treatment based on the entirety of a patient’s medical history. KFF also found that most adults aren’t confident they can tell fact vs. fiction regarding information from AI chatbots. AI tools, however convenient, cannot easily replace the trust and confidence found in human experts.
- AI Can Provide Biased Healthcare Recommendations: The Lancet Digital Health put this risk to the test. They assessed GPT-4’s diagnostic and treatment recommendations and found that across 19 independent cases, GPT-4 was “significantly less likely” to recommend advanced imaging for “Black patients compared with White patients.” Additionally, GPT-4 was “significantly less likely to rate stress testing of high importance for female patients than for male patients.” [Note: Stress tests show how well your heart works when it’s pumping hard.] Overall, the AI tool was much less likely to recommend the stress test relative to the cardiologists in the study.
- AI Tools Can Leak Private Employee Information: According to The HIPAA Journal, one AI company, Cense AI, exposed 2.5 million patient records over the internet. This breach allowed unauthorized access, with the potential for data to be viewed, altered, or downloaded while it was accessible. This serves as a clear example of how AI can fail and inadvertently put patient security and safety at risk.
What Should Brokers Do Next?
As part of their role, brokers help employers stay up-to-date on trends in healthcare that may impact their benefits offerings. With AI becoming more prevalent in healthcare, brokers should be prepared to educate employers on both the potential benefits and the risks, helping them create benefits packages that promote safe, informed healthcare decisions.
If employees rely on AI tools for health advice and receive poor or inaccurate guidance, it can lead to delayed or inadequate care, which ultimately affects employee well-being and productivity. Brokers should ensure that health resources within benefit plans encourage employees to seek professional medical advice rather than relying on AI tools alone.
Additional Resources:
- Brokers’ Corner Podcast—watch and subscribe to the Brokers’ Corner podcast, which dives into the topics that affect your agency and industry and identifies strategies so you can protect and grow your book of business
- BerniePortal Brokers’ Council—a council of benefits brokers from across the country that advises BerniePortal on industry concerns, trends, and the ways technology can best support their agency and employer groups
- BerniePortal for Brokers—leveraging technology to increase your agency valuation and support your employer groups is easier than ever with BerniePortal’s software solution, built for brokers by brokers