Tom and Amy recently discussed the impact of AI in the health sector and challenges and opportunities this technology presents. Below is an excerpt from their conversation.
With the dramatic surge in interest and investment in healthcare AI, what legal and ethical challenges do you see on the horizon in the healthcare sector?
Amy: That is a tough question because AI is being used to help improve patient care and outcomes in many ways. However, for the sake of this conversation I’ll focus on what this could mean for how we diagnose and treat patients.
I work closely with innovative medical device companies. These technologies have become increasingly sophisticated, collecting and feeding data back to clinicians to analyse and make decisions on treatment—it’s very much a human-led process. Legally and ethically, this ‘human-in-the-loop’ is how these companies most commonly operate, but the technical ability exists in some cases for algorithms to make a diagnosis. It isn’t difficult to imagine algorithms playing a pivotal role in diagnosing and prescribing treatment for patients.
However, it does pose legal and ethical questions. How will the industry adjust, and what will patients come to expect as these technologies advance and improve?
Tom: This seems like a quantum leap in care delivery, but as someone who wears a fitness tracker and checks my data throughout the day, I can see how having access to this data can help inform doctors and patients alike. As we increasingly rely on these technologies, there needs to be a way to prioritise patient safety.
How can healthcare organisations balance the promise of AI to improve patient care and efficiency with the risks associated with its use?
Amy: Again, there are many possibilities, but if we think about electronic health records (EHRs) we can see significant improvements in care and efficiencies achieved by digitising patient data. As we harness AI in EHRs, I can see what I’ll refer to as “micro adjustments” being made where we allow these technologies to make more recommendations to clinicians based on biometrics and patient and family history. This will help enable clinicians to evaluate their patients more quickly and thoroughly and make more informed decisions for diagnosis and treatment.
Tom: I can see how AI will play a key role here, but to ensure sound administration of care—and more importantly patient safety—organisations will need to proceed prudently and with a mission-driven focus.
Amy: I completely agree with you, Tom. I advise my clients that they can do precisely this by showing them their work. Processes must be well documented. There must be transparency and accountability to ensure that patient safety is paramount. The goal isn’t just to change care but improve it.
How can healthcare organisations identify and mitigate algorithmic biases?
Tom: Healthcare organisations have been going through a lot over the past few years, and providers find themselves facing fierce fiscal headwinds. Understandably, these organisations are looking for automation to help improve efficiency and quality of patient care. However, AI must be deployed thoughtfully and monitored to identify and counter bias.
Amy: Tom, you’re correct. This is something which executives need to take seriously. This happens with AI because many large language models (LLMs) ‘hallucinate’. What I mean is that LLMs can generate incorrect responses because they can lack the ability to accurately assess the reliability and currency of data. This can lead to potential inaccuracies in their outputs. This can be accounted for and overcome with rigorous human oversight, carefully training these systems with the right information and putting safeguards in place, especially measures like easily retrievable citations to underlying sources.
What key considerations should compliance leaders, executives and board members address as AI becomes increasingly integrated into healthcare?
Tom: Artificial intelligence has been around longer than most people realise, but has only recently become mainstream vernacular. It reminds me of when the internet took off. The potential is mind-boggling, but there is also a spectrum of associated risks.
Amy: I like your reference to the invention of the internet. Satya Nadella said, “It’s the difference between life before and after the light bulb”. This technology is now very much top of mind with executives and board members, and they are looking for guidance on how to proceed.
Tom: I agree with you, Amy. Some guiding principles were included in BRG’s AI and the Future of Healthcare. The report surveyed healthcare executives and interviewed thought leaders, and from that research made helpful recommendations, including:
• Engage proactively with regulators. Collaborate and be part of the discussion to help shape the developed regulatory framework.
• Advance thoughtfully. Ensure AI is helping to solve meaningful problems for your organisation.
• Track and adapt to new technologies. Monitoring AI advances can better position your organisation in the future.
• Have a robust governance model. Establish a strong interdisciplinary governance model—don’t assume IT is solely responsible for managing this.
• Build on existing AI efforts. Continue to build on your investments and target AI solutions which solve specific challenges.
CONTACT INFORMATION