Our dodgy headline for the week comes from a terrific new paper by the University of Queensland’s Bevan Koopman and Guido Zuccon on the major risks consumers are probably now taking when using generative AI to research their health conditions, much as search engines like Google were the boogeymen in the very recent past.

The research shows that contrary to received wisdom, providing large language models like ChatGPT with extra evidence can actually decrease their accuracy, particularly if you add in uncertainties like allowing an answer of “unsure” to a yes or no question.

The research shows that contrary to received wisdom, providing large language models like ChatGPT with extra evidence can actually decrease their accuracy, particularly if you add in uncertainties like allowing an answer of “unsure” to a yes or no question.

As the authors write, previous studies have recognised that ChatGPT can be used to spread misinformation about public health topics. “In our study, we empirically demonstrate this to be the case, including the systematic biases present in the current ChatGPT model that exacerbate this risk,” they say.

Still in its very early stages, Dr ChatGPT is experiencing the growing pains that Dr Google did, only a billion times more quickly. And now, with search engines integrating LLMs and bringing the technologies together in a process unhappily known as retrieval augmented generation (RAG), the risks are endless. Good times!

The good news is that the risks and benefits of AI for healthcare applications are being taken very seriously indeed, particularly though initiatives like the Australian Alliance for Artificial Intelligence in Healthcare (AAAiH). The bad news is that the pony is out of the traps with consumer applications, and there’s no bringing it back in again.

As for the new discipline of prompt engineering, heaven help us. We recently came across an AI policy expert on LinkedIn who was dedicated to “infusing machine cognition with human essence”, which prompted two things in us: one, a grimace, and two, a reason once again not to venture onto LinkedIn.

Elsewhere, old-fashioned health IT moves apace. We had a great story this week by our new New Zealand-based journalist Reesh Lyon on the development of a cloud-based clinical management system for inflammatory bowel disease (IBD) that has been seriously well thought through and allows patient interaction.

We had a wrap-up of our big trip to the Gold Coast for the fabulous ITAC conference, where data standardisation was a big topic despite AI making inroads too. The star of the show though was Abi, who despite butchering the German and Chinese languages with an appalling American accent still utterly charmed the audience.

Next week, more AI but also some of the age-old challenges of interoperability. We asked about this in our poll question last time: Are you confident of real progress on interoperability? Two-thirds said they were: 64 per cent voted yes, with 36 per cent voting no.

We also asked: If yes, are there real world examples? If no, what’s the main challenge? Here’s what you said.

This week, we ask: Has the horse bolted on consumer use of genAI for health information?

If yes, what should be done about it? If no, what is reining it in?

Vote here, and leave your comments below.