Why ‘empathetic’ AI can be too persuasive for Canadians to resist

Vered Shwartz says the human-like tone of AI-generated responses paired with their confident presentation can easily convince users that the information is trustworthy, even when it’s inaccurate.

New research shows artificial intelligence can be very persuasive with medical advice and experts say our faith in it is eating into a patient’s valuable time with an actual doctor.

The study, from researchers at the University of British Columbia, found that large language models, such as ChatGPT, are not only offering medical information, they’re also often more convincing and pleasant to deal with than real people.

“The conversations with large language models were more persuasive than the ones with people,” said Dr. Vered Shwartz, a UBC assistant professor of computer science and author of the book ‘Lost in Automatic Translation.’

“They were also considered more empathetic.”

Shwartz says the human-like tone of AI-generated responses paired with their confident presentation can easily convince users that the information is trustworthy, even when it’s inaccurate.

“When it’s written in such an authoritative way and has answers to everything you ask it, of course you assume it knows what it’s talking about,” Shwartz said.

Doctors say they’re seeing the effects first-hand. Patients are increasingly showing up to appointments having already made up their minds, often citing diagnoses or treatment suggestions from AI tools. Suggestions that are sometimes wrong, requiring their physician to take the time to try to talk them out of it.

“Oftentimes people get it bang on,” said Dr. Cassandra Stiller-Moldovan, a family physician in Colwood, B.C. “But other times, you spend a lot of time convincing them that that’s not exactly what’s going on. That education piece takes a lot of time.”

A team of Canadian researchers found when it comes to accurately diagnosing a health problem, services like ChatGPT can be wrong more often than they’re right.

Researchers at the University of Waterloo evaluated the performance of ChatGPT version 4, asking it a series of open-ended medical questions. Just 31 per cent of the responses from ChatGPT were considered entirely correct.

The Canadian Medical Association says the growing reliance on AI is understandable, given that millions of Canadians don’t have access to a family doctor.

“Part of the reason people go online is that they can’t get the access they deserve,” said CMA president Dr. Kathleen Ross. “When you’ve had a primary care provider over a period of time, you build a trusting relationship and can start having those difficult discussions.”

But with long wait times at walk-in clinics and emergency rooms, and fewer face-to-face interactions with trusted providers, many Canadians are opting to consult AI tools instead.