Melbourne psychiatrist requires AI note-taking consent for new patients

A Melbourne psychiatrist has refused to accept new patients unless they agree to allow her to use an AI scribe to transcribe their therapy sessions, a practice that is growing in popularity but raising concerns about data security and accuracy.

AI-driven note-taking tools are now used by two in five general practitioners in Australia, according to the Royal Australian College of General Practitioners (RACGP).

However, concerns have emerged about the security of patient data, how AI companies might use the information, and the accuracy of transcriptions.

In a registration form obtained by Guardian Australia, psychiatrist Dr Hemlata Ranga outlined the requirement to potential patients.

The form states: “I consent for use of AI transcription (such as Heidi health AI/ Microsoft) software to assist with notes taking during the appointments, for preparation of clinical letters, reports, and other clinical documents to assist in my clinical care.”

Patients are told that if they do not wish AI to be used, they will “need to arrange [their] regular doctor/ referring doctor to refer [them] to a different service provider”.

The RACGP reports that the uptake of AI scribes has doubled in 12 months, amid increased demand for mental health services.

One patient, who requested anonymity, told Guardian Australia they objected to AI being used in their sessions after being referred to Ranga.

The patient said: “The way [the form] was worded – so defensively – it made me think that maybe there are a number of other people who have said something and that it’s gotten to the point where they just thought: ‘I just don’t want to deal with their objections any more. Let’s just make them sign something that says that we have to use it.'”

The patient noted that finding a psychiatrist is not like going to another chemist, especially since Ranga came recommended, and they have not sought another psychiatrist.

The patient added: “She’s not easily replaceable, and the process of having to go back to your doctor, make an appointment and often wait on another wait list to get back in … all just because you don’t want to use AI … it’s really weird.”

Ranga, whose practice is at the Melbourne Clinic in Richmond, did not respond to a request for comment.

A spokesperson for the Melbourne Clinic said it does not direct its accredited psychiatrists on their use of AI, but understands they are increasingly incorporating it to help manage administrative load.

The spokesperson said: “In those cases, psychiatrists will disclose the use of AI to all patients and request consent prior to its use. If consent is not granted, they will not use AI.”

The spokesperson added that practitioners at the clinic are ultimately independent from the clinic.

Heidi AI, one of the most popular AI scribes, has been used in 115 million sessions in the past 18 months.

Dr Tom Kelly, the Melbourne co-founder of Heidi AI, said in March that data is processed in the country where the patient is located and is not used to train the AI or sold to others. The company uses third-party testing and auditing to keep data secure and aims to ensure high standards in transcripts, but noted that doctors still need to check their notes.

Tom Sulston, head of policy for Digital Rights Watch, said AI transcription is imperfect and often trained on male, white, heterosexual, English-speaking users, meaning mistakes are more common for other demographics.

He said there is also a “very real risk” of medical data being compromised or leaked, and patients wary of AI scribes may self-censor.

Sulston said: “That’s particularly important with healthcare that may carry stigma in wider society – sexual and mental health, for instance.”

He added: “AI is not implemented to improve healthcare outcomes or patient experience, but to reduce administration costs for a clinic.”

“In this scenario, patients are not the customer, they’re the product,” Sulston said.

According to the RACGP, consumer groups have raised concerns that consent conversations with patients over use of AI tools are either lacking, or there is a power imbalance between patients and doctors.

Sulston said no one should be denied healthcare because they do not want to be forced to give private data to AI, and called for stronger regulation, given the tools are now exempt from Therapeutic Goods Administration regulations because they do not diagnose patients.

Sulston said: “Regulators and legislators need to step-up and provide Australians with a legally enshrined right to refuse AI systems without facing repercussions to our health.”

The patient said people should be able to opt out of scribes and suggested they were not confident the notes made would be accurate.