Clinic calls police after woman refuses to allow AI to take notes

An Australian artist’s protest against the use of artificial intelligence (AI) to take notes during her dental appointment quickly spiralled into chaos. Here’s what patients need to know about their rights.

When Caerwin Martin from Melbourne, Australia, booked a periodontist appointment in an undisclosed clinic, she didn’t expect that it would end with a police encounter.

Martin, who shared her story on Facebook, said the clinic called law enforcement after she refused to let the doctor use AI to take notes and write referrals during her visit.

While the police officers let her go shortly after the interview, she is still bringing the case to the medical board.

Martin told the media that she objected to the use of AI for ethical reasons and because she doesn’t want it to have access to her medical records.

“I’m a musician and I'm offended that the government had to step in to stop AI from hoovering up all of our stuff,” she told DailyMail.

Even if the situation was a grave misunderstanding, it raises important questions about patients’ rights when AI is used in medical settings.

No federal law requires the disclosure of AI use in the US

Regarding the situation in the United States, Star Kashman, a founding partner at Cyber Law Firm, says there is no blanket federal rule in forcing doctors to inform patients about the use of AI during a visit.

However, disclosure and consent can also be required indirectly through other pathways. For example, if the visit is virtual and is audio or video recorded, some relevant state wiretap and recording-consent laws may apply.

Information about a patient's health conditions, records, and personal information is considered protected health information (PHI), primarily regulated by HIPAA, short for Health Insurance Portability and Accountability Act, which indirectly addresses AI use.

Common consumer AI platforms rarely offer BAAs or proper privacy protections to safely be used by healthcare providers for patient health information, including note-taking during appointment.

Ron Harman King

Under HIPAA, the use of any information system or software for collecting and storing PHI poses risks, because the PHI is typically accessible by the system’s vendor, including an AI vendor, says Ron Harman King, a CEO at Vanguard Communications Group.

Suppose an AI vendor lacks double encryption and a Business Associate Agreement (BAA), which binds a contractor to the same legal privacy safeguards as healthcare providers. In that case, it is well-served to require patients to sign a carefully drafted disclosure and waiver form before using AI.

“Common consumer AI platforms rarely offer BAAs or proper privacy protections to safely be used by healthcare providers for patient health information, including note-taking during appointments,” King says.

In other words, some popular AI note takers aren’t HIPAA-compliant and don’t offer BAAs, while others can handle PIH only under special subscriptions.

With federal laws lagging behind a rapid rollout of AI tools, some states are stepping in, requiring disclosures when AI is involved.

Aaron Maguregui, a partner and AI expert at Foley & Lardner LLP, notes that California, Texas, and Utah have enacted rules that require or encourage notice when generative AI is used in patient communications or clinical decision support.

“Broader informed-consent and ethical principles may also compel disclosure when AI meaningfully influences care. The rules remain fragmented but are tightening quickly,” Maguregui tells Cybernews.

Patients are generally wary of AI applications in healthcare. Nearly four in ten (37%) Americans believe that using AI in healthcare would compromise the security of patients’ records, according to a 2023 survey.

When it comes to making health decisions, skepticism of AI is even stronger – nearly 58% say they have low trust in their healthcare systems to ensure an AI tool would not harm them.

“Being your own healthcare advocate”

Despite patient skepticism, the technology is here to stay. Two in three American physicians now use AI, including for tasks such as writing visit notes, discharge instructions, or care plans.

As in every interaction with providers, patients must be diligent to be their own healthcare advocates, says Lani M. Dornfeld, a healthcare attorney at Brach Eichler. This includes asking questions about the use of AI tools during patient encounters.

“What tools are being used, why such tools are being used, what are the risks and benefits such tools offer, will the patient’s information be disclosed to outsiders, and what human oversight is involved,” Dornfeld explains.

While patients may request limits on data use, providers can decline if restrictions interfere with care, Maguregui warns.

Kashman says opting out of using AI tools in a medical setting should be recorded in a patient's chart, and the patients shouldn’t forget to inform the front desk.

Unlock more exclusive Cybernews content on YouTube.