AI will transform primary care! AI does better diagnostics than doctors! Patients like AI responses to their portal messages better than their doctors’ messages! Providers save hours a day in charting with the help of AI scribes!

Lots of hype. What’s real?

I haven’t engaged in AI yet in my doctoring, so I have suspended my judgement, though I have a Luddite’s approach to technology. This past week I headed to Rhode Island for a family medicine residency reunion (fun!) and a few days of primary care didactics, including an introduction to AI in doctoring.

What I heard: AI isn’t here to replace doctors. Nor is it here to allow doctors to see more patients. Instead AI will be used by family physicians for more efficient charting, allowing us to keep seeing the same numbers of patients we’re already seeing without sacrificing our personal lives and burning out. The goal of AI in primary care is maybe improved quality and maybe improved patient satisfaction – but definitely more efficient charting so we don’t need a second work shift of “pajama time,” staying up late to finish the overwhelming charting required in the last two decades by inefficient point and clinic electronic medical records.

What I saw: a thoroughly UNimpressive demonstration of an AI product based on a large language model that was touted as “stunningly good” at translations. Its goal was to help take notes.

The three demonstrations: First an English to English doctor patient encounter. Second Spanish to Spanish. Third an English doctor – translator – Spanish patient encounter.

The first encounter hit a technological issue, and the demonstrators were unable to show us how their AI system decoded the visit in English. Technical glitches rendered AI useless.

The second, Spanish to Spanish interview, the AI note taker botched its rendition of the encounter. The patient came in saying “I have asthma, I need help.” Further questioning by the doctor revealed she’d never been diagnosed with asthma before, nobody in her family had it either, she’d never been treated for it, never taken any meds. But what she did have was shortness of breath, and after searching google thought she might have asthma. Or maybe cancer, she was scared. She was super overwhelmed at home, caring for three kids, and her symptoms got worse when she was alone chasing around after them. And she was so so tired. And stressed.

The doctor appropriately said – “we’re not sure exactly what’s going on with your symptoms. Let’s try albuterol for two weeks and check in to see how you respond. Let’s also have you talk to the behavioral therapist. And have you visit the lab to check for anemia with the fatigue.”

Coming out of the encounter, my doctor trained brain translated the doctor’s words to: 1) shortness of breath – asthma vs other, possibly somatization of stress and depression, possibly anemia. Let’s try asthma meds to see if it gets better, and if it does, it’s probably asthma. If no help from asthma meds and her labs are normal, it’s more likely to be how her mind and body are processing her stress – then we’ll explore her mental health more, maybe starting psychiatric meds, and make sure she has connected to behavioral therapy. 2) fatigue – check labs for anemia 3) psychosocial stress – behavioral therapy referral.

AI chose otherwise. 1) Asthma, not otherwise specified. 2) Fatigue 3) Stress.

The demonstrator, without thinking, clicked on AI’s chosen diagnoses.

Without a thinking doctor intervening, when the patient came in saying “I have asthma” (…or at least I think I do after an internet search…) then the primary diagnosis became “asthma.” And the note was written not to reflect what the patient actually said, but what they would have PROBABLY said if they actually did have asthma.

Because the doctor followed the suggestions of AI without thinking, clicking on the suggested diagnosis without processing if it was the correct one, AI generated a history that assumed the patient ALREADY had asthma, was ALREADY “taking the medications as prescribed” (despite the patient later saying no history of asthma, no family history, never tried meds). Then AI generated a plan that embedded the faulty assumption that she already had asthma and was already taking meds into the plan. The incorrect diagnosis of asthma generated instructions of “CONTINUE asthma medicines as prescribed.” Nowhere did it say “TRY asthma medicines for two weeks to see if you feel better, and find out if you have asthma”. The AI generated note simply did not reflect the actual encounter. The treatment and plan did not reflect the implicit differential diagnosis of shortness of breath with a plan for rule in and rule out of different possibilities based on the treatments and studies that the doctor had said. Instead AI offered a simplified fake plan based on a faulty self-diagnosis offered by the patient.

The English-Translator-Spanish AI adopted other short cuts of poor physicians. A chief complaint of “knee pain and back pain” generated a fake invented physical exam of knee tenderness and back tenderness. With other faulty diagnoses – the patient saying “I think I have diabetes” – created a diagnosis of “prediabetes” – which the clinician did not discuss once, but whose treatment, AI decided, after the doctor and patient talked about tylenol for the musculoskeletal pain – would be tylenol.

If this is the future of AI, and humans adopt it without thinking, pointing and clicking on AIs suggestions, and signing off on notes without reading them, we’re in trouble. Especially if future AI systems are trained on the faulty AI notes being generated by AI assisted charting today.

I recognize that I only saw one AI charting assistant of many on the market. Like the hundreds of EMRs that proliferated the market (and didn’t talk to each other across systems), there will be good ones and bad ones, terrible ones and better ones.

The American medical system doesn’t need AI to help with mindless charting (though I have been told it can be very helpful when paired with a human mind, and I didn’t see the note generated in the English-English encounter). We need a way to connect all of a patient’s data in a single place so the primary care provider can know what everyone else who has seen the patient has already done – the data from all specialists, imaging studies, laboratories, hospitalizations, medications in one place. I would love it if AI had already synthesized this so I don’t have to spend hours reviewing studies from hospitalizations and signing and faxing releases of information and waiting for imaging studies that never come to guide next steps. THAT has the power to transform the care I provide my patients – easeful integration of information across fragmented systems, with AI synthesization. I hope there is an AI system working on that – please let me know if there is! And I’ll keep looking for a different AI system to help me with charting I can trust!

Full disclosure: as I prepare to publish this article, an AI embedded in wordpress just suggested a better title for me, that I chose (…revolution or risk?!). It also suggested two others that didn’t reflect what this article was about, that I didn’t choose. AI DOES have the power to make our work better, when used in conjunction with human minds.