CME | AI as a partner in whole-person care: Empowering clinicians, enhancing workflows

Welcome. My name is Olusola Adegoke, MD, and with me here is Lance Braye, MD. We’re going to be talking about AI as a partner in the whole concept of whole person care. And I know that a lot of people have dabbled, played with AI. Some people are AI skeptics, some people are AI curious, some people are worried about what AI is going to do to us.

Those are the things that we really want to explore here. And the big thing too, as we're getting into this, you know, in the title we mentioned AI as a partner, right? This isn't really about how do we set this so we can go off and, you know, go lift weights or play golf or anything during the day. It's really about how we use it to augment what we do and really succeed at what's always been our job and will remain to be our job.

From a disclosure standpoint, unfortunately, we're not tech bros, we are just family docs. We don't have anything to disclose here. Learning objectives. So what are the things that we really want to highlight in this presentation? Again, big caveat is small group leaning into that conversational piece. We really want to talk about what are the capabilities, what are the limitations, what are the ethical considerations of the use of AI within your workspace.

We might explore how you're using AI also outside your workspace, but it's very focused on clinical workflows. We also want to explore the documentation piece and workflow integration and optimization, and then we want to talk more about how do you take the things that you've learned here, and how do you take it back home and start thinking about how are we truly integrating AI into our workflows in our clinic?

Yeah, because you'll see as this goes on, and I mean, all of us that have been watching this, I'm sure if you're in this room, you're somewhat interested in AI, have probably been following the news headlines, but this thing has changed so rapidly, even in just about the six months that we've been working on this presentation, right?

And so our hope for this isn't just to leave you with a little bit of like, "This is what AI is," but also more for you to really be empowered to see how you can leverage it when you get back to work. Questions will come up. We will try as much as possible to address your questions in real time. Again, that's our picture. We're thinking of using an AI-enhanced picture, but we felt we actually look better in the real thing.

So limitations of AI. Wait, is your, is your picture AI or-

No, that's my real self.

Oh, that's AI me.

Oh. Nah. Little different. So, that was one of the cool things as we, as we were putting this together of just kind of like playing around with like, could AI do this? So yeah, I did not—I don't even own a tie that that's that color. My stethoscope has an Incredible Hulk little dude on it. So yeah, no, that's completely AI. Well, that's the real me. He's AI. Yeah. I'm the real person. Yeah. Tells you who is AI skeptic- Yeah ... and who has fully embraced AI.

And so, a little bit more about each of us before we get started. So yes, I've always been one of those tech nerds just from like kind of having my Game Boy attached to my hip as a kid to just always wanting to do the latest and greatest technology.

Unfortunately, I'm not a tech bro, but I'm the best thing I could be was as a family doc. And so, with that, I'm super excited about what it really leverages for us to do. Professionally, I'm the director of family medicine at a rural FQHC in Greenwood, South Carolina, and I know for me an exciting thing I see about it is when we are in that kind of environment where we don't necessarily have all the funding and larger hospital systems, we don't necessarily have the workforce that we could draw from like larger cities, that maybe AI is how we can finally, you know, is a lot of those like really basic tasks, kind of close that gap and move forward.

I work in a huge health system that spans urban to rural, and I oversee the clinical informatics mostly in the inpatient space. So when I was coming out to this conference, I felt, geez, I'm going to be away from AI, because over the last one year, every single day of my life has been AI this, AI that. I get away from work, I go on the news, it's AI this, AI that.

And I now have a sixth grader that came home and told me that one of her classmates actually gave a presentation about the pitfalls of using AI. So again, it is everywhere, and hence why we're here. All right. And so welcome to the show. We were not kidding with y'all. This is actually going to be a live podcast.

That was a style we wanted to take with this as being such like a tech-heavy topic with AI. And so what we're going to do is we're going to condense this whole season down into one presentation. So we are going to have about four episodes. They'll be about 10 minutes each. Sol and I will be discussing topics.

We're going to go through the episodes in a bit, and then you all have your show notes that have been handed out to you to kind of help you guide along with things through the season. And then after each 10-minute episode, we're going to give you all five minutes in your groups to really kind of discuss what was going on, just like you would do kind of with a regular podcast.

You know, Curbsiders comes out this week, and then you talk about it with your friends over the next couple, and then you hear the next episode. But during the episodes, please feel free to raise your hand, put your questions down on that QR code we had you scan, because we want this to be interactive, you know. So these episodes are really kind of more in the, in the style of like a live stream podcast if I don't say so myself.

So episodes, what is AI? Can it be controlled? How can I use it? How will we use it collectively? Those are the topics that we're going to explore here. And please raise your hand if you do have questions, if you want to interject, or we have two seats here if you're also interested in joining the conversation.

So what is AI? Show of hand, who has been using AI? Who has used AI in the last... Keep it up, keep it up, keep it up. Who has used AI in the last 24 hours? Impressive. Who is... You can keep it down. Who is an AI skeptic in the house? Good. Skepticism is good. Excellent. Yeah, so as we just kind of shown within our own little sample size of this group, the reason we're talking about this is because it, it's a thing, it's happening, and it's kind of like you just, you're just not going to be able to really get away from it, right?

When we first started making this presentation, it was about the surveys at that point said maybe about sixty-six percent of us as U.S. physicians were using AI, and then a few days ago, we realized that number has jumped up to 80%, which means maybe right now it's, like, 83% or something like that.

Whole point is this thing is really growing. And when you think of where we started from, AI's really kind of become very ubiquitous now. But just as recently as 2023, it was barely 40% of us that were using this thing, right? And a lot of us use it in our day-to-day probably for administrative tasks.

Raise your hand if you use an AI scribe or something like that. Right? Like, a good chunk of us. I know my organization, we have Epic, and so we also have the AI summary of the chart. But as we see it's not really there yet with us, with it kind of replacing what we do best, right, which is that clinical reasoning.

And so with all that, what we want to do talking about this is, like, what is this gap that we have? Because, you know, we have AI, we have these capabilities. But I know for me, this has always been my sticking point even before AI became a thing when it was just EHRs. Like, I wish that my EMR could do as much as my Xbox could do.

Like, there was always a part of me that just wondered if Microsoft or Apple just decided to dabble in the EHR game, oh man, it might be a world changer.

Excellent, excellent thought. And I think, again, it's the blessing and the cost of technology. It's the understanding that as clinicians, as physicians, clinician is my buzzword at work.

So as physicians in the house, one of the things that really makes it difficult from an AI standpoint is there's a lot of hype around the whole concept of AI within a health space. One of the huge drivers of how AI is going to be your Xbox is workflow designs. How do you actually think about your day as a hospitalist, your day as a clinic doc, and how do you start thinking about building and integrating AI into those workflows to make AI work for you versus having a tech bro build a tool based off their perceived workflow for you?

Those are the things that we really want to explore here. One of the things I tell people about AI is AI is embedded in every single thing we do. Texting, if you start using your predictive text message, and you see that magical pop-up, "Where are you?" That is AI. It's as simple as that. It is as complicated as running massive algorithms at the back end.

So when we talk about AI, we wanted to get away from the complexities of what people throw at you about what AI is and what AI is not. And we really wanted to frame this from a clinical standpoint of what is AI from a physician standpoint and in the healthcare space. We think about narrow AIs, which is very specific for a very specific task.

So you think about one of the AI FDA-approved tools is for diagnosing diabetic retinopathy. It is specifically done for that. You have some tools that will read chest X-rays. You have, for people that use Epic, you have your sepsis Epic predictive model. It is an AI-generated model that is a narrow tool. And then we have the generative AI.

That's the large language model. That's where you see ChatGPT, you see Claude. Those are the common ones. And what they do basically is they just predict the next word. It is predictive, which is why hallucination becomes a big thing. It's gone through multitude of data, and it's trying as much as possible to predict the next best thing.

Yeah. And I think too when we're talking about AI and what it is AI, I find it very helpful to kind of boil it down to kind of the first principles we're talking about in artificial intelligence, right? So artificial, obviously fake, but, you know, when we think of intelligence, like what, what does intelligence really mean, right?

Intelligence I think at its most basis, like this thing can like take information, process it and spit it out. You know, a lot of us in the room are probably biology majors, and I still kind of read some of the new biology findings from time to time, and we're talking about like intelligent life forms sometimes just because there's a bacteria that knew don't go there because it's hot, right?

And so I think sometimes it really kind of helps calm down that hype because it's like we're just talking about something that can process and think. And I know just in the ways I've used AI in my own personal life, like I used Claude to create an app to help me just kind of work on my budget. So that's like a narrow AI because it just looks at my spendings from my checking account and says, "You messed up again." I'm like, "I know I did." But it was worth it. All right. And then you get into, like, does anyone know a large language model? Just raise your hand if you're familiar with what that actually creates a large language model. Okay. So with that, the large language models, it's really more like these AIs, they're thinking and they're processing a little bit deeper from like that narrow one, like it was coded just to look at my bank data and then say, "You made your budget or you didn't."

But these large language models, these things like have to learn from somewhere, right? And so again, thinking of what intelligence is, when we were in med school, we were kind of like a large language model. We just got a bunch of textbooks and lectures stuffed in our head, and then someone said, you know, "Krebs cycle," and then we went blah, blah, blah, blah. Then we just spit stuff out. Yep. And that's kind of what a large language model does. It thinks, it processes the information, but only based off of what was fed into it. So some of you may have seen recently in the news, I think it was ChatGPT kept talking about goblins or something for some weird reason, and then Claude was telling everyone, "You should go to sleep." And I think sometimes we have a tendency to want to humanize these things. Like, "Oh, Claude cares about me," but it's like, no, the data that was fed into Claude is more like, "Humans don't get enough sleep." Yeah. And so Claude is like, "This is me processing information. You really should get some sleep, man."

We saw the same thing with Grok, where you can actually change what is being fed into your model, and that can change the output. What we're trying to paint here is an understanding of trust but verify, trying as much as possible to understand. Now, this is the scary one. It's still, as of now, not used in the medical space. It's the artificial general intelligence. That's what you see in, if you have huge fans of sci-fi, and it will actually have human-like intelligence. Yes.

There's a lot of work that's been done in that space. As of now, based off yesterday, I have not seen anything in the healthcare space that is doing that. That's like your Ultron-level stuff. Thankfully, not there yet, right? And so we're trying to give you the tools to get on top of that before Ultron becomes a real thing.

Yeah. So again, based off the statistics in this room here, it basically mirrors what we see in here of a lot of people, and this statistic keeps evolving every single day. There was an article yesterday in NBC about open evidence and the 83% percent of people using it. This is evolving. What this is showing for us is a lot of people are using it. A lot of people are skeptical about the use of AI. How are we being intentional with the use of AI? I think the major takeaway for me as a specialty is, as of 2025, 12% actually, which is relatively low, felt that AI was going to replace our specialty.

And I think what that speaks to is the uniqueness of what we do and the relationship that an AI model cannot mirror at this point. So what we want us to do now is start thinking within your group, think about in your healthcare space, at home, what you're using from an AI standpoint. How is that working, right? What are you skeptical about? What are you curious about? And then if you have questions for us, also, just thinking about the concept of AI, hype versus reality, what are the things that you guys are interested in exploring? We're going to talk.

Yeah. Yeah, yeah. So we'll give you five minutes, and then we'll come back for the next episode.

So before we get into episode two, you know, our discussion prompt was asking about a tool that you've already seen in your practice through AI, kind of what your reaction to it was. And so I'm, I'm wondering if anyone would like to share who was curious the first time they came across AI in their practice?

Any folks took the curiosity approach? I saw a hand over here. Okay. Anyone willing to share? Yes, sir.

Sorry. It's been a bit of a disappointment for somebody who's a, you know, late-stage, practitioner. I saw it when it came out and had all kinds of high hopes, and it kind of crushed a lot of that. So now with AI, it seems like at least it's making some forward progress.

Yeah. So definitely, I mean, I've been right there with you when you're kind of perpetually drowning in notes, and not because you're a poor documentarian or a bad doc. It's because, I'm sorry, I actually talking to people and connecting with them, right? You kind of get in that spot, you're like, "Anything that'll help." So definitely understand that. Any skeptics first time you came across? I know we had some self-identified skeptics earlier.

So I think my biggest, well, two concerns when I was first hearing about AI scribes coming to our practice is that I already had a person scribe, who was disabled, working from home. This was a job that she could do, and I knew that I was going to have to replace her with a technology tool, which really sucked. So I put that off as long as I possibly could. So that was my first hint of skepticism. I now have an AI scribe, and I will say that it is better than the person scribe, unfortunately.

The second thing I was skeptical about is just how much energy it uses and how we can be responsible stewards of our impact on the environment as we think about using this tool in our practices. I do, again, use it regularly, but skeptic.

Yeah. And then that's kind of the, the icky part of it, right? Both kind of that human cost of jobs that'll be lost, because I know yesterday, I think it was when Dr. Green had mentioned, "Oh, use it for operations." And I'm like, oh my goodness, we are going to have so many fewer PSRs probably in a couple years. But, you know, it is super efficient, but then also that environmental cost.

My hometown actually in the low country of South Carolina, has recently been involved in one of those battles of a data center wanting to come in and just take all this beautiful marshland we got and just put a data center on top of it, and so far we've been winning that fight. But definitely. It reminds me when I used to work at Waffle House, we'd be like, "We're tired, but the money's so good." We were making a lot of great tips in Charleston, but.

I went to the AMA Wellbeing Conference in Boston in the fall, and one of the keynote speakers talked about rather than it being, because you had mentioned helping underserved populations, that it would create a bigger separation where basically rich people will get the people and underserved would get the AI.

Yeah. And I never thought about that. Me neither. Yeah. And I was like, "I don't like that." Yeah. Although some of it they did talk about, where having, on the discharge from a hospital, a nurse doesn't have the ability to spend 60 minutes explaining, can you use it in that sense?

But then again, is it taking a job? Is it more efficient than the human? And that kind of made me nervous.

No, it's tricky and that's actually a really good thought as we get into the next episode because we're going to talk about can we control the AI, right? Because even that bit you mentioned about, okay, well, could this end up in the future where your underserved systems and clinics have the AI workforce that is prone to hallucinations and other wonky stuff, and then all the high-performing, high-paid humans are at another spot.

But that's also something I would argue, and not to dovetail this into another conversation, right? But, that's kind of where we are with APPs Right? Like, I think sometimes, we are just flooded with APPs in rural areas and, "Well, you know, we don't get docs out here, and they're cheaper."

But it's like they're not the same, right? So we've kind of seen that fight before. So interesting thing to think about as we kind of approach, what do we do with this kind of next wave of a solution to fix the physician or just employee shortage.

So one of the things that has been striking, so I'm an early adopter of technology. I love technology. I've been using my AI for the last three to four years. Actually made a song, Michelle can testify to my music, of which we probably might be able to do if we have enough time, with AI. But over the last couple of months, I've actually been thinking about the ethical considerations of AI.

Thank you for bringing that up. Thank you for being a voice for that. AI is not going to go away. I think as leaders, bringing that ethical lens into AI is going to be a huge piece of what we do next. Want to explore this. This is going to be very theoretical, big concepts, right? There is this whole concept of perceived accuracy.

So again, I work in the hospital, and we rolled out this AI tool that they sold to my organization that could do magical things. But one of the things we found out overall was adoption and accuracy are two very different things, right? You know, from a statistical standpoint, they talked about that area under the curve, 80% sepsis.

But when I'm on call and I get four, five, six alerts, and then I have to dig into the chart, and I realize that patient is tachycardic because they're in pain. Why am I doing this, right? So always thinking about adoption and what’s your vendor selling to you, and how has it been used in other organization? What does adoption look like?

Yeah, and I think too, you know, I'm always big on kind of thinking of what we've already done to show how we can handle in the future because it's kind of the same game we play with drug reps, right? Like, they come in and whatever they're pushing that day is like, "Oh, this is the greatest thing I've ever seen."

Like so many people lost weight, so many people had their pain fixed, whatever. But we always have to ask ourselves, or even when it's not, you know, something with that corporate interest, but it's just like a new research thing, right? I feel like especially us boots on the ground primary care is like, "But how is this clinically relevant? Is it useful? Is this something that's actually going to make a difference in the day besides just give this person another check?"

So if you look at your show notes, and I just want you to circle this. This is a principle that is something I want everybody to just have at the back of it, their mind when they're exploring AI. It's the four principles for evaluating AI in medicine.

We talked about, and there's been a lot of conversation around the human-centered augmentation, the human in the loop of however you're using AI, who is making that final decision, the whole concept of AI drafts. AI will make that suggestion. As physicians, the decision lies with us. Accountability is a huge piece of this.

If you've been following the news, there have been a couple of legal cases around the use of ambient AI in health systems. There is one in California currently ongoing now. The next thing is workflow velocity. Big word is, if I am using an AI tool, does it actually improve my efficiency, right? If it doesn't, if it's adding extra clicks, extra time, extra chat review, is it really worth it?

Can you explain the logic? Again, it's not about getting into the weeds of, oh, I need to know the why. You might if you're a computer science guru, but can somebody just explain to me what's the reasoning behind this? Mago had talked about if you have an AI that's doing chat summarization for you, and it's pulling in this information from the EHR, can I get a hyperlink that gets me back to that source of truth of why are you saying we had an AI in our organization that basically saw ALF, assisted living facility, and suggested acute leukemia Not made up, right?

You're saying rule out AFib, and it gets pulled in as atrial fibrillation, right? So again, it's, what's the logic? What's the source of truth behind the AI? Equity and fairness is a big thing, some models you have to ask the question, what were they trained on, right? What's their source of truth? Where did they get their data from?

Again, I don't have any beef with anybody, but they went on Twitter. Now think about the interactions you have on Twitter, right? And then you train off that. Is that the best of humanity? I'll leave that to people. Okay. But thinking about what the bias built into the model, what populations did they use?

Where did they get, is this thing transferable from an academic institution down to FHQC in South Carolina? What does that look like?

Yeah. And raise your hand if you've worked with medical students. I think a big thing, you know, when we looked at these principles, and this is just about everybody in this room, and like I was saying earlier, you think of a large language model, and we functionally were large language models as med students.

But really, you should be evaluating these AIs and treat this AI just as you would if you had just a new student rotating with you, right? Like, nobody in this room would just take whatever the student said as the gospel truth. We'd be like, "Eh, where'd you get that from?" You know, unless it sounded very obvious.

And so when you think through these four principles, again, it's stuff that we've done before. You think of the human-centered augmentation. We decide even the best med student I've worked with is like, "I'm still going to make sure we're going to talk about this plan, and then I'm going to go in the room, and we're going to talk about this and get it through."

Workflow velocity, right? We know the student that we're probably going to give high marks is the one that you didn't feel like you had a student with you. Maybe your day was moving even smoother. Whereas the other one who I just had to always check behind him, and my day took twice as long. You're not going to evaluate that as high.

The explainability and the auditability principle, again, when a student gives you something, you say, "Well, how'd you get that answer?" Especially if it's something you never knew. And you're impressed when they say, "Oh, well, I was reading AFP Journal last night, and this is what I noticed was a new treatment of arthritis," versus, "I don't know. My last clerkship told me that." And they, you know, then again, but also with the equity and fairness, because even now is the way we evaluate medical students. Like, you know, we want to have someone that we feel like this, the patients that we put them on opposed to the patients we saw 100% by ourselves that day didn't have a different outcome just because the student was there.

Or you will have a lot of heavy concerns if your student kind of walked in the room and started saying a lot of that patient-shaming language of like, "Oh, we got a fat diabetic in room two." You're like: Whoa. Stop right there. What are you talking about, right? And so again, these are things we've done and I think it's helpful to think of in that context of having an intelligent being work with us that we also know that we have to look after and control in a sense.

You know, with the med students, and that's not too dissimilar from how we should be using these AI tools. So, go ahead, sir.

Yeah, could you cover the override authority a little bit more? Are they developing systems that'll put in the order and you can't change it? So, I can, I can briefly speak to that.

So one of the EH- EMR, EHRs, right? What they're doing now, I saw a demo of this last month, is they have an ambient listening tool, and then I tell my patient during the visit, "Oh, I'm going to get a chest X-ray. I'm starting you on lisinopril. We will change your statin from a low intensity to Crestor, and then I want you to follow up in two weeks."

Basically, the tool now pens your orders. Doesn't sign it. It will pen your order, and then you can go in and verify and make sure that those orders are the appropriate thing. Does that answer your question? Yeah. So again, they're building these models where it's scary fast. It's evolving really, really fast.

It's an understanding of, as the physician, the final decision lies with me. Same thing with your ambient documentation. The final decision of what is in the notes is not AI. It is your medical license that's on the line. So thinking about, yes, AI will write this wonderful note for you, the final decision of reviewing the note lies with you.

Any thoughts, questions Next thing, this is all big. If you're working in a small organization where you have a lot of influencing decisions being made around AI tool, or you have the CMI of your organization thinking through low stakes, right? What are the things that you think about how much autonomy do I want to give my AI versus don't touch it?

You know, when you think about documentation, scheduling, prior auth, in-basket triage, patient logistics, where at the end of the day really doesn't impact clinical care, it's not making decisions. Decisions still come up to you as a physician. You can always low stakes back. High stakes, when we start getting AI into the realm of clinical decision-making tools, which are out there.

They're not just validated enough. And if you have a good salesperson, take it with a pinch of salt of what's the data behind that. Go through those four principles we talked about of are you making decisions for me? What will that look like? And that last point you see on the slide about the closer AI gets to those high stakes type applications, you got to tighten that human review, right?

So in future episodes, we're going to talk about kind of how do we put this into practice in our own organizations. But that's the thing to remember too, is your organization ready for that capability? So in that sense, it's getting your great-grandmother the iPhone 72 SE or whatever is like she's not going to take advantage of all those things.

You just spend a whole lot of money for nothing, right? And so again, it's the same as we think of the hype and this control factor. Let's not get too far ahead of ourselves and get us this like souped-up thing that we were never really capable of truly governing in the first place.

So again, you have all this in your slides, just giving you a sense of what's low, fair, medium, and then the high risk.

Huge rule, as AI approaches diagnosis or treatment decisions, make sure that the human in the loop is very big. And also from an organizational standpoint, think about what your AI governance looks like.

All right. And so that ends episode two. And so we're about to give you all five minutes to kind of go over this next discussion prompt in your group, which is we want you to think of and name a high-stakes decision in your typical week.

And now whatever that decision could be, and it's probably different for everyone, what would it take for you to trust an AI's recommendation in that moment? Again, not the AI making the decision. You can't do anything about it. We're going to have that clear override authority, but just to trust an AI recommendation. And what would make it a hard no for you to trust that recommendation?

Anybody willing to share thoughts, conversations? Do I just walk up to any table? It's your show, man. I work in residency with students and residents, and we were having a discussion at our table about how you address the use of AI with, in general, younger folks who are much more skilled than I am, ahead of the curve with using the technology, but still nascent in their clinical decision-making skills, so they don't have those frameworks for validating what they're getting out of AI. How do you teach that?

Yeah. Well, that actually came up at the table that I kind of—Should we just jump in? Yeah, let's do the next episode.

Okay, I'll just talk about this. So this is a concept, it's called de-skilling.

D-E-skill. And there are two ways you can look at it. It's, as doctors making a differential diagnosis, I'll tell my wife, who is a non-physician, of, she tells me, "Oh, your daughter has a fever." And I'm like, "Eh, she's okay." And then she looks at me and she's like, "Stop telling me she's okay. Tell me why." And I'm like, "Ah, jeez."

I've gone through that processing in my head. I know what a sick kid looks like. I know what a sick kid doesn't look like. She's running around. She's not puking. Eh, she's fine, right? That is a skill that over the years we've practiced, we've honed down. Unfortunately, new clinicians, new residents don't have that skill yet.

And what we're seeing is a lot of them are just using AI as their go-to. They stop thinking, and they basically type in the patient's clinical scenario. I've seen this among my residents, and it pops out an answer. So it goes back to your AI governance. Now, you raised something that was really fascinating, and it is true.

The kids know about AI. It's part of their existence. You cannot take it away, right? So it goes back to AI governance. In my residency program, we don't allow, that we know of, AI use in the first two years, right? And then, as attendants, we've also been teaching ourselves how to go beyond what, how we learnt medicine, of being, probing, asking point questions, of, "Okay, let's take a pause.”

Yes, you've given me that very broad differential and that very specific treatment plan. Let's reframe. What if that patient came in with XYZ? How would you address that? So what you are trying to do is you're trying to think about their on-the-spot thinking process. The flip side of that is, there was a study that came out among GI physicians that were using AI for adenoma detection, and they actually saw that among those guys that were using AI, they had a higher detection rate.

When they stopped using AI, their adenoma detection rate actually dropped So it starts begging the question of it's not just happening among the new learners, it's also creating a sense of dependence on AI, on our skill sets, and how we're making diagnosis. It's a food for thought. I don't have answers, but it is something that you should, as you're using AI, as you're thinking about AI, it is something that you should keep exploring and pause and ask yourself the question, "Do I need to use this at this point?"

Yeah, and this is just spitballing here, right? But I think when we first started talking about AI use in our community health center, one thought that popped in my head, and I'm a pretty blunt guy, so I'm just, yeah, I was like, "We're about to find out who the dummies are." Because I feel like with, with AI, you can kind of tell when they've, if everyone's using the same thing, everyone starts sounding the same.

You know what I mean? Hmm. and so outside of clinical practice, there's a nonprofit that I work with, and one of the guys that's in the nonprofit with me, we've known each other since high school. I know this guy. We grew up together. And I noticed over the past couple months, his emails are a lot longer, using vocabulary he doesn't normally use.

There's emojis just popped in, and I'm like, "AI." Yeah. Uh-huh. Right? I haven't had the heart to call him out because they're beautiful emails. I'm like, "This is actually a net positive," right? As, you know, from his leadership position. But, it's just like, “I think you're using AI, bro." And I think the same in the different ways that it shows up.

I know I always pride myself on writing good notes. My first time using DAX Copilot wrote a good note much longer than I had learned to do as a doc that was just kind of typing my own, because I remember telling people, like, "I ain't wrote a note this detailed and thick since med school," right?

And so if all of a sudden, if I, if you see somebody is, like, kind of just having these really weirdly long notes for somebody that gets the work done on time, that's another tell, you know, that's going on, right? And I think also when you put people on the spot, like there's, one of the APPs that work under me. I love her because she's just awfully honest about it. We're talking over a case. Like, "What made you do that?" She's like, "I just wanted to open evidence." I'm like, "Oh, okay." Yeah. You know? At least you told me, right? But if she was trying to hide it, and then I'm really kind of probing, like, "So what made you do that?"

Boom, boom, boom. And eventually we'll get down to, "You don't really know why you did this, right?" And so I think there will be a way, as long as we're creative about it, that we can kind of find those tells. And then this is more just me coming from a military family sticking out.

Maybe it just means in training we have more, kind of survival drills. You know? Like, sometimes I think when the power goes out and we gotta do paper charts instead of the EMR, everyone is freaking out. Mm-hmm. And thankfully, because I work at a free clinic where, I'm the only one that actually still uses the computer, I know how to do paper charts because I have to, like, work with two paper charting docs every other week.

And so, I do fine in those environments or through mission trips I've done paper charts, so I'm trained on it. But I think that, I don't know. That might be something too, is like maybe I'll just have a little boot camp week is like no EMR, no AI. Let's just do a residency-sponsored mission trip and see how everybody does without that stuff, you know?

So just, I don't know, something to think of there, but very good questions. We'll get into the meat of this episode on how we can use AI. So as we've been talking about a lot, right, AI shows up in different ways in our practice. We've been talking at length about it being in documentation, so everybody knows that's obviously the best use case right now.

We've also talked a bit about imaging and kind of detecting different types of pathology and so, you know, pretty low risk, right? Because there's probably still that human control over it. The system brings it out, then a radiologist can read over it and confirm. And so that's another thing that we can kind of deploy confidently.

And then, we've talked a bit about open evidence as well, right? Which kind of gets into that literature synthesis point, which is really kind of picking up. I signed up for open evidence last week, like a day or two before that article came out, so I was like, "Oh man, Washington Post knows about me." But it's something that's really popping up. They're super high satisfaction, really cut down on my up-to-date time now, and it's ready to go. But then we have these things that are a little bit kind of just out there a bit more. We think of the admin stuff, which is not really something that all of us think on a day-to-day because at most we probably have chief medical officers in here, but maybe not necessarily any COOs or anything.

But really when you think about it, it's non-clinical. Might be a really high ROI and something that we can kind of just get piloting now. Like I said, I think that really is going to be the next hurdle for us in my health center is using it for admin and things like that. And then Sol has already talked about that risk prediction, right?

Very high accuracy but low adoption because we all get that fatigue. Think also, in risk adjustment, like when we think of everyone that's got Epic like me probably has that little pop-up like, "Do you want to add this?" But yes, question. I just have a question about that, like, sepsis predictor. Yep. I heard that that was incredibly bad Dr. Adegoke It is accurate based off, again, it goes back to the build, right? So, it's what is your model meant to do, right? So, sepsis is a metric. It's basically CMS telling us, does your patient meet tachycardia? Does it meet tachypnea? What's their oxygen? Do you have end organ? Are they on antibiotics? Is it lactate? That is the accuracy of the sepsis. From a clinical standpoint, that's where most physicians actually struggle. The numbers from sepsis, again, I just got back from Epic conference. The numbers based off those metrics are very, very, very accurate.

So I—Go ahead. So this, I was reading the book, I just told you guys, A Giant Leap. Okay, which is all about AI in healthcare, and they were talking about that Epic sepsis prediction model. Yeah ... as like, "This is an AI failure." It created way too many alarms. It created too much alarm fatigue. Mm-hmm. It doesn't know if a patient's in AFib with RVR, and thus they have a high lactate, and thus they have a high heart rate, and thus they're hypotensive.

Yeah. And, and so we would know automatically, looking at that patient's chart, that that is not a patient in sepsis. We don't need an alert for that. We know. We're smart. It doesn't know that. It just tells you this patient might have sepsis, and now you've got an alarm for it. I think we're saying the same thing.

Yeah, again, it's AI, it’s not magic, right? That's basically what I tell people, and that's why if you look at that line there, it's the accuracy is talking more about a speed ball. I think probably sensitivity. Sensitivity, specificity, exactly, right. We're talking about predictive. Yeah, agreed. Yes. Right? But when you think about adoption, it gets really annoying because, again, it pops up on, we're just adopting this. It's the bane of my existence, and I get email every single day because my clinicians are like, "This is just annoying," and it's a huge burnout risk for me because I have to constantly click in and yeah. And if I could get on my physician leadership empowerment spiel right quick, because we have to ask ourselves, who's building these things, right? And I think that was kind of like the unfulfilled promise of EMRs. When I was in medical school, I got to be a credentialed trainer for an Epic go-live at MUSC down in Charleston, and there was such a disconnect.

In theory, it did cool stuff, but it wasn't really built by anybody that actually did the job. And so, and I find a lot of EMRs, that's where they fail. It's coders that mean well. It might be coders that spent a lot of time with physicians, but there's not a lot of us there to tell people, like, "We actually don't care about that, like, we actually want this, right?

And so I think again, kind of just a lot of really AI is new, but it's a lot of the same battles we've been fighting before, right? And so, I think as we're evaluating these different things and these different applications, we have to ask ourselves who built it, right? And then not only who can control it, but who can give that feedback to make it work a bit better.

Because I think that is one thing about AI. I know for me using ChatGPT, Claude, some of the different more like kind of personal use ones, you could kind of teach it to be what you, what you want it to be, right? Like, hey, ChatGPT knows that I speak in sports analogies, so it's always giving me the ESPN version of things.

But you know, definitely I think that's also the biggest thing. And not just who controls it, but who's at the driver's seat in the first place. But very, very thoughtful comments there. All right, but let's dive a little bit into Ambient Scribe since that's kind of the best use case we’ve got. Kind of gets us around some of those icky points that we've already talked about and really discuss, like, why are they so successful, right?

We already talked about here in this group, probably one thing that all of us in this room have probably struggled with is the burden of documentation. You have 15 minutes to see this patient and write a note that's going to get all the things approved and answer all the patient's questions, and then you have to do that, like, times 25 for the day.

Fun, fun, fun, right? And so with Ambient Scribe, it's a need that's clearly there, right? From a business perspective, you’ve got a need that a market can use, and then it integrates seamlessly. Some of these things sit right on top, whether you're using, I keep wanting to say DAX, because that's what we use and—We use Abridge. Yes, Abridge. That's the other one. I know that was the other option we had. Epic has its own coming out now. Yep. Yep. Right. And so it integrates. There's a human in the loop, all of these things. It's not like you write your note and then you just sign off on it. It's asking you to kind of look it over.

And I wish my providers would definitely read over it a bit more, but it's okay. We're not here for me to vent. And then we get that immediate feedback. You can verify the accuracy, right? Like one thing I've noticed is when a patient's in a room with a spouse and the spouse is talking a little bit too much about their case, I'm like, "Ma'am, you're going to be next. Please stop." The AI will kind of pick it up and throw Mrs. information right into Mr.'s chart. And then, you know, it's pretty low risk because you can catch the risk before the patient sees the note. And then you don't have to convince people to do it. I remember I had one of my providers, she was a skeptic about it, and she, she's a skeptic on everything. I love her. She's very appropriately cautious. And she didn't know about using the DAX. I let her test drive the DAX one time. She's like, "Oh my gosh, you changed my life." I was like, "I know. Now please keep using it." Like, you don't have to be doing all this pajama time. And so Ambient Scribes definitely are probably one of our best, most successful things.

But I'm curious, since we have a lot of users of the things in the crowd, has anyone seen it fall apart? Yes, ma'am. I'm actually curious because we have the Abridge, and they're making us use it for all primary care, but they'll take it away our license if we don't use it. Yeah. But I've found that I'm actually faster if I don't use it. Oh. I finish my note after I see, because I type while I'm talking to the patient, and when they leave, I'm done. Yes. Especially if a patient I know. Yeah. When I start doing Scribe, I have to read everything. They add things that were not true. So it, actually I have to do pajama time if I use it.

Mm. So it's useful for me for new patients and transfer of care because I don't know those patients well. So is there a way, and I think they have a pilot actually now going on in our system, for them to input what I already know, because these are patients I know, that I don't have to start every visit, I have to say everything. This is a patient I already have that in my previous notes. Is there a way to make AI do that stuff so I can just do what I'm doing faster?

Yeah. I think that's where it's headed. Like, right now we're at a point to where it’s not helping right now. Yeah. Any thoughts in the house? I just want to acknowledge that frustration. Anybody has shared thoughts. Yeah, no, I agree that it, when I first used it, I thought, "This is so helpful. Okay, I close my note and we're done for the day." And then six months later I go back and try to read this note before I see the patient for follow-up, and I have no idea what actually happened because this is not written in my voice.

Yeah. I'm just struggling to how to make it more useful. Okay. Two things. Again, I don't have any disclosures to make, but I use Abridge. What they're building now is they're building a carry-forward functionality where it will do chat review for you, will get an understanding of what's been going on with the patient, and then you don't have to necessarily reinvent the wheel.

That's one. Number two is the voice thing, right? Back to what Lance had talked about of you read emails now, and I know an AI-written email. I'm like, "Oh, God." Just even unhighlight that line. I know that's AI, right? In Abridge, again, specifically, that's what I use, you can actually build your own prompts of how you want your note to sound, right?

So you think about prompt languaging of this is who I am, this is how I write. So specifically for my assessment and plan, I tell it, "When you're writing my assessment and plan, I want a bullet point, I want a summary statement, I want labs, vitals, and medications that will make a plan. I want an a differential diagnosis," because that's how I write. And then I actually created two copies of my prior non-AI notes, loaded it into the prompt, again, giving it context. And so now when my AI writes, it actually sounds more like me.

Would you take an old note and put it in or how do you do that? So again, this is very—this is where we're getting into the specificity of the tools, right? What I will encourage you to do is talk to your clinical informatics lead. Get back to your organization. Tell them that you went to this conference, and the two ask is, "Can I get where are we with our ambient listening? Can it go back in time and build a picture of the patient I'm going to see carry forward?"

That's number one. Number two, it's prompt engineering. The option, I know that in Abridge you can do prompt engineering there. I know somebody was nodding. Do you use Abridge And you do prompt engineering? Yeah. Do you want to talk about how you use it? So my work bridges across sort of different schemes, as I'm sure all of ours does. So when I'm doing an addiction medicine visit, I want that to sound very different than when I'm doing a diabetes visit. So my addiction medicine prompt says, "Include as many social details as possible." Those will get left out of a lot of the notes if you don't prompt it. And then I do, there's a functionality where you're supposed to be able to set pronouns, but I do a lot of gender-affirming care, and it often gets the pronouns wrong, and it does it with all of my staff members because they're break the glass. They can't see the pronouns for some reason with the Ambient function. So I often use it. I say, "This patient uses these pronouns. Please change the entire note," and it'll change the entire note. So there's, like, little, you can do small prompts just for that one case, or you can do preset prompts that you would use frequently.

And I think I said this already, but we're going to have our emails at the end of the slide deck, so y'all can hit us with those very specific questions. And as we're short on time, we're going to move along. We'll probably go through our, I mean, we're having great discussion now, so we'll just call that a discussion break.

But, a thing I want to say that was also a sentiment that popped up with one of the groups that I kind of had poked my head in on as we were talking earlier and what we're getting at here is, there's a lack of transparency with some of these AI things, right? Because the technology is cool, and we think of Ambient Scribe.

But I think it's okay to say that it's not going to help everybody, right? And so hearing your situation of, "This thing actually slows me down," then I'm not going to be pushing for you to do it. But if you're giving me that feedback saying, "But it actually helps for new patient visits," then, you know, then I think that's really trying to get your leadership to say, "Hey, can we make a business case to keep her license active because it really helps her with the new patients?"

And when they want to be like, "No, you have to do a certain amount of notes," it's like, can you just look at how much money we may have saved or generated overall because of the—what it's done for the whole team, and does that pay for her to keep that license, even if she only uses it for new patients, right?

And the table I was sitting at, we were talking about how there's already this lack of transparency even before we add AI into it sometimes from our leadership of like, "Well, how did y'all figure out my proposed patients per hour?" Or, "How did you figure out this productivity bonus stuff," right? And so definitely always encourage people, question your leaders. If all 20-something of my folks hit me up with a text right now of like, "These are the problems I asked for," right? And I think everyone that's in a director or above position, that's what they've asked for. And if your leadership doesn't want you to do that, then Carolina Health Centers is always hiring.

When you get back to your organizations, you should have an IT department. You should have a clinical informatics department. Talk to them, ask them questions, specific questions about what are the functionalities that I have. Think about, I find prompting very interesting, and I wrote this down because I knew I was going to forget, is just a broad concept around if you want to prompt anything, how I do it is I think about the task I want it to do.

I think about what's the context of the prompt. I also look for references of the more specific you are with your AI, the better output you get. So task, context, references, evaluate. Usually, your first output is not the best. Go back in and change your prompt. But always think about what's the task.

So when I'm writing my discharge summary using AI, I basically tell it I'm a hospitalist working in a level one trauma center. These are the things I want. The context is provided, and then I give it a lot of my prior notes just to use as a reference point.

So moving into episode four, we've talked a lot about how we're using it now, but I want us to get a bit creative and think of like, well, how can we use this thing going forward in the future, right? And we talked about that gap earlier that we have in practice because the solutions are out there, but they're not really hitting everything that we need right now, right?

We want the privacy. We always want to protect our patients' privacy. We want it to integrate with our EMR. We don't need another layer of things on top of stuff to slow us down, and we want to have those feedback loops. But then when you really speak to a lot of what the vendors might be coming, they're just like, "Ah, cutting edge. Ah, capabilities. Ah, this and that. Boom, Silicon Valley." And you're like, "I don't care," right? And so because of that, even if you're not in, like, an official administrative leadership position within your organization, right, I always say if, as soon as you decided to be, and even to my APPs, you decide to be a provider, congrats, you're a leader now.

Like, you're the leader of that clinical team. So we're all leaders, right? And so we need to bridge that gap by making sure that communication gets there, right? Like saying, "Hey, my colleague," even if, you know, even if you're not in a position of leadership to say, "Hey, my colleague feels like this thing is leaving them behind. How can we go with that," right? So that's why it's good to have that skepticism. It's good to ask those questions. And then when you are leading the implementation for your AI things, right, we have to think with the design. Again, you don't want to give a solution without a problem, right? And so really think of your organization, think of the problems that you have. If you're an organization that doesn't use Ambient Scribe, but everyone goes home on time and all the notes are done, congrats.

You don't have to give DAX Copilot or Abridge any money, right? But maybe you might need something that can do better with same-day scheduling or things like that. And then, like I mentioned earlier, when you think of that design, really be cognizant of what are the limits of your organization, right?

Like, I know that the Ambient Scribe has done very well for my full-time practice. We are not ready for that at the Free Medical Clinic of Newberry County because I need my two colleagues just to get on the EMR and off the paper. So, not the place for that project right? And then when you implement, make sure it's something that can go in seamlessly.

We all hate having extra clicks. We all don't have enough time. That is a well-studied fact. There's not enough time in the day for primary care. So make sure it's something that doesn't burden people, right? Make sure that you identify champions. Maybe you need to become the champion yourself and really measure what matters, right?

Like we could say, "Oh the cycle times dropped. But I think what we all would care about more is, like, the pajama time dropped, right? And so really measure what matters. And then have it fit into the context of your culture. Every technology that we use needs to fit within the context of our culture.

And I think when we look through a lot of different technologies over the years, every culture takes it on a bit differently, right? Robots are cool. Japanese cartoons did robots a little bit better than American cartoons, and that's why we love things like Voltron and all that, right? And so we just make sure that whatever you, however you take this technology, it fits into what name your organization here already does.

Make it normal to talk about errors and really invest in people not just using AI, but becoming literate in AI because then they can really call out the issues that are in it, right? And so when they see that the sepsis score just really isn't good, then they know, like, "Well, how do we train this?" You know, did you have the folks at Epic headquarters in Wisconsin do this or did you actually ask the IT department to do this?

And, we've also given you this checklist, and so I'll let you kind of go through this a little bit. So again, this is something I'm not going to bore anybody with, but if you hold any position of authority, if you have your own sphere of influence, this is more of a checklist for clinicians of when you are assessing your AI, thinking about feedback loop, validity of the population that was used for that.

What's the source logic? Is it efficient? Does it actually fit the whole concept of that whole person? What's the safety overdrive, right? I don't want something that will put in an order, sign that order without my overview there. Can I explain the logic and what's the bias testing behind that? So when your vendors come over and they're all around, they will come to your organizations, just be very thoughtful about what they're selling to you. AI hype is real. I am a lover of AI, but there is a lot of hype behind AI. It is good for some things, but it's not where a lot of the vendors are selling as of now. Yeah. And also because I love the power of saying no. I'm just going to highlight that last line that these little laser there. If they can't check seven of those eight those boxes, don't do it. Simple as that, right? Because in our clinical care, we are never going to compromise patient safety just because, I don't know, like they took us, I don't know, they brought the office a nice lunch or that person like is really cool or whatever.

We went to the same college, right? And so we just need to hold that line for this as well because like with everything else, no one's going to fight for our patients like we will, right? And, and I think that's a sacred honor that we have that we need to hold onto even as technology advances. So as we bring our fourth and final episode of this season of the ITFP is to an end.

Some key takeaways we want for y'all. Just remember that despite all the hype, AI is a lot narrower than you think. At most right now for what it's able to do, we are talking about a digital medical student. That's it. And I don't think any of us are worried that the med students or the interns are going to take our job anytime soon, right?

For now, right? But for now, that's not where it's at. It's narrower than you think. And that skepticism, keep doing it. It's wisdom. It's not resistance. And so, you know, ask those questions because a lot of times leadership needs to be taking a test. When we think of who leads our healthcare organizations, they're not necessarily people that have MD or DO behind their name, right?

And so again, we have to kind of sit there and be that barrier to protect our patients and our teams. And when we're adopting those things, remember it's about fitting more of the culture. What's the folks that already exist there as opposed to just doing it because technology is cool, and that human judgment is always going to have to remain central to what we're doing.

When you're out there, when you're advocating for it, whether you're being a healthy skeptic for it, make sure we're communicating what clinicians want. We don't want it just because the ACO we joined said that we can do this overlay, right? We want it because this actually helps my team. And as always, as we talked about through the different things, that implementation is, and how you go about it.

Again, lessons we learned from the EMR situation. There are places that paid a whole lot of money for the Epics and completely fumble the implementation, and nobody uses the Epic. Don't let that be with AI. The implementation beats accuracy in predicting that real world success. So that's all we got. So we'll just take it into a question and answer session. Questions, if anyone raised their hand. Appreciate your attention with all this. Thoughts.

Is there any available product right now for the actual inbox to help sort your inbox on this is what you need to do today. This will take you 30 seconds. This prior auth will take you five minutes. You know, who's leading? I think that's the next big thing. And who, which companies are leading that? I know Epic. That's what I know. They have the tool, and last year Epic was selling individual AI tools. Now it's a bundle. So they have in basket management. They have prior authorization. They have app for documentation. They actually have a patient-facing AI tool also that will help your AI summarize. So I know about Epic, as an organization, that's leading in that space.

I just thought of this when we were saying should we trust AI? because we do have clinical tools that I trust. It has the FIB-score, CHAD score, and all of that, and it pulls in the data. So now I'm thinking I just trust it because can I trust it?

I mean, I don't have to manually put all their, before these clinical tools became available, I have to put the HDL, but now it actually pulls all that for us. Sometimes the only things that I have to put in, and it would put a question mark, is male, female, because some people don't answer it, or African, non-African, and, and then it pulls. So, do I have to check if they're putting the right numbers?

Any thoughts? It depends on your level of confidence of your AI tool and also the data source at the back end, right? Yes, I have an AI tool that puts in CHADVAS score for me. I'm like, "Eh." But if you put in a number that doesn't make sense based off what I know of the patient, then that affects my trust level.

We bought a tool in the inpatient space that after using it for two weeks, I'm like, "I'm not going to use this again," because I just felt the Yes. Yeah, I would say that to draw from what you all said earlier in the presentation, think of it as a med student. How do we build trust ever? And the answer is that you interrogate it, and the more you interrogate it, the more you will learn whether it's accurate or not.

So you don't have to do it for every one, but for this tool that you're using in AI, interrogate where did this information come from? Is it accurate? And if you find out that it was accurate, you know, five out of five times, ten out of ten times, it's going to build your trust in the tool. So I love that analogy to thinking of it as a med student because you will do the same thing in interrogating it to decide.

Yeah. And this has been really great. We got time for one more question, and then I have to move this furniture back before I get fired.

My question is about the future of AI. What excites y'all about it most, and what keeps you up at night? That's a great ending question. I think the thing that excites me the most about it, and again, this is from being the guy that gets excited for tech, is I remember when I was in residency, we had one of our attendings. He actually had work experience as a computer coder, and he was just teaching us about EMR efficiencies. And he was saying that a proper EMR should not feel like a medieval suit of armor that you kind of have to go through and then do a champions competition, right? But it should feel like an Iron Man suit. Like you could fly with this thing, shoot lasers out of your hands. And I believe that if we kind of lean on the principles we've talked about in this session today, that we make sure that there are physicians, family physicians especially, at the decision-making tables, this is going to get us closer to that Iron Man suit. Because the Iron Man suit had Jarvis, which is an AI. Mm-hmm.

My what keeps me awake at night is AI literacy among family docs. When I started practice, I had one of my older physicians that typed like this and what he told me was when they made the switch from dictation to EHR, he was very resistant, and he never caught up with the time.

AI, being a skeptic, being curious, is going to be with us. It's going to change the world as it is. We need to keep up with the times. All right. Well, thank y'all so much. Thank you.