Criteria for Excellence | Chapter 8C: Artificial intelligence: Tools and challenges for residency programs and family medicine

• Creating notes from patient interactions: A growing number of companies have developed “virtual scribe” applications that create a SOAP (Subjective, Objective, Assessment and Plan) note from a transcript of the patient encounter. These applications effectively eliminate a clinician’s need to type, use templates or dictate to produce an accurate record of the clinical encounter.
• Patient education handouts: Several LLMs, including OpenAI’s ChatGPT and Google’s Gemini (formerly known as Bard), can produce patient education handouts on almost any clinical topic at different reading levels and in different languages.

• Diagnosis and treatment recommendations: Given sufficient context in a query, LLMs can produce a comprehensive differential diagnosis, including information about the likelihood of a particular diagnosis, and provide detailed treatment recommendations.
• Population health: AI can utilize its vast databases to create detailed maps and highlight population-level causes of disease that might otherwise go unnoticed.
• Risk prediction: AI applications can use data from a large number of patient care records to identify people at risk for deterioration and alert clinicians.

• Advice and triage: Visual and speech facsimile AI that can generate digital avatars that communicate with interactive natural language has rapidly evolved. This technology could be used to collect clinical information from patients and provide advice based on unique answers. In this sense, AI could eventually be viewed as an essential part of team-based care.
• Hospital at home: Wearable monitoring equipment can continually monitor patients at a distance and alert clinicians when the data suggest a possible clinical deterioration.
• Digital health coaching: Coupled with home digital assistants, AI can design and implement a sophisticated set of reminders and encouragements to help patients establish and maintain new health habits.

• Didactic education materials: Learners can use LLMs to generate patient education materials that provide clear explanations for common medical conditions and treatments.
• Curriculum development: Faculty members can use AI to provide guidance and generate drafts of curriculum, learning objectives and knowledge development plans.

• Virtual simulated patient consultations: LLMs can create realistic patient scenarios that allow learners to practice interviewing skills and clinical decision-making and communication skills in a virtual simulated environment.
• Differential diagnosis: Interactive exercises powered by LLMs can help learners develop clinical reasoning skills by inputting symptoms and receiving potential diagnoses.
• Exam preparation: LLMs can design practice questions and quizzes for learners so they can test their knowledge, identify knowledge gaps and receive feedback to enhance their exam preparation.

• Scholarly work: LLMs can quickly produce research proposals, including references and suggestions for what types of statistics would be most appropriate for the proposed study.
• Development of individualized learning plans: An LLM can produce a detailed ILP draft based on information about a particular resident’s educational goals, weaknesses and strengths.

• “Hallucinations:” LLM responses to queries may appear to be authoritative and logical. However, on close inspection, they may not be based on real, factual material. Also, because of the dynamic nature of LLMs, submitting the same query on different days may result in somewhat different responses.
• Bias: The huge database used to create the original base AI model (i.e., the contents of the internet) is inherently biased. Because internet content often reflects inherent biases, LLM queries can produce biased responses.
• Lack of transparency: Due to the complex architecture of the neural net, it is presently very difficult or impossible to track what data inputs an LLM actually used to create its response to a query. In early 2024, the Office of the National Coordinator for Health Information Technology published recommended standards​ for data transparency in LLMs.

• Copyright infringement: Training an LLM requires providing the AI program with templates of useful responses to queries, which are nearly always copyrighted material. AI developers are facing lawsuits from multiple sources for copyright infringement. It is unclear how the courts will resolve this issue.
• Plagiarism: It is easy to produce seemingly well-researched responses to a query using an LLM, but it is difficult to determine whether the content it produces is plagiarized. Educators in other fields are discovering that this can be a huge issue among learners. It is an extremely challenging problem because most content is derivative of previous work in some ways. There must be clear standards for citation and disclosure of AI-generated content and sources.
• Confidentiality breach: Collaborative databases like Epic’s Cosmos contain de-identified patient data from EHRs. However, as data sharing becomes more common, there are concerns that breaches of actual patient information will occur.

Atrophy of analytical skills: A basic tenet of learning analytical skills is that it takes repeated practice to improve competence. AI is such a powerful shortcut that it will be difficult to keep learners from using fast LLM queries rather than “slow learning” techniques.

• Inequitable access: Although some LLMs and AI applications are currently available to use at no charge, eventually development companies will need to recoup their investments and monetize access. This could put smaller entities, rural entities and entities that serve historically marginalized communities at a serious disadvantage and further exacerbate health inequities.
• Potential for harm: Nearly every transformative technology has been used to cause harm to humans. Although AI has the potential to revolutionize the practice of medicine, it could also be used to do things like create unique and deadly pathogens, generate misinformation that promotes fear of effective treatments or produce incentives to remove humans from the healing process. Also, even if companies attempt to use filters to remove the most objectionable LLM responses, humans with malicious intent will continue to discover ways to “jailbreak” these restraints to access potentially dangerous content.
• Replacement of primary care physicians: The first AI applications will likely be used as tools to unburden physicians from time-consuming tasks. For example, automated scribes can produce office notes, and AI programs can read and summarize ancillary paperwork. However, as AI algorithms are perfected and tested, much of routine primary care could become a candidate for automation. Additionally, AI algorithms could lead to more encroachment on primary care physicians’ scope of practice by family nurse practitioners and physician assistants, who already provide a quarter of all health care visits in the United States.²

Ultimately, family medicine educators will need to find good answers to the following key questions:

• “When any question can be immediately and reliably answered by a chatbot at the point of care, how will doctors become effective lifelong learners?”
• “What will health care look like when medicine is assisted by AI models that filter ethical decisions through opaque algorithms?”
• “What will it mean to practice as a physician when much of that cognitive work could be offloaded to AI?”

Develop and implement an AI curriculum and a faculty development plan to cover AI core competencies in the following domains^4-5:

• Foundational knowledge – Understand AI concepts and health care applications.
• Critical evaluation of AI tools – Assess the quality, accuracy, safety, bias and appropriateness of AI tools.
• Utilization of AI tools in patient care – Integrate AI tools into clinical workflows to provide patient-centered care.
• Workflow optimization with AI integration – Analyze and adapt workflows to smoothly implement AI tools.
• Patient communication – Communicate information about AI tools effectively in simulated and real-world settings.
• Consequences – Demonstrate awareness of how AI can impact justice, ethics and equity in health care.

Establish protocols regarding the transparency of AI sources and queries.

Given current issues regarding the accuracy of LLM responses and the potential for inherent bias, Programs of Excellence should ensure that all responses are curated by a knowledgeable human. Faculty members should confirm that AI-generated information is accurate before applying it to patient care or presenting it to learners as factual.

Create guardrails around activities to preserve acquisition of critical thinking and analytical skills.

Programs should determine a sufficient number and variety of educational experiences to be completed independent of AI. They should also require learners to gather original source data, analyze the data and present their analysis so that they develop mastery of these skills.

• Collaborate with other family medicine and primary care programs on AI curricula and responsible use and adoption of AI in practice.
  • It will be essential for Programs of Excellence to collaborate on the rapid development of best practices regarding this transformative technology. Collaboration should occur not only among family medicine programs but also across primary care disciplines at a local level in institutions that support multiple programs.
• Collaborate with their existing hospitals and sponsoring institutions to ensure that any use of AI aligns with existing policies.
  • If no current policies on AI use exist or local policies and implementation plans are in flux, a Program of Excellence should take a leadership role in developing and implementing AI policies.

Review and refocus curricula on skills that are unlikely to be supplanted by AI.

As LLM responses rapidly become more focused and reliable, much of the cognitive work done in routine primary care will likely be done by chatbots. As Programs of Excellence train residents now to work in a health care environment that will certainly be transformed by this technology in the future, they should ensure that graduates master skills that will not be replaced by AI.

These could include the following:

• Empathetic communication: Humans will always have an advantage over AI in providing personalized care, particularly care provided by an empathetic physician who has exceptional skills in active listening and coaching.
• Broad procedural skills and training: AI will never be able to “do” things in the real world. As robotic procedural assistance evolves and enhances procedural interventions, physicians with a wider range of procedural skills will continue to thrive in an AI environment.
• Complex care and problem-solving: Increasingly, subspecialists manage the care of patients with complex medical problems. This results in fragmented care for patients who have more than one complex medical problem. AI could work in favor of well-trained generalist physicians who can offer high-quality care and continuity for patients with complex medical needs. For example, a family physician could use AI to help personalize and refine a medication strategy for the treatment of congestive heart failure so that a cardiology referral is not necessary.
• Community health and health coaching: AI could greatly enhance family physicians’ ability to expand their scope beyond the office to address social determinants of health in their communities and support healthy behavior change in their patients’ homes. For example, a physician might use AI to help create an advocacy strategy for confronting a major community polluter who is contaminating the local environment and directly causing an increase in respiratory illness.
• Implementation science: The health care system will need generalist physicians to find creative uses for AI to help achieve the quintuple aim of enhancing the patient experience, improving population health, supporting the care team’s well-being, reducing costs and advancing health equity. Programs of Excellence should produce physicians who have the skills to lead this effort.