Clinical Practice Guideline Manual

Introduction

The American Academy of Family Physicians (AAFP) develops evidence-based clinical practice guidelines (CPGs), which serve as a framework for clinical decisions and supporting best practices. Clinical practice guidelines are statements that include recommendations intended to optimize patient care. They are informed by a systematic review of evidence, and an assessment of the benefits and harms of alternative care options. CPGs should follow a sound, transparent methodology to translate best evidence into clinical practice for improved patient outcomes. Additionally, evidence-based CPGs are a key aspect of a patient-centered medical home (PCMH).

This manual summarizes the processes used by the AAFP to produce high-quality, evidence-based guidelines. The AAFP’s development process adheres to the following standards and principles:

Clinical practice guidelines should be developed using rigorous evidence-based methodology with the strength of evidence for each guideline explicitly stated.

  • Clinical practice guidelines should be feasible, measurable, and achievable.
  • Clinical practice guidelines, from which quality performance measures will be developed, should be reviewed by representatives of the physicians they will impact.
  • Clinical performance measures may be developed from clinical practice guidelines and used in quality improvement initiatives. When these performance measures are incorporated into public reporting, accountability, or pay for performance programs, the strength of evidence and magnitude of benefit should be sufficient to justify the burden of implementation.
  • In the clinical setting, implementation of clinical practice guidelines should be prioritized to those that have the strongest supporting evidence, and the most impact on patient population morbidity and mortality.
  • Research should be conducted on how to effectively implement clinical practice guidelines, and the impact of their use as quality measures.
  • Clinical practice guidelines, from which quality performance measures have been developed, should be updated as new evidence is available, and the producers of the performance measures should be notified of the work in progress.

I. Development of Evidence-based Clinical Practice Guidelines (CPGs)

a. Definition: Clinical practice guidelines are state­ments that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options. Rather than dictat­ing a one-size-fits-all approach to patient care, clinical practice guidelines offer an evaluation of the quality of the relevant scientific literature, and an assessment of the likely benefits and harms of a particular treatment. This information enables health care clinicians to select the best care for a unique patient based on his or her preferences.

b. AAFP’s Commission on Health of the Public and Science (CHPS) and Board of Directors provides oversight for the development and approval of its clinical practice guidelines.

c. Principles for Development (IOM 1.1, CMSS-P 11, CMSS-C): The IOM identified eight standards for developing trustworthy guidelines. The standards reflect best practices across the entire guideline development process, including attention to:

  • Establishing transparency
  • Managing conflict of interest
  • Guideline development group composition
  • Clinical practice guideline–systematic review intersection
  • Establishing evidence foundations for and rating strength of recommendations
  • Articulation of recommendations
  • External review
  • Updating

II. Joint Development of CPGs with External Organizations

The AAFP advocates the development of explicit patient-centered clinical practice guidelines which focus on what should be done for patients rather than who should do it. When clinical practice guidelines address the issue of who should provide care, then recommendations for management, consultation, or referral should emphasize appropriate specific competencies rather than a clinician's specialty designation. The AAFP may participate with other medical organizations in the development of clinical practice guidelines (also known as practice parameters or clinical policies) when appropriate criteria are met.

 When AAFP enters into a joint development of a CPG with external organizations, a Memorandum of Understanding (MOU) should be developed to guide the process.

III.  Identification of a CPG Clinical Topic

a.    Subcommittee on Clinical Practice Guidelines (SCPG) of CHPS

i.    Criteria for topic selection and creation of clinical practice guideline
      1.    Relevance to family medicine
      2.    No current evidence-based guidelines on the topic available that are suitable for use by family physicians
      3.    Guidance on this topic will support AAFP Strategic Objectives and Strategies
      4.    A systematic evidence report is available, potential for nomination to Agency for Healthcare Research and Quality (AHRQ),
             or there is a funding source for creation of an evidence review
ii.    Needs assessment
iii.    Determine potential collaborators
iv.    Determine CPG publication venue

b.    AAFP Board approval for topic nomination and collaborators

c.    Contact potential co-nominators/collaborators to involve them in the process

IV.  Systematic Evidence Review of CPG Clinical Topic (IOM 4; CMSS-P 4.2.2, 5.6, 5.7 and 5.8)

The AAFP utilizes the Agency for Healthcare Research and Quality (AHRQ) for development of a systematic evidence report on the key questions identified for the CPG.

a.     Develop topic nomination proposal to AHRQ

        i.    Include key clinical questions and parameters
        ii.    Include collaborators for co-nomination (if applicable)

b.    AHRQ Evidence-based Practice Center (EPC)

         i.    Establish staff contact with EPC staff for evidence report
        ii.    Include one or more family physicians to serve on the technical expert panel (TEP) of the EPC
        iii.    SCPG and content experts help in forming/providing feedback on key questions

c.    AHRQ EPC Evidence Report on Clinical Topic

        i.    Provide review of the evidence report as requested by AHRQ throughout the process
        ii.    Final EPC evidence report published and available

V.  Development of CPG Panel (IOM 3; CMSS-P 4.3, 4.4, 4.5, 4.6, 4.7, & 4.8)

a.    Identify AAFP CPG Chair through the CHPS and approved by the Board of Directors

b.    Identify family physicians panel members including any CHPS representatives

c.    Identify CPG panel members from collaborators including a patient representative or patient advocacy group(s) when available/appropriate

d.    Conflict of Interest Form from all panel members (IOM 2.1 & 2.2)

VI.  Conflict of Interest (COI) Policy and Process (IOM 2.1, 2.2, 2.3, & 2.4; CMSS 3, 4.1, & 4.2.1)

a.    Prior to selection of the Guideline Development Group (GDG), individuals being considered for membership should declare all interests and activi­ties potentially resulting in COI with development group activity, by written disclosure to those con­vening the GDG.

  • Disclosure should reflect all current and planned commercial (including services from which a clinician derives a substantial propor­tion of income), non-commercial, intellectual, institutional, and patient/public activities pertinent to the potential scope of the CPG.

b.    Disclosure of COIs within GDG

  • All COI of each GDG member should be reported and discussed by the prospective develop­ment group prior to the onset of their work.
  • Each panel member should explain how their COI could influence the CPG development process or specific recommendations.
  • Each panel member will update any COI at each meeting of the GDG.
  • COI conflicts will be reviewed and resolved by the GDG chair and staff. The resolution may involve having the panel member recuse themselves from relevant discussion and/or voting.

c.    Divestment

  • Members of the GDG should divest themselves of financial investments they or their family members have, and not participate in marketing activities or advisory boards of enti­ties whose interests could be affected by CPG recommendations.

d.    Exclusions

  • Whenever possible GDG members should not have COI.
  • In some circumstances, a GDG may not be able to perform its work without members who have COIs, such as relevant clinical spe­cialists who receive a substantial portion of their incomes from services pertinent to the CPG.
  • Members with COIs should not represent a majority of the GDG.
  • The chair or co-chairs should not be a person(s) with COI.
  • Funders should have no role in CPG develop­ment.

VII.  Clinical Practice Guideline Panel Collaboration

a.    Clinical practice guideline time-line and expectations
AAFP staff members, in collaboration with the GDG chair, will develop a   time line for the guideline being developed. This time-line will be distributed to GDG members during the first meeting of the GDG. Though this time-line is developed with the goal of adherence, it is recognized that circumstances during the development process may affect the time-line. Thus, this is a living document throughout the guideline process and should be updated as appropriate.

  • Writing assignments: Writing assignments may be made throughout the guideline development process. GDG members will be asked to volunteer for certain tasks and may be assigned to subgroups to develop recommendations and write supporting evidence for those recommendations.
  • Deadlines: Clear deadlines will be agreed upon during the process of guideline development. However, as stated above, circumstances during the CPG development process may arise that warrant adjusting deadlines. The panel chair and staff members at the AAFP will work with the GDG on any changes in deadlines.

b.    CPG outline
The GDG will work with the chair and staff members to ensure that appropriate clinical questions are used for the guideline and an outline of what is to be included in the guideline is written.

c.    Conference calls
Conference calls will be convened at the start of the guideline development process and throughout as needed. AAFP staff members       will work with GDG members to ensure availability for calls. When a member cannot be present on a call, that member will be given a chance to provide any written comments prior to the call, and after the call in reaction to the meeting summary.

d.    Electronic communication
Electronic communication will be used throughout the guideline process. Reasonable response times are expected for electronic communications and dates will be included in the communications by AAFP staff members.

e.    CPG publication(s)
When AAFP collaborates with others on a joint guideline, it will be decided where publication is expected at the start of the collaboration. All parties will agree to the publication plan.

VIII.  Review of Published Evidence Report (IOM 4; CMSS-P 5.5, 5.6 & 5.7)

Section IV of this manual described the AAFP process for nominating topics to AHRQ for a systematic review of the evidence. Once the systematic review has been completed, the evidence report is published by AHRQ. The AAFP Commission on Health of the Public and Science examines the published AHRQ evidence report and determines if a clinical practice guideline is necessary. If so, AAFP forms a guideline development group or GDG in accordance to what is outlined in section V of this manual.

a.    Systematic evidence review performed by the AAFP
There are two different instances when a systematic evidence review may be performed by the AAFP. The first is after the publication of an AHRQ evidence review when the CHPS decides a CPG is necessary. The GDG and staff members at the AAFP will work with a librarian to perform an updated review of the literature. The librarian will use the same search criteria that were used in the AHRQ systematic review. AAFP staff     members review the updated search results and obtain articles relevant to the systematic review.

The second instance in which the AAFP performs a systematic review is to ascertain whether literature published subsequent to a clinical guideline or policy publication warrants an update to the guideline or policy.

As outlined in section IV, AHRQ has a process for performing systematic reviews that is consistent with the 2011 publication from the IOM about Standards for Systematic Reviews(www.iom.edu). The AAFP also uses this as a guide to ensure the systematic reviews we are performing are compliant with the best standards available at this time. These standards include: establishing a team with appropriate experience and expertise to do the review, including those with content expertise; providing methodological expertise and other expertise as appropriate; ensuring any conflict of interest is managed with regard to the team; ensuring that there is user and stakeholder input as the review is designed and conducted; managing conflict of interest with regard to any individuals providing input into the review; and formulating the topic for review.

The standards also discuss “finding and assessing individual studies.” This includes steps such as:

        i.      Conducting a comprehensive search for the evidence. This step will likely include:
                1.    Working with a librarian, and
                2.    Searching appropriate databases, citation indexes and other sources for relevant information.
        ii.      Taking action to address potential bias in reporting of research results.
        iii.      Screening and selecting relevant studies. Here it is very important to include and exclude studies according to specified criteria
                 developed in the protocol. It is recommended that two or more people screen studies and that these reviewers are tested for
                 accuracy and consistency in their reviews.
        iv.      Documenting the search strategy, including dates of searches and how each item identified in the search was addressed. If
                 excluded, include the reason for exclusion.
        v.      Managing data collection. It is recommended that developers:
                1.    Use two or more researchers to extract relevant data from a report;
                2.    Link publications from the same studies to avoid duplication of data; and  
                3.    Use data extraction forms that are pilot tested.
        vi.      Finally, guideline developers should critically appraise each study using the specified protocol and forms derived for the review.

IX.  Grading Evidence and Strength of Recommendation (CMSS-P 5.8; IOM SR standards)

Compiling evidence and assessing it for quality are important steps in a systematic review. The quality of the evidence should be tied directly to the recommendations made in a guideline, and the strength of the recommendations in that guideline. Consistent with the IOM standards for systematic reviews, the AAFP uses a specified framework for assessing the quality of studies and providing strength for each recommendation. The AAFP uses a modified Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach described below.

a.    GRADE methodology
The AAFP uses a modified version of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) method to systematically examine research in order to rate the quality of the evidence, and designate the strength of a recommendation based upon that evidence. The GRADE system provides a transparent process and framework for developing evidence-based recommendations.

 GRADE uses the following system to rate the quality of evidence:

        i.       High Quality (Level A): Further research is very unlikely to change our confidence in the estimate of effect.
        ii.      Moderate Quality (Level B): Further research is likely to have an important impact on our confidence in the estimate of effect, and
                may change the estimate.
        iii.     Low Quality (Level C): Further research is very likely to have an important impact on our confidence in the estimate of effect, and
                is likely to change the estimate.
        iv.      Very Low Quality (Level D): Any estimate of effect is very uncertain.

b.    Strength of Recommendation
GRADE uses the term “strength of recommendation” to rate the extent of confidence that the desirable effects of an intervention outweigh the undesirable effects.

        i.        Strong recommendation: Most informed patients would choose the option recommended, and clinicians can structure their
                  interactions with patients accordingly.
        ii.        Weak recommendation: Patient choices will vary based upon their values and preferences, and clinicians must help to ensure
                  that patient care stays true to these values and preferences.
        iii.       GRADE also allows for “good practice points.” These are recommendations that can be made when it is deemed they will be
                  helpful to the clinician, such as recommendations for shared decision making, but there is no direct evidence to support the
                  recommendation. These should be used sparingly in guidelines.

c.    Upgrading and downgrading evidence:
The GRADE system allows the evidence to be upgraded or downgraded based upon specific criteria.

        i.        Downgrading evidence: Evidence may be downgraded due to the following reasons:   
                1.    Risk of bias refers to factors that make it less likely that the answer found in the study may not represent the true answer in
                       the population. Faulty randomization, such as lack of concealment at allocation to the study group; lack of blinding to the
                       study group when assessing outcomes; large losses to follow-up; the failure to analyze everyone in the group to which they
                       were randomized; stopping the study early when the benefit seems too great to ignore; or failure to report all outcomes that
                       threaten the validity of the study.
                2.    Inconsistency of findings across a number of studies must be explained. Were the interventions really the same? Were the
                       samples very different? Inconsistencies that cannot be explained make it very difficult to assess the true effect of the
                       treatment.
                3.    Directness refers to the extent to which two interventions are being compared to each other in similar populations. Indirect
                       comparisons are more difficult to interpret. Two types of indirectness exist.
                        a)    The first includes indirect comparisons. For instance, if two drugs are being examined for an outcome, but there are no
                               studies that directly compare the drugs, which is an indirect comparison.
                        b)    The second includes differences in population, intervention, comparator, and/or outcome.
                4.    Imprecision refers to a study that may show statistically significant effects, but the sample is still small and the measure of
                       benefit is imprecise, meaning that it has a wide confidence interval.
                5.    Publication bias may also exist. Investigators are more likely to submit studies for publication when the results are positive
                       and journals may be more likely to accept them for publication. An effort should be made in a systematic review to uncover
                       studies that have not been published. This is a particular issue when the studies are funded by industry.
        ii.        Evidence may be upgraded based upon the following factors:
                1.     Large effect size: A large effect is much less likely to be spurious than a small effect. Small effect sizes can much more
                       easily result from bias in studies.
                2.     Dose response: This exists when there is evidence that differences in dosage result in different effects/outcomes. This is one
                       aspect of a finding that suggests an association is really cause and effect.
                3.    All plausible confounding: In observational trials it is particularly difficult to measure and control for all plausible confounding.
                       When all unmeasured plausible confounders and biases in an observational study would result in an underestimate of an
                       apparent treatment effect, then it is more likely that a finding is real rather than the result of unmeasured confounding. For
                       instance, if only sicker patients receive an experimental intervention or exposure, yet they still fare better, it is likely that the
                       actual intervention or exposure effect is even larger than the data suggest.

X.  Writing the Guideline

a.    Scope of the guideline (CMSS-P 5.1-5.4)
The AAFP includes the intent, rationale, and scope in all guidelines. This includes, but may not be limited to the appropriate users of the guideline, and situations in which the guideline should be used. This includes information about appropriate patient populations for the guideline. Key questions that were addressed in the systematic review and any relevant clinical questions related to the key questions are included.

b.    Moving from evidence to recommendations (IOM standards 5 and 6; CMSS-P section 6 and 7)

        i.          As noted in section IX, the AAFP uses a modified GRADE methodology for rating the quality of the evidence, and guiding the
                   strength of recommendations. It is worth noting that moving from examining the evidence to making a recommendation is where
                   much of the disagreement happens in guideline development. Different groups that develop guidelines may disagree on how
                   much weight they give to lower-level evidence; may not fully take into account benefits and harms, costs or burdens; and may
                   give differing emphasis on patient or provider preferences and values.  However, all of these factors should be considered when
                   making recommendations. AAFP’s use of the GRADE system helps to systematically examine many of the factors mentioned
                   to determine the quality of the evidence and strength of recommendations.
        ii.        The AAFP strives to only make strong recommendations based on high-level evidence. However, there are few instances where
                   strong recommendations can be made based on moderate or low-level evidence. In these instances, there must be certainty
                   that benefits outweigh harms.
        iii.        Recommendations made include an explanation of the reason for the recommendation; description of benefits and harms; a
                   summary of the relevant available evidence; any explanation of values and preferences that went into the recommendation; a
                   rating of the level of evidence and strength of recommendation; and differences in opinions of GDG panel members, if they exist,
                   for that recommendation.
        iv.        Recommendations made are specific and actionable and worded in a way that is clear that they are (1) strong
                   recommendations, (2) weak recommendations, or (3) good practice points.

c.    Panel assignments
With direction from the GDG chair, members of the GDG will be given writing assignments to complete during guideline development. When possible, GDG members will be asked for preferences regarding sections of the guideline they would like to write.

d.    Making the CPG implementable

        i.         For implementation, the recommendations should be specific and provide clear direction. The number of recommendations
                   should be kept to a minimum.
        ii.        Access to the guideline should be provided through publication in a journal, the AAFP website, and the guideline clearinghouse.
                   (CMSS 10.1-10.2)
        iii.        When available or appropriate, actions should be taken to incorporate the recommendations at point of care through electronic
                   health records (EHR) reminders or toolkit/checklist for physicians. (CMSS 10.3)
        iv.        Additional implementation methods include mass media campaigns (news article, leadership blog, other avenues as suggested
                   by the AAFP content strategy team—see dissemination section), and interactive educational meetings with quality
                   improvement resources as appropriate (expanded learning session at Family Medicine Experience [formerly Assembly],
                   workshops)

e.    Compilation of draft(s)
All drafts of the guideline should be sent (or made available) to the GDG chair and staff members at the AAFP. Most often, staff members at the AAFP will compile all sections of the draft guideline and the chair will review the draft(s) before it is sent to other members of the panel.

XI.  CPG Peer Review (the following sections are in accordance with IOM 7 and CMSS 8.1.1, 8.1.2, 8.2.1 and 8.2.3)

a.    CHPS and GDG
The first round of peer review of the CPG is sent to the AAFP CHPS, as it is the main body that oversees guideline development, and the GDG. All reviewers are given four weeks to complete and return their review form to the staff members at the AAFP. (see Appendix A for an example of the review form)

Upon receipt of the reviews, all comments will be recorded. Comments will be addressed when the chair determines that there is a need. A written record will be kept of the rationale for responding or not responding to all comments received.

b.    External reviewers (including collaborating organizations)
All relevant stakeholders are included in the external review, including collaborating organizations, and organizations that may be affected by the guideline. All reviewer comments are collected and recorded. A record of how the comment was addressed is kept. Reviewers’ names are kept confidential unless a reviewer wants to be recognized for his or her review.

The draft guideline will not routinely be made available for a period of public comment, but will be reviewed by key stakeholders including patient advocacy groups if a patient voice was unavailable for inclusion on the guideline panel.

XII.  AAFP Approval Process (section 8.1.3 and 8.1.4 of CMSS)

a.    CHPS via its Subcommittee on Clinical Practice Guidelines (SCPG)
The SCPG, a subcommittee of the CHPS that is responsible for guideline review and development, will review the final draft of the guideline. Upon approval, a recommendation is made to the full CHPS, which upon approval makes a recommendation to the AAFP Board of Directors for approval.

b.    Board of Directors
The AAFP Board of Directors reviews the guideline. Any questions from the Board are addressed by the GDG, and staff at the AAFP. Upon approval from the Board, the guideline is sent to collaborators for possible endorsement.

c.    Endorsement by collaborators
Collaborators on the guideline are given the chance to endorse the guideline before it is published. The collaborators will be sent the embargoed guideline, and given a month to decide upon endorsement.

XIII.  Publication (CMSS 8.2.2)

a.    Annals of Family Medicine
Upon completion, the AAFP submits its guidelines for publication in the Annals of Family Medicine. The guideline manuscript undergoes independent editorial review, and a decision is made about publication. Due to the nature of journals, all supporting materials (such as tables with quality ratings of studies) may not be able to be published. All supporting materials that are relevant to the guideline that are not published will be made available on the AAFP website.

b.    Copyright issues
Copyright issues are negotiated with the publication journal with appropriate licensing agreements made to the AAFP.

c.    AAFP website (CMSS 10.1)
After publication, the guideline is placed on the AAFP website for easy accessibility. Supporting documents that were not published with the original guideline will be available on the AAFP website as well.

XIV.  Dissemination (CMSS 10.2 and 10.3)

a.    Dissemination/marketing plan
The AAFP staff members work with the HOPS Content Strategy Team (see below) and the Marketing Division to disseminate the guideline upon publication.  Staff members also submit the published guideline to the National Guidelines Clearinghouse for dissemination. Any derivatives made relating to the guideline will also be publicized via a marketing plan.

b.    Health of the Public and Science (HOPS) Content Strategy Team
The HOPS Content Strategy Team is in charge of managing content for the HOPS Division. Staff members in charge of the guideline process will work with the team to ensure that all avenues for dissemination are utilized.

XV.  Five-Year Update of CPG (IOM 8.1, 8.2 and 8.3, and CMSS 9.1, 9.2, 9.2.1, 9.2.2, and 9.2.3)

a.    Determination by SCPG/CHPS
All guidelines developed by the AAFP are scheduled for a review five years after completion. However, literature pertaining to a guideline is monitored regularly, and if it is deemed necessary, a review can be initiated sooner. Whichever the case, when a guideline review is initiated, a preliminary search of the literature is completed and brought to the SCPG to determine if a new systematic review is necessary. If so, the topic will be nominated to AHRQ for a full systematic evidence update. If not, the SCPG will decide whether to reaffirm the guideline for another set time period not to exceed five years, or sunset the guideline.

XVI.  Endorsement of External Guidelines

a.    External guidelines may be reviewed for endorsement by the AAFP initiated by a request from another organization, or identified as having a high applicability to family medicine.

b.    The guideline will then be reviewed using set criteria (see Appendix A for review form) by members of the commission, or AAFP members at large with appropriate expertise and/or review experience. In certain cases, staff may review guidelines from selected organizations, and make a recommendation to SCPG and CHPS.

c.    The member reviewers will then submit comments and a recommendation to staff (see Appendix B for AAFP’s Endorsement Policy).

        i.        Endorse—the AAFP fully endorses the guideline
        ii.        Affirmation of Value to Family Physicians—the guideline does not meet the requirements for full endorsement, or the AAFP is
                   not able to endorse all the recommendations, but feel the guideline is of some benefit for family physicians
        iii.        Not Endorse—the AAFP does not endorse the guideline and the reasons are stated. The AAFP may also choose to remain
                   silent.

d.    Reviewer comments and recommendations will be collated and reviewed by staff and the chairs of CHPS and SCPG. If substantial differences occur, with chair approval, the reviewers will discuss and determine if a consensus can be reached.

e.    A recommendation will then be given to the SCPG for approval, and upon passing, a vote will be taken at the commission.

f.     A recommendation describing the commission’s action will be submitted to the Board or board chair for approval.

g.    Once approved, the organization will be notified by staff and a summary of the key recommendations, and a link to the full guideline will be posted on the AAFP website. Only guidelines with endorsement or affirmation of value will be placed on the website.

h.    External guidelines designated as endorsed, or affirmation of value will be reviewed every five years following their date of publication. Guidelines may be reviewed earlier if new evidence warrants an update. 

For information about the AAFP Guideline Endorsement Form, please contact Janet Leiker at (800) 906-6000 extension 3161, jleiker@aafp.org or Melanie Bird at (800) 906-6000 extension 3165, mbird@aafp.org.