Annals of Family Medicine

G-TRUST Instrument Identifies Useful Clinical Guidelines

October 30, 2017 03:03 pm Chris Crawford

Family physicians have thousands of practice guidelines from which to choose, and often, these recommendations conflict with one another.  

[Physician at computer]

Guidelines may be of poor quality, fail to answer the right questions or be affected by conflicts of interest.

That's why a team of researchers led by Allen Shaughnessy, Pharm.D., M.Med.Ed., professor of family medicine at Tufts University School of Medicine and director of the Family Medicine Master Teacher Fellowship at Tufts University Family Medicine Residency at Cambridge Health Alliance in Malden, Mass., has developed a tool to cut through the clutter.

The development of the Guideline Trustworthiness, Relevance and Utility Scoring Tool (G-TRUST)(clinicalinformationsciences.com) was highlighted in an article in the September/October issue of Annals of Family Medicine.(www.annfammed.org)

"We developed the tool to help physicians find guidelines that are relevant, useful and trustworthy so they can provide better care," Shaughnessy told AAFP News.

Story highlights
  • A research team has developed a new tool -- highlighted in the September/October issue of Annals of Family Medicine -- that can help family physicians find useful, evidence-based clinical guidelines.
  • The Guideline Trustworthiness, Relevance and Utility Scoring Tool (G-TRUST) offers physicians a way to quickly evaluate a practice guideline and decide whether a guideline is useful, may not be useful, or is not useful.
  • The researchers' G-TRUST checklist accurately identified 92 percent of low-quality guidelines evaluated and disqualified many so-called high-quality guidelines because of a stricter definition of trustworthiness.  

The G-TRUST instrument outlines eight questions physicians can use to quickly evaluate a practice guideline and, using the tool's scoring system, decide whether it is useful, may not be useful, or is not useful, said Shaughnessy.

"It's especially valuable when two or more guidelines conflict or if physicians are being asked to follow guidelines that they doubt," he added.

Shaughnessy said physicians told the research team their main concern with previous assessment tools was that, unlike the G-TRUST, they were long and didn't result in a clear-cut score to differentiate between good and bad guidelines.

Tool Development

To develop the G-TRUST, the researchers used a modified Delphi approach for their study design to "obtain expert consensus on items for inclusion, to hone the wording of the items and to develop a ranking system."

The authors selected their volunteer experts from two populations: those who had helped develop practice guidelines known to be high-quality and those who self-identified and were recognized as experts in evidence-based medicine. Both groups included family physicians.

The G-TRUST's eight items were derived from several sources, including the Clinical Practice Guidelines We Can Trust(www.nap.edu) consensus study report from the National Academy of Medicine (formerly the Institute of Medicine), the Appraisal of Guidelines for Research & Evaluation II (AGREE II)(www.cmaj.ca) instrument, and previous research by the authors of this study and others on guideline validity.

"The items hew most closely to the National Academy of Medicine standards and are critical for evaluating and recognizing flaws in the evidence development process, the relevance of recommendations to clinical practice, and the threats to the judgment process of creating recommendations from the evidence," the authors said.

Results, Limitations

Using AGREE II scores as a reference standard, the researchers' G-TRUST checklist accurately identified 92 percent of the low-quality guidelines evaluated and disqualified many high-quality guidelines because of a stricter definition of trustworthiness.

"G-TRUST is more stringent than AGREE II, in that it stipulates an independent (i.e., nonconflicted) research analyst or methodologist be part of the process, based on recent research findings that including independent methodological experts may better ensure evidence-based and conservative recommendations," the authors said.

Additionally, the tool is more stringent in flagging financial conflicts of interest among guideline developers -- barring them rather than attempting to address the issue.

"Using the stricter requirement for conflicts of interest reflected in the G-TRUST led to many guidelines being rated as 'may not be useful' that would be rated as 'of high quality' by AGREE," the study said.

The authors acknowledged the limitations posed by the G-TRUST's conservative cutoff score, which they said would prevent false-positives (i.e., identifying lower-quality guidelines as high-quality) but also would exclude some high-quality guidelines.

"Given the large number of guidelines, this emphasis is needed to ensure that fewer low-quality guidelines will be incorrectly identified as useful," the study noted.

A second limitation is that users may have difficulty determining conflicts of interest and the presence of a research analyst in a guideline development group.

"Despite extensive searching, we could not determine the answer to these items for almost one-half the studied guidelines," the authors said. "In a previous study, we found that more than one-half (57 percent) of the guidelines for the treatment of major depressive disorder did not include a conflicts of interest policy or disclosure statement."

In addition, neither the G-TRUST nor any of the other guideline assessment tools evaluate whether guidelines provide enough information to support shared-decision making, the study said.

However, even with these limitations, the tool appears to be another step in the right direction to simplify decisions for family physicians with busy schedules. Further research should help determine the reliability of the G-TRUST by comparing scores calculated by individual users.

"We hope physicians will start using G-TRUST rather than being confused by conflicting guidelines or feel like they are being 'forced' to follow a guideline simply because it's from a professional society," Shaughnessy concluded.

Related AAFP News Coverage
Choosing Wisely: Five Years of Focusing on 'the Right Care'
(10/27/2017)

AAFP Webcasts Offer Insight Into Clinical Practice Guideline Development, Implementation
(10/14/2013)

More From AAFP
American Family Physician: Top POEMs of 2016 Consistent With the Principles of the Choosing Wisely Campaign
(8/15/2017)

Additional Resource
Annals of Family Medicine: From Authority- to Evidence-Based Medicine: Are Clinical Practice Guidelines Moving Us Forward or Backward? (www.annfammed.org)
(September/October 2017)