AFP Journal Club
The Story Behind the Study
If It Seems Too Good To Be True…
Am Fam Physician. 2015 May 15;91(10):729-730.
In AFP Journal Club, three presenters review an interesting journal article in a conversational manner. These articles involve hot topics that affect family physicians or “bust” commonly held medical myths. The presenters give their opinions about the clinical value of the individual study discussed. The opinions reflect the views of the presenters, not those of AFP or the AAFP.
Vinson JA, Burnham BR, Nagendran MV. Randomized, double-blind, placebo-controlled, linear dose, crossover study to evaluate the efficacy and safety of a green coffee bean extract in overweight subjects [retracted in: Diabetes Metab Syndr Obes. 2014;7:467]. Diabetes Metab Syndr Obes. 2012;5:21–27.
Does green coffee bean extract help promote weight loss in overweight patients?
Jill: Overweight and obesity are reaching epidemic proportions in the United States. With 69% of Americans classified as overweight or obese,1 physicians and patients are clamoring to find solutions, and the market is ever ready to welcome them. Recent literature has suggested that coffee may help some people maintain a healthy weight. This study investigates green coffee bean extract, high in chlorogenic acid (which is destroyed in the roasting process and is hypothesized to reduce carbohydrate absorption and/or improve fat metabolism to promote weight loss), to determine if it reduces weight in otherwise healthy individuals.
What does this article say?
Jill: The authors studied 16 adults 22 to 46 years of age without thyroid dysfunction, diabetes mellitus, or hypertension and with a body mass index (BMI) between 25 and 30 kg per m2. The 22-week double-blind crossover study compared weight loss among participants receiving low-dose green coffee bean extract (350 mg twice daily), high-dose green coffee bean extract (350 mg three times daily), and placebo (350 mg inert capsules three times daily). It was given in differing sequences to which the participants were randomly assigned. Each phase of the sequence was six weeks, followed by a two-week washout (no supplementation taken). Weight, height, and body fat percentage were measured at baseline and at six, eight, 14, 16, and 22 weeks. All participants were counseled about diet and exercise at each visit. Data were analyzed using repeated measures analysis of variance with factors including the sequence, treatment arm (low-dose, high-dose, placebo), and time (pretreatment [zero, eight, and 16 weeks], posttreatment [six, 14, and 22 weeks]). Results revealed a reduction in weight (average of 10.5%), BMI (10.3%), and body fat percentage (15.8%) in the low-dose and high-dose treatment arms, but not in the placebo arm. However, things are not always as they seem.
Should we believe this study?
Mark: It's a small study, but enticing, right? A randomized, double-blind, placebo-controlled crossover study? Those are all things we want to see. It seems like a well-designed trial, and the authors report no conflicts of interest. I question, though, whether you can really call a study blind when the dosing regimens differ. It's pretty clear whether you're taking a pill twice or three times a day. I also worry about the Hawthorne effect (observation bias), which occurs when people change their behavior when they know they are being watched. Those changes in behavior could certainly promote weight loss.
Bob: The reported results are significant. A weight loss of 10% in a few weeks without severe caloric restriction? It appeared to be an effective weight-loss pill with no reported side effects. With Dr. Oz proclaiming it as “magic,”2 when this stuff hit the market, sales soared.
Jill: I agree, it looks great on the surface. These data are published in a peer-reviewed journal. So, why wouldn't we suggest this drug to all of our patients? Well, there's more to the story—the article was retracted in 2014 after a Federal Trade Commission settlement. It turns out that the company who made the extract paid a researcher in India to conduct this study, and the authors allegedly falsified the data (weight measurements, treatment arm, trial length). When the original author couldn't get the study published, the company hired two more researchers to write the study results for publication.3
Mark: The latter researchers made a big mistake. They didn't verify the data and thus had to retract their paper when all of this came out. So, there was industry funding that wasn't disclosed, falsified data, and authors who weren't involved in conducting the trial. A healthy dose of skepticism would have saved these authors (and a lot of consumers) a significant amount of money and distress.
Bob: Sometimes we want a solution so badly that we don't ask all of the appropriate questions, particularly with weight-loss pharmaceuticals. The U.S. Food and Drug Administration approves many drugs for weight loss, and later withdraw the approval because of dangerous adverse effects.
What should the family physician do?
Mark: Don't believe everything you read, and wait for other studies to confirm results. There may, indeed, be some health benefits to green coffee bean extract. There are some animal studies that suggest antioxidant effects and beneficial effects on glucose and lipid metabolism. The mechanism of effect is scientifically plausible, but we need reliable and reproducible research to tell us if this is the case in humans.
Bob: Be wary of drawing conclusions from small studies that have not been replicated—this is a red flag to me. Perhaps the most recent well-known retraction involved the extraordinarily small (n = 8) 1998 study published in the Lancet that suggested that the measles, mumps, and rubella vaccine is associated with autism. Since its publication (which resulted in decreased immunization rates), 10 of the 13 authors publicly retracted the interpretation they reported (again, because of fraudulent data), and the Lancet's editor has acknowledged that had they appreciated the full context of this paper, “publication would not have taken place in the way that it did.”4,5
Jill: Indeed, we have many cases to illustrate the need for repeated studies. Fraudulent practices take time to uncover, but there are also other potential problems with early adoption of new clinical strategies. Even well-done studies may be refuted by later evidence, or we may find out that a new therapeutic intervention has dangerous consequences. We don't know yet whether green coffee bean extracts are effective and safe. In the meantime, a healthy diet, appropriate caloric intake, and regular physical activity remain our best defense against overweight and obesity.
There are dangers in the early adoption of treatment modalities without repeated verification in the literature.
A huge market exists for weight-loss products, and we must use caution in evaluating the effectiveness and safety of such products.
The Hawthorne effect (also known as observation bias) occurs when individuals modify their behavior when they know they are being observed.
Blinding is ensuring that participants, clinicians, and/or investigators do not know which participants are assigned to each study group. Using seemingly identical products (in terms of taste, appearance, odor, and even texture) with the same dosing regimen is a common practice to help achieve blinding.
Retraction watch (http://www.retractionwatch.com) is a blog about retractions in the scientific literature. There is currently no database cataloging retractions.
REFERENCESshow all references
1. Centers for Disease Control and Prevention. FastStats. Obesity and overweight. http://www.cdc.gov/nchs/fastats/obesity-overweight.htm. Accessed March 24, 2015....
2. Phillip A. Researchers retract bogus, Dr. Oz-touted study on green coffee bean weight-loss pills. Washington Post. October 22, 2014. http://www.washingtonpost.com/news/to-your-health/wp/2014/10/22/researchers-retract-bogus-dr-oz-touted-study-on-green-coffee-bean-weight-loss-pills/. Accessed March 24, 2015.
3. Retraction Watch. Authors retract green coffee bean diet paper touted by Dr. Oz. http://retractionwatch.com/2014/10/20/authors-retract-green-coffee-bean-diet-paper-touted-by-dr-oz/. Accessed March 24, 2015.
4. Murch SH, Anthony A, Casson DH, et al. Retraction of an interpretation. Lancet. 2004;363(9411):750.
5. Horton R. The lesson of MMR. Lancet. 2004;363(9411):747–749.
For more information on evidence-based medicine (EBM) terms, see the EBM Toolkit at https://www.aafp.org/afp/ebmtoolkit.
This series is coordinated by Sumi Sexton, MD, Associate Medical Editor.
A collection of AFP Journal Club published in AFP is available at https://www.aafp.org/afp/jc.
Copyright © 2015 by the American Academy of Family Physicians.
This content is owned by the AAFP. A person viewing it online may make one printout of the material and may use that printout only for his or her personal, non-commercial reference. This material may not otherwise be downloaded, copied, printed, stored, transmitted or reproduced in any medium, whether now known or later invented, except as authorized in writing by the AAFP. Contact email@example.com for copyright questions and/or permission requests.
Want to use this article elsewhere? Get Permissions
More in AFP
MOST RECENT ISSUE
Feb 15, 2018
Access the latest issue of American Family Physician