Homeopathy is represented negatively in mainstream journals: this is the true “publication bias” posted on 10-10-2005

The meta-analysis of Egger et al., recently published by Lancet [1] , presumes that in the literature the positive outcomes of clinical studies about homeopathy are essentially due to the fact that studies with negative outcomes are not published (positive publication bias).

This is a very common problem in conventional medicine: almost all the studies about drugs are financed by manufacturing companies, and this situation determines a greater probability of publishing studies with positive outcomes in comparison with those with negative outcomes. This obviously undermines the possibility of receiving from these studies trustworthy information for the clinical practice, even though these studies are formally correct (randomised studies, double blinded, and so on). Paradoxically, Evidence Based Medicine risks becoming a Biased Based Medicine.

Also academic medicine [2] is realizing this situation, so that it proposes recognition to the researchers who are not linked to pharmaceutical companies. As an attempt to reduce this problem, some methodologists have proposed several statistical methods; Egger has been one of the authors who worked most on this point: that is how to  try to reduce the publication bias in meta-analysis [3] carried out on clinical studies about conventional drugs. Briefly, one of the methods uses the funnel plot, another one considers studies with a large number of subjects for the final analysis, because by doing so the publication bias is reduced.

Can we apply this statistical method to homeopathy? First of all, it’s necessary to check the applicability, and ask:

Are the published clinical studies about homeopathic medicine essentially financed by homeopathic pharmaceutical companies (as happens for conventional medicine)?

The answer is negative: almost none of the clinical studies about homeopathy published until now has received any financing by homeopathic pharmaceutical companies. There is a study [4] which has analysed the quality of clinical studies in homeopathy in a detailed  way, and it has explained this aspect It claims that:

From a total of 59 studies, 84.7% haven’t received any financing (that is to say it was financed by homeopaths themselves), while only 11.9% were financed by foundations or private donor.

This explains also why these studies are made with small numbers: because they didn’t receive any financing (or very poor financing). So the problem of positive publication bias for homeopathy simply doesn’t exist.

Having said this, the method used in conventional medicine to reduce this bias can’t be applied. The authors of the study published by Lancet don’t consider this fact: they apply simply the procedures used in conventional medicine to homeopathic medicine (and they make a basic scientific mistake and a mistake of evaluation: applying procedures appropriate in  one context to another context automatically).

According to Egger and his colleagues, there would also be for homeopathy a selection of studies in the literature with positive results, whose outcomes are not trustworthy; the final meta-analysis of Egger (based only on 8 trials for homeopathy and 6 for allopathy) should reduce this publication bias using only studies with a large number of participants (large studies).

There is a second question: where is the demonstration that publication bias is positive as regards homeopathy? In other words, which are the studies that, for homeopathy, show that mainly studies with positive outcomes are published instead of those with negative outcomes?

The meta-analysis of Egger is silent also about this point: the positive publication bias is taken for granted.

In fact, until June 2005 in the scientific literature there were NO studies which addressed the prevalence of publication bias (positive or negative) about homeopathy. A recent explorative study about this important subject, published in BMC Complementary and Alternative Medicine [5] has showed that:

- mainstream journals publish 69% clinical studies about homeopathy with negative results
- complementary medicine journals publish studies with 30% of negative outcomes about homeopathy.

The authors of this study, researcher of the University of Alberta, Canada, conclude: “these results suggest a publication bias against homeopathy exists in mainstream journals”.

To conclude, there is a “publication bias” towards homeopathy, but the one which is predominant is the negative one (that is mainly trials with negative results are published by mainstream journals). And the importance of mainstream journals in the evaluation is greater for one simple reason: the majority of published journals are in fact mainstream
journals.

The whole meta–analysis published in Lancet starts from a prejudiced position: homeopathy can’t work. If it works, it’s only a positive bias, even if data show just the contrary when they are published. So the meta–analysis by Egger is included in the studies published in mainstream journals which affirm that homeopathy has negative outcomes: since this position is systematically denied by existing data, here lies the true publication bias.

Andrea Valeri Clinical Research Department Italian Society of Homeopathic Medicine
email: avaleri@omeomed.net
web-site: www.omeomed.net

Bibliography (web links active on the Italian Society of Homeopathic Medicine)

[1] Shang A, Huwiler-Muntener K, Nartey L, Juni P, Dorig S, Sterne JA, Pewsner D, Egger M.  Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet. 2005 Aug 27-Sep 2;366(9487):726-32 [Medline] [Free Full text]

[2] Miller FG, Brody H.  Viewpoint: professional integrity in industry-sponsored clinical trials. Acad Med. 2005 Oct;80(10):899-904.[ Medline]

[3] Egger M, Smith GD. Bias in location and selection of studies. BMJ. 1998 Jan 3;316(7124):61-6.  [Free Full text]

[5] Caulfield T, DeBow S. A systematic review of how homeopathy is represented in conventional and CAM peer reviewed journals. BMC Complement Altern Med. 2005 Jun 14;5(1):12 [Free full text]