Attacco di Lancet all'omeopatia: un anno dopo posted on 09-08-2006

Società Svizzera dei medici omeopati (SVHA):
20 ragioni per cui lo studio di Egger pubblicato su Lancet non è affidabile

A distanza di una anno, la critica metodica della SVHA è di stringente attualità: il lavoro di Egger non è un corretto studio scientifico e le sue conclusioni sono del tutto inconsistenti.

Link alla versione italiana di Lancet (Abstract)

Link alla pagine del PEK da cui si può scaricare l'articolo pubblicato su Lancet  (inglese)

Ecco la versione inglese integrale della presa di posizione della SVHA:

(il grassetto e le note sono del curatore, Andrea Valeri)

____________________________________________________
 

Swiss Association of Homoeopathic Physicians SAHOP SVHA/SSMH/SSMO/SAHOP
Dorfhaldenstr. 5, 6052 Hergiswil / tel. +41 (0)41 630 07 60 / sekretariat@svha.ch 1/4

Statement of the Swiss Association of Homoeopathic Physicians on the homoeopathic study of the Institute of Social and Preventive Medicine (ISPM) in Bern (“Egger study”)
(Are the Clinical Effects of Homoeopathy Placebo Effects? Comparative Study of Placebo-controlled Trials of Homoeopathy and Allopathy. Shang A., Huwiler K. et al., Department of Social and Preventive Medicine, M. Egger, University of Berne, 2004)

From a homoeopathic point of view, the ISPM homoeopathic study demonstrates considerable shortcomings and is not tenable for the following reasons.

Unfair procedure:

1. Breach of agreements: ISPM designed the study as part of the Complementary Medicine Evaluation Programme (PEK). With the preview presentation of the study in Basle at the end of August 2003 and the distribution of presentation documents to the media, ISPM breached several PEK agreements. ISPM a) did not submit the study for PEK internal discussion beforehand, b) did not meet the obligation of enlisting specialists, c) did not allow specialists to examine it after the media reports and therefore also disregarded usual scientific practices, d) did not meet the PEK obligation to maintain secrecy and e) did not comply with the Institute’s own publication guidelines. Whether the contract with PEK management was also breached is beyond our knowledge, since the wording of the contract is as unclear to us as the actual purpose of the work. The PEK executive committee and programme management have apologised to homoeopaths for certain errors made in assigning the ISPM study.

2. Untenable proceedings in the media: The ISPM study was the linchpin for the biased report in the SonntagsZeitung of 31.8.2003 [a] . Since then, Profe[ssor Egger and ISPM have repeatedly let it be reported in print and on the radio, that the effects of homoeopathy can be explained by the placebo effect. This damages the reputation of homoeopathy and unfairly condemns it as being simple and generating a lot of media attention, while being rash and wrong (cf. criticism of content). Both the scientific position and practice of Prof. Egger are untenable. The quiet counterstatements of PEK management and the Federal Social Insurance Office (BSV) in September 2003 received no publicity. The homoeopathic side has met the obligation to maintain secrecy within the framework of PEK, even though one of the co-authors of the ISPM study again referred to the negative result at a phytotherapy conference in November 2004 .

[a] Egger article publication date on Lancet is 27/08/2005

3. No examination of the study: Contrary to the agreements, ISPM has long resisted an examination of the content of its study within the framework of PEK. After refusing for a long time, ISPM finally allowed homoeopaths to examine the study at a private crisis talk on 3.6.2004 - 10 months(!) after it had been distributed to the media. And only 1½ years later, from the middle of January 2005, were homoeopathic specialists really able to see the study, which had been submitted for publication in August 2004.

4. No use of specialist knowledge: As the sole PEK contractor, ISPM did not enlist homoeopathic specialists. To avoid criticism about a lack of specialist knowledge, ISPM subsequently included a former homoeopath, now employed by ISPM, in the list of authors, who had not been named as a co-author until 3.6.2004.

5. Odd acknowledgements: ISPM included – without prior request and unknown to them – homoeopathic physicians, who took part in the crisis talks, in the acknowledgements list of its publication. This is a peculiar and odd action, which is seen by homoeopaths as an attempt by the authors to avoid criticism about the non-participation of experts and lack of specialist knowledge. In the meanwhile, all the homoeopaths named in the acknowledgements have prohibited ISPM from naming them in the article.

6. Summary: The approach of Prof. Egger and ISPM with regard to PEK and in the media is in breach of agreements, is unfair, does not conform to scientific practice and is untenable.


Criticism of content:

7. False message: The simple message of the ISPM study, “Homoeopathy equals placebo”, admittedly generates a lot of media attention, but is untenable. The study claims that it measures homoeopathy but in fact the study is unable to assess homoeopathy at all using its methodology. This is because firstly, the homoeopathic studies examined have almost nothing to do with homoeopathy as it is really practiced, secondly, almost all other large surveys and meta-analyses of homoeopathic studies have a positive result, and thirdly, the study demonstrates a series of further shortcomings.

8. Distorted homoeopathy for study purposes: Most homoeopathy effectiveness studies do not comply with basic homoeopathic rules, in no way correspond to real treatment practice and are – unlike studies of conventional medicine – irrelevant to practice. These studies are only carried out as external justification for homoeopathy. In real practice the selection of homoeopathic remedies is carried out completely on an individual basis. The remedy does not have a universal effect; rather it proves to be effective for the individual. In contrast, for study purposes, homoeopathic practices are usually distorted and standardised and forced into irrelevant research schemes. The result of this is a considerable risk of obtaining falsely negative results. The homoeopathic attitude to such studies is conflicting.

9. Lack of homoeopathic knowledge: Disregarding or in ignorance of these facts, ISPM makes no differentiation, either in its publication or in press statements, between homoeopathy as it is really practiced and distorted homoeopathy for study purposes. A lack of effect in studies is equated with the remedy not being effective and therefore a lack of homoeopathic effect. The ISPM authors’ handling of homoeopathy as an object of study is scientifically dubious. There is no indication that their study is in any way representative of homoeopathic practice or of the work of holders of the SVHA/FMH (Swiss Medical Association) homoeopathic certificate of proficiency in Switzerland.

10. Other meta-analyses are positive: Despite non-compliance with basic homoeopathic principles, a large number of homoeopathic studies have succeeded in proving experimental effect and clinical effectiveness. In almost all large synthesis studies, meta-analyses and systematic reviews homoeopathy proves to be effective (see HTA Homoeopathy Report PEK 2005). Among them are both the most well known and rigorous studies from a conventional medicine point of view, e.g. by Kleijnen et al. 1991, Boissel et al. 1996, Linde et al. 1997 and Cucherat et al. 2000, and the large new surveys, which are informative in content, such as those by Wein 2002, Mathie 2003, Dean 2004. The authors in no way explain or analyse their diametrically opposed results, which, to a large extent, are drawn from a similar selection of studies.

11. Problematic nature of RCT studies: The ISPM study only examines RCT (Randomised Controlled Trials). It is sufficiently well known that this type of study only has limited significance not only with regard to homoeopathy, but also with regard to more complex therapies in general (HTA Homoeopathy Chap. 3.2.3. and 12.; Kiene 1993). The authors would have known this from literature and from homoeopaths, but they have not taken any notice or indicated this.

12. The ISPM study, which was finally made available to homoeopathic specialists in the middle of January 2005(!), demonstrates several other shortcomings and raises many questions:

13. Dubious study selection: The ISPM study compares 110 homoeopathic studies with 110 conventional studies. However, the study selection is questionable. By 1991 Kleijnen et al. had already found 107 RCT studies, 81 of these showing a positive result for homoeopathy. In 1996 Boissel et al. had already analysed 184 RCT studies with high significance for homoeopathy (17 qualitatively best studies with p<0.001). Wein found 132 studies (2002) in addition to those mentioned by Kleijnen et al. Why did ISPM only find 110 studies that could be assessed and why did it make this particular study selection? How did ISPM come to the erroneous conclusion that it had recorded almost all the RCT studies when its selection is so incomplete?

14. Hardly any classical homoeopathy studies: In the ISPM study only a small number of RCT studies using individual single remedy homoeopathy can be found (17 studies, some of which only with limitations). With these too the methodology is partly distorted due to the study design and the significance is limited because of the short study duration and the problem associated with long-term assessment of RCT studies. The whole ISPM study practically has no external validity with regard to homoeopathy as it is really practiced.

15. Intransparent study details: One of the serious shortcomings of the ISPM study is that it has no information on the individual studies used. Only a list of references is provided. Therefore, it remains unclear which and how many of the individual studies give a positive or negative result, how large they are and how they are evaluated!

16. Problematic analysis: The assessment is solely carried out using the funnel plot method, a graphical statistical analysis. According to the authors, 21 homoeopathic studies and 9 conventional studies demonstrate higher quality. So the main emphasis is on a very small number of large studies. The result from only 8 “good” homoeopathic studies with a large number of participants is less positive than that from only 6 “good” large conventional studies. With the funnel plot method, the authors draw a straight line from the small and medium-sized studies in the direction of the few large, but less effective ones. The continuation of this straight line moves more in the direction of the placebo line for homoeopathy than for allopathy. Therefore, from the overwhelming number of positive studies (a 3:1 ratio insofar as visible on the graph), a negative result emerges for the authors. The result is suspicious. How can a negative result be produced from so many positive studies? The negative result for homoeopathy is entirely the outcome of a statistical extrapolation from a few large studies. (Here is a comparative example: If the method is used to answer the question of whether there are red apples and in order to do this 10 apple trees with red apples and 3 trees each with a slightly larger number of green apples are examined, the funnel plot method produces the result that - there are no red apples.)

17. Overemphasis of the size of the study and unsuitable study method: The funnel plot method with its biased high evaluation of the number of study participants may be justified as graphical statistical analysis for the examination of homogenous standardised interventions of conventional pharmacotherapy, however, it is not appropriate for homoeopathic studies with their heterogeneity and complex study problems. If for example one takes a range of studies concerning beta-blockers for a certain indication, the size of the study may be important and the ISPM method may contribute to clarification of the effect. With homoeopathic studies this is not the case for the following reasons: 1. The studies include different study models and different distorted homoeopathic methods, which cannot all be regarded in the same way. Otherwise it is like comparing apples to pears. 2. The ISPM method would only be correct for examining the same type of study models or homogenous interventions and methods. 3. As a rule the larger the study, the smaller the chance that adequate homoeopathy has been examined and the less valid it is to real practice (low external validity). 4. The homoeopathic studies hide the risk of falsely negative results because of their low validity to real practice. In the context of this distorted homoeopathy for study purposes there exist trials with both, positive and negative results. A negative result just means that the model is not suitable for proving effectiveness. When counting results, positive and negative ones cannot be set off against each other (HTA Homoeopathy Chap. 12). Proof of effectiveness with distorted homoeopathy for study purposes can only ever be exemplary.

18. Correct application? The technical quality and correct application of the complex statistical graphical analysis method cannot be assessed by us; this must be done by other analysts.

19. Intransparent analysis: In their study the authors do not make it clear which point in their graphical statistical analysis corresponds to which study and ought to be critically analysed, which is clearly a shortcoming. They do not demonstrate in their study, which large studies lead to a negative result. Why this intransparency in making it possible to comprehend the results? Information on the individual studies is missing so that the ISPM study can only be checked by laboriously sifting through the list of references and assessing the individual studies oneself! Only then does it become clear on which studies the negative result of the ISPM study is based. 
 
20. Here are a few necessary additions: Some of the large studies deal with the treatment and prevention of influenzal infections: Attena et al. (1995) was able to establish just as little preventative effect from Oscillococcinum C200 in 1,595 patients as Ferley et al. in 1987 with a complex remedy in 1,182 patients. In other large studies there was evidence of positive effects (Ferley et al. in 1989 with Oscillococcinum C200 in 487 patients, Rottey et al. in 1995 in 501 patients with Mucococcinum C200, Papp et al. in 1998 in 372 patients with Oscillococcinum C200, Diefenbach et al. in 1997 with a complex remedy in 258 patients), but the still larger participant numbers of the negative studies have more influence when applying the funnel plot method. In a further large study on 1306 school children in India in 1992 Mokkapatti et al. found that Euphrasia C30 had no effect during a conjunctivitis epidemic. An example from another field is that in 1998 Vickers et al. found no prophylactic or therapeutic effect of Arnica D30 against pains from running in 519 long-distance runners on stretches of 2-50 miles, in contrast to earlier studies by Tveiten. A few large studies like this led to the negative result of the ISPM study. We ask what should we conclude from these studies? Why should the symptoms all be treated with a standardised remedy and why particularly with the one used? In whom and to what extent does long-distance running lead to symptoms worth to be treated at all? Why do the authors use this study in particular and only this study from the third world, when in India alone there are more than 2000 homoeopathic outpatients’ clinics? From the point of view of conventional medicine such studies may be valid, from a homoeopathic point of view they are qualitatively bad, not representative and unacceptable. Such studies force homoeopathy for study purposes into standardised schemes that are lacking in practical relevance, and have low external validity. Whether the remedies used in certain studies can have effects remains open. The studies represent dubious attempts to prove the effect of high potency homoeopathy. From their results it can only be concluded that the study design is not suitable and not that homoeopathy is not effective (falsely negative results with low external validity). The absence of proof is not the proof of absence! It is scientifically inadmissible and not compelling to try to prove the ineffectiveness of homoeopathy on the basis of such studies and to publicise, without comment, such studies on the basis of one-sided statistical thinking of conventional medicine without taking the basic principles of homoeopathy into consideration. At the same time there is the question of whether statistical extrapolation for the elimination of sources of error does not itself become a source of error. In any case, the scientific consequence of ISPM’s inadmissible extrapolation and generalisation of such studies is enormous! It is tremendously negligent and scientifically untenable.

21. Subgroups: The authors also claim that in the same way, diagnosis-related subgroups do not show any other results. This statement also raises doubt because in some clinical fields the positive evidence significantly prevails (e.g. pollen allergy studies by Reilly/Taylor and Wiesenauer; see also Mathie 2003, HTA Homoeopathy Chap. 3.2.).

Conclusion1: Nowhere do the authors acknowledge the homoeopathic quality and the lack of practical relevance of the studies examined (low external validity). They claim to be measuring something which they are unable to assess on the strength of their study. In summary, the ISPM study of Egger and colleagues demonstrates, in addition to questionable procedures, considerable shortcomings with regard to content. The study’s assumption that homoeopathic effect is due to placebo therapy is scientifically untenable on the basis of the facts.

Conclusion 2: Despite a breach of basic homoeopathic rules, numerous studies have succeeded in producing fundamental and exemplary evidence for the experimental effect and clinical effectiveness of homoeopathy. Recent large observational studies (outcome studies), e.g. in Germany on up to 4,000 patients also show evidence of effectiveness under real practice conditions and with higher external validity (HTA Homoeopathy Chap. 3.2.).

Literature note: for references see the ISPM study and the Human Technology Assessment HTA Report on Homoeopathy, PEK 2005.

For information please contact: Swiss Association of Homoeopathic Physicians SAHOP