The basic principles of evidence-based medicine

A spoon put into an opened champagne bottle is supposed to keep the champagne fresh and bubbly for longer. British researchers tested this theory with a simple experiment (1). To do this, they put not just one but two open and half-empty bottles overnight in a refrigerator: one with a spoon, one without.

On the next day two experts drank champagne from those bottles. The first finding was that the champagne stayed drinkable for a surprisingly long time. It was more than four days before it was flat.

The second finding was that the experts could not tell the difference between champagne from a bottle with a spoon and one without. Both bottles went flat as quickly as each other.

Such a simple experiment, with such a clear answer: we get the impression that a spoon keeps the champagne fresh because the champagne would have stayed fresh anyway.

You may well ask, what does this experiment have to do with evidence-based medicine? Quite a lot, actually. Because the same strategy that showed the spoon is no help can be used to test almost all medical treatments.

Swap the champagne bottle for "Illness X" and the spoon for "Therapy Y", and you have the basic elements you need to test medical claims. Of course, it is a bit harder to judge health care, but the principle is the same.

This kind of test has become normal in health care in the last 50 years. They are called clinical studies, randomised trials or controlled trials. There are now more than 20,000 of them published every year.

The core idea, when you want to test a new medicine for example, is to randomly divide a group of volunteers into at least two groups. One of the groups will use the new medicine, and the other will get another treatment or a placebo (dummy tablet). The people need to get assigned to their groups completely by chance so that the two groups are really comparable: the only difference between the groups has to be the treatment. At the end, the experiences and outcomes of the participants can be compared.

Just how these trials need to be done to deliver a useful result depends on the illness, the treatment and the consequences of both. Sometimes, watching 20 or 30 people for a few days would be enough. But sometimes you need to follow more than 10,000 men and women for many years to know what might have happened. Sometimes, you need even more people to be sure. For longterm chronic diseases like coronary artery disease, cancer, arthritis or diabetes, a really useful result will only come after years of study.

Not just one study at a time

Only rarely is a single trial big and strong enough to provide, on its own, a reliable answer about the effects of a health care treatment. If a trial has too few participants, it can paint a misleadingly optimistic or pessimistic picture. That is why it is usually necessary not to rely on just one trial, but ideally to consider the results of all the trials that have been done.

Researchers have developed a technique for doing this. It is called a systematic review. These reviews bring together the results of whatever trials have been done. This gives you an overview of the state of medical knowledge on that treatment.

Getting the best answer - step by step

A systematic review can only come to a trustworthy answer if it is thorough and comprehensive. This very careful and extensive search for, and evaluation of, trials means that a systematic review is often a major piece of work.

This is how it works:

1. The question: The researchers have to decide exactly what question they want to find the answer to. The exact nature of the question decides what specific studies are needed to answer it.

2. The search: Once they know what they are looking for, the researchers have to search as thoroughly and comprehensively as possible for all the possible studies that might help answer the question. That could even mean hunting through many hundreds or even thousands of articles.

3. The selection: Every study that is found has to be checked against very specific pre-defined criteria. The studies that do not meet the criteria have to be rejected.

4. The analysis: The studies that make it through need to be quality-assessed, and analysed. This can then provide a big picture of what is known and what is still uncertain.

5. Discussion: The researchers need to report in detail the steps they took and what they found. This draft needs to be assessed by experts, often from around the world. This is called "peer review".

6. Publication: After the peer review and the review has met the medical journal's standards, it can be published.

One of the main journals where systematic reviews are published is the "Library" of the Cochrane Collaboration. It contains the Cochrane Database of Systematic Reviews. Behind this name is a mountain of an international scientific network. Over 10,000 people are involved in the specialised work of producing systematic reviews for Cochrane alone.

Systematic reviews theoretically allow everyone to know the state of knowledge on what treatments work.

How does that help you?

There are treatments that make you feel better straight away. But sometimes it only seems as though it was the medicine or other treatment that worked. In reality, it might have been something completely different that helped - like staying in bed for a few days. Or your body could have handled the illness itself, and you would have got better with or without treatment. And sometimes, the side effects of the treatment can be worse than the disease itself.

If you want to know if it is really worth trying a treatment, you might want solid information about which of these possibilities you may need to take into account.

You cannot know which of the possibilities will happen to you. But at least you can know what the chances are that you might be helped.

Systematic reviews are a relatively new scientific development. This means that there are many areas of medicine where these reviews have still not been done. Often a systematic review is done, only to find that there are simply have not been enough trials, or the trials were not of high enough quality to provide the researchers with enough confidence in their results.

That is why it is common for a systematic review to conclude that there is simply not enough evidence to be able to answer their question.

This result can seem disappointing, when you are looking for an answer. However we believe it could be better to know when there is uncertainty. Being open about not having all the answers protects you from the consequences of the errors that over-optimism or over-pessimism can cause. Uncertainty is less dangerous than false security.

Evidence-based medicine

This process of looking at the state of the evidence before making a decision is called evidence-based medicine or evidence-based health care. It means based on scientific evidence or supported by proof. It means not relying only on the views of experts, but on the most objective knowledge as well.

The goal is to help patients get the most appropriate treatment for them. After considering the evidence and the person's own values and judgment about their options, a patient and experienced doctor can come to a better decision. The best health care is not necessarily where "everything" is done for the patient, but rather, the most appropriate. This can protect people from harmful and useless treatments.
What's more, it is only when we know what the limits of knowledge are, that we can know what needs to be done in the future to fill the gaps in what is known.

What about when there is no clear answer?

Evidence-based medicine is not universally popular. Part of the reason is because there is so often no clear answer. Many people are concerned that knowing how much is uncertain might make people insecure or confused instead of helping them.

That might happen sometimes. But we still want to be open about when we are uncertain.

It is important to remember, too, that just because there is no clear answer now, it does not mean that there is nothing a patient can do and that someone should take over for them. If there is no conclusive evidence about a treatment, patients need the best description possible of the potential advantages and disadvantages of the alternatives they face so that they can choose what they want to do. A variety of medical and personal issues can influence this decision for the individual.

Of course no one can foresee every issue that might be important for you as an individual. This is one reason why we do not make recommendations and give specific advice. We limit ourselves to explaining what the state of knowledge is, so that after discussion with your doctor and your family, you can decide together what is right for you personally.

The Informed Health Online/Gesundheitsinformation.de Team: IQWiG

(1) New Scientist, 13.5.2000, page 39.