Blood sugar, cholesterol levels, or bone density: Can measurements show if a treatment works?
In the 1980s, millions of people were treated with drugs intended to prevent sudden cardiac death. But it later turned out that the exact opposite happened: the drugs increased mortality. What had gone wrong, and what can we learn from this experience?Some people develop a specific form of irregular heartbeat (arrhythmia) after a heart attack. They have an increased risk of sudden cardiac death. In the 1970s, researchers developed drugs called 1c antiarrhythmic agents, which aimed to normalize irregular heartbeat. Clinical trials showed that these drugs did successfully normalize the heartbeat in electrocardiographs (ECGs, also sometimes called EKGs). Because of this supposedly positive effect these antiarrhythmic agents were used a lot in the 1980s.
In the late 1980s, a group of researchers initiated a trial on class 1c antiarrhythmic agents, called CAST trial. This trial studied not only the effect the drugs had on the heartbeat, but also how they affected mortality from sudden cardiac death. The results were sobering: the rate of sudden cardiac death was twice as high in the group who had used an antiarrhythmic agent as in the group who had taken a dummy drug (placebo).
Plausible is not enough
Why had people for years been treated with drugs that double the death rate? Because experts had drawn the wrong conclusions: Irregular heartbeat was known to increase the risk of sudden cardiac death. So they concluded that drugs against irregular heartbeat might be able to lower this risk. From a medical point of view, this conclusion seemed to be perfectly plausible. But it still turned out to be wrong.
The results of the CAST trial are now considered a prime example of why measurements alone cannot be relied upon. For a long time, the ECG measurements were considered to be a substitute criterion for the risk of dying. Criteria that are used in trials to substitute an important endpoint are also called surrogate endpoints or surrogate markers (from the Latin surrogatum, meaning substitute).
Endpoints that are important to patients – such as mortality, heart attacks, quality of life or the length of hospital stays – on the other hand, are called patient-relevant endpoints. The term “patient-relevant” reflects the fact that it concerns issues that are important to the people who have a disease – for example if a treatment helps them live longer, spares them from going to the hospital, reduces their symptoms, prevents complications, or helps them cope better with their disease in daily life. The following table shows some examples of surrogate endpoints and the corresponding patient-relevant endpoints:
|Surrogate endpoint||Patient-relevant endpoint|
|High cholesterol levels||Heart attack|
|Low bone density||Broken bone|
|Irregular heartbeat||Sudden cardiac death|
|High blood pressure||Stroke, heart attack|
|Tumor does not respond to treatment||Death, reduced quality of life|
Correlation does not say anything about cause and effect
Another example of a misleading surrogate endpoint is bone density as an indicator for the risk of broken bones in women after menopause. In the 1980s, a trial was done to test whether sodium fluoride taken in addition to calcium can lower the risk of broken bones in women with osteoporosis. Examining the bones revealed that the bone density of women who took sodium fluoride increased. Yet they had more broken bones than did women who had taken only a placebo in addition to calcium. The reason for this is that sodium fluoride did increase bone density, but at the same time changed the composition and quality of the bone material. This made the bones brittle.
Someone who has a particular disease often has abnormal laboratory data or measurements. Such a correlation can help understand a disease better – but it does not describe a relation of cause and effect. This is why choosing a treatment that brings laboratory data to normal values will not necessarily reduce the risk of a disease.
Most surrogate endpoints cannot take into account the complex processes happening in the body. Sometimes a particular value will deviate from the norm even in a healthy person. Or a treatment influences the patient-relevant endpoint, but not the surrogate endpoint. A treatment can also affect a surrogate endpoint without influencing the patient-relevant endpoint. This is why most surrogate endpoints are not reliable on their own. They can be misleading when it comes to evaluating how helpful a treatment is.
Surrogate endpoints: tempting, but only rarely reliable
The reason that studies often only use surrogates and not the endpoints that are important for patients is that they are a lot easier to measure: a study will show quickly whether a medication lowers blood pressure, for example. But it can take years before researchers find out whether this also prevents diseases like heart attacks.
Another reason is that studies on surrogate endpoints need far fewer participants. This has statistical reasons: heart attacks, for example, are quite rare, so a large number of people must be monitored to see clear differences between the different treatment groups. In a trial on blood pressure, however, changes in blood pressure can be measured for each individual participant, so that only few participants are required to make out an effect.
Caution is needed when laboratory data and physical measurements are used as surrogates in trials to measure the benefit a treatment has for patients: just because a drug lowers blood pressure, it does not automatically protect against heart attacks or strokes. This has to be tested in trials that examine not only blood pressure levels, but also the effect of the drug on cardiovascular disease.
Using surrogate endpoints only makes sense if they really say something about the benefit a therapy has. This is the case if the effect a treatment has on the surrogate endpoint predicts how the treatment affects the patient-relevant endpoint. In exceptional cases, researchers might have to content themselves with correlations that are less conclusive.
Laboratory data have their place in medicine
Laboratory data and physical measurements are anything but useless in medicine, though. They are needed to make a diagnosis, predict or control the progress of a condition, or check whether a treatment is working or a dose is right. Someone with type 1 diabetes, for example, will regularly monitor blood sugar levels in order to adjust his or her insulin dose. Laboratory tests and ECG are used to diagnose a heart attack.
Even if measurements replacing patient-relevant endpoints are usually problematic: sometimes there are ethical reasons for using surrogate endpoints in studies. If there has not been a treatment for a serious disease yet, for example, it may make sense to initiate a new treatment even without knowing exactly what effect it has.
This was the case with the first HIV drugs, for example: Trials had shown that they could significantly reduce the number of human immunodeficiency viruses (HIV) in the body. But there were no trials showing that this leads to fewer people developing AIDS or dying. Because there were no alternative treatments and HIV progresses rapidly if untreated, the drug regulating authorities approved these drugs anyway. Today we know that this has saved thousands of people with HIV from an early death.
Our information is based on trials using patient-relevant endpoints
The publisher of this website, the German Institute for Quality and Efficiency in Health Care (IQWiG), has been given a legal mandate to assess the benefits and harms of medical interventions. To do this, IQWiG analyzes the available trials with patient-relevant endpoints or with reliable surrogates. It generally examines whether patients can actually benefit from a particular treatment. This is why the IQWiG assessments do not refer to effectiveness, but to the benefit and/or harm of a medical intervention. The fact that a treatment is effective does not automatically mean that it also has a benefit for patients. Often the possible benefit has not been sufficiently proven, or the trials only show indications of a benefit.
IQWiG is also required to provide consumers with unbiased and independent information about health issues on the website Informed Health Online. The research this information is based on must fulfill certain criteria and examine patient-relevant endpoints or reliable surrogates. You can find more information about these fundamental principles in our information “Evidence-based medicine.”
Published by the Institute for Quality and Efficiency in Health Care (IQWiG)
- January 30th 2013 09:37
- December 31st 2009 21:34
Bucher HC. Studien mit Surrogatendpunkten – Nutzen und Grenzen in der klinischen Entscheidungsfindung. Internist 2008; 49: 681-687. [Summary]
Bucher HC, Guyatt GH, Cook DJ, Holbrooke A, McAlister FA. Users’ guides to the medical literature XIX. Applying clinical trial results. A. How to use an article measuring the effect of an intervention on surrogate end points. JAMA 1999; 282: 771-778.
Epstein AE, Hallstrom AP, Rogers WJ, Liebson PR, Seals AA, Anderson JL et al. Mortality following ventricular arrhythmia suppression by encainide, flecainide, and moricizine after myocardial infarction. The original design concept of the Cardiac Arrhythmia Suppression Trial (CAST). JAMA 1993; 270: 2451-2455. [Summary]
Fleming TR, DeMets DL. Surrogate end points in clinical trials: are we being misled? Ann Intern Med 1996; 125: 605-613. [Full text]
Grimes DA, Schulz KF. Surrogate end points in clinical research: hazardous to your health. Obstet Gynecol 2005; 105: 1114-1118. [Summary]
Institute for Quality and Efficiency in Health Care (IQWiG). General Methods: Version 4.0. September 23, 2011 [Accessed: October 24, 2012]. [Full text]
Institute for Quality and Efficiency in Health Care (IQWiG). Validity of surrogate endpoints in oncology: Rapid Report; Commission no. A10-05. November 21, 2011 [Accessed: October 11, 2012]. (IQWiG Reports; Volume 80). [Executive summary] [Full text – in German]
Mangiapane S, Velasco Garrido M. Surrogatendpunkte als Parameter der Nutzenbewertung. Köln: Deutsches Institut für Medizinische Dokumentation und Information (DIMDI); 2009. (Health Technology Assessment; Band 91) [Full text – in German]