Sunday, May 10, 2015

Proposed Mechanisms and Observational Studies can be Inconsistent with the Results from Randomised Controlled Trials

As risk factors, whether you decide to use total-C, LDL-C or the total-C:HDL-C ratio, all of them would predict that replacing SFA with PUFA would reduce the incidence of CHD*.  In addition, some** meta-analyses of observational studies would also predict that replacing SFA with PUFA would reduce the incidence of CHD.  However, this prediction didn’t come true in the ‘adequately controlled’ clinical trials (blog), and based on limited data the opposite may be the case for women (higher risk of CHD) (blog).

At this point, many conventional dieticians/doctors will claim that the trials are flawed/difficult and appeal to the ‘totality of the evidence’, emphasising the effects of SFA and PUFA on blood lipids and a selection of favourable meta-analyses of observational studies, before recommending that we continue to replace SFA with PUFA.  This is despite meta-analyses of RCTs being the gold standard in medicine.  For example, in the NHMRC evidence hierarchy, meta-analyses of RCTs are at the top (level I), and are followed by RCTs (level II), pseudo-RCTs (level III-1), then cohort studies (level III-2) [1]

We should be cautious of basing recommendations on observational studies and/or proposed mechanisms.  There are some number of examples of things that improve the blood lipid profile and/or are inversely associated with CHD in observational studies, but don’t reduce CHD in RCTs.  Replacing SFA with PUFA is not the first time this has happened and won’t be the last:

·         Hormone therapy (estrogen or estrogen + progesterone) lowers the total-C:HDL-C ratio and is associated with a lower incidence of CVD in postmenopausal women, but a recent Cochrane meta-analysis of RCTs found no benefit for hormone therapy in postmenopausal women [2]
·         CETP increases the total-C:HDL-C ratio, but a recent meta-analysis found that CETP inhibitors do not reduce CHD in RCTs [3]
·         Whole grain consumption and grain fibre is inversely associated with CHD in observational studies [4] [5] and certain types of grain fibre lower LDL-C [4], but DART (the only RCT to test the effect of grain fibre on CHD), found no benefit and instead found a near-significant increase in the risk of CHD events (1.23 (0.97-1.57)) and total mortality (1.27 (0.99-1.65)) (figures are after adjustment for confounders) [6].  In DART, grain fibre did not lower total-C [6]

And here are some other examples that aren’t related to the lipid hypothesis:

·         There are many proposed mechanisms suggesting that lower glycemic index carbohydrates are better for satiety, weight management, insulin resistance and blood lipids compared with higher glycemic index carbohydrates [7], but the evidence from observational studies and RCTs are mixed [8] [9]
·         A high intake of salt is suggested to be a major cause of high blood pressure as due to sodium increasing blood volume, and in ecological studies salt intake is associated with blood pressure, but a Cochrane meta-analysis of RCTs found reducing salt intake had only a very slight effect on blood pressure [10]
·         Carnitine increases TMAO and TMAO is associated with CHD [11], but carnitine supplementation reduces CHD events and total mortality [12] (post)
·         Elevated homocysteine is considered a risk factor for CVD [13], and there are a number of proposed mechanisms [14], but lowering homocysteine with vitamins B6, B9 and/or B12 doesn’t appear to reduce CVD [14]

Observational studies are non-randomised, and therefore there’s a much greater potential for there to be other differences between certain categories of people (for example: those who take a drug/supplement vs. don’t take a drug/supplement; or those who eat a diet low in SFA and high in PUFA vs. those who eat a diet high in SFA and low in PUFA***)****.  These other differences (known as confounding variables) can include other dietary and lifestyle differences, differences in healthcare, etc, may substantially affect the results (known as ‘the healthy user effect’) [15].  While many observational studies attempt to control for a number of confounding variables, this clearly doesn’t work perfectly all the time as evidenced by the examples above, as you can only control for the things you measure

The mechanism by which replacing SFA with PUFA is thought to reduce CHD is by decreasing the total-C:HDL-C ratio.  This proposed mechanism, known as the diet heart hypothesis, is a reasonable hypothesis, but shouldn’t be assumed to be true.  Asserting that replacing SFA with PUFA will reduce CHD based on their effects on cholesterol levels assumes that: replacing SFA with PUFA only affects cholesterol levels and/or that cholesterol levels are the only factor in CHD, a position I doubt anyone would take.  That being said, many people recommend reducing SFA or replacing it with PUFA based on their effects on cholesterol levels.  I don’t think they realise how reductionist those recommendations are, particularly when extended to food, because when discussing nutrients they are ignoring all the other biological effects the nutrient has related CHD, and when extending it food they are also ignoring the combined biological effect of all the other nutrients that is related to CHD*****.  As such, sometimes the mechanism may be correct, but it depends on how it’s targeted.  The salt, blood pressure and mortality connection is a good of example of this.  In meta-analyses of observational studies [16] and RCTs [17] (sometimes they are consistent) the relationship between salt intake and total mortality seems to follow a U-shaped curve, whereby a moderate salt intake is associated with lower mortality compared to low and high salt intakes.  So although reducing salt slightly decreases blood pressure, which would be expected to reduce mortality, it also substantially increases the renin-angiotensin-aldosterone system [10], which has a several very important functions in the body but elevated activation of this pathway has many adverse health effects [18], which is probably the main factor responsible for the higher risk of mortality associated with a lower intake of salt.

* People who decide to use total-C or LDL-C as risk their preferred risk factor (which would predict SFA would be positively associated with CHD) should realise that meta-analyses of observational studies have consistently found no association between SFA and CHD Jakobson, Skeaff, Mente, Siri-Tarino and Chowdhury.  Whereas those who use the total-C:HDL-C ratio or don’t make assumptions don’t experience any cognitive dissonance as SFA doesn’t increase the ratio.  (See below on how replacing carbohydrate with SFA, MUFA or PUFA effects different measures of cholesterol) [19]


** The results from Jakobson and Farvid suggested a benefit in replacing SFA with PUFA, whereas those from Skeaff, Mente and Chowdhury did not

*** Observational studies tend to group people in tertiles, quartiles or quintiles and then compare the highest vs. the lowest groups – in other words comparing the incidence of CHD among the top 20% consumers of SFA vs. the bottom 20% consumers of SFA

**** This can also be seen in inadequately randomised trials like Finnish Mental Hospital Study, which had a substantial difference in cardiotoxic medication use between the groups and hospital stay (blog), as well as many minor differences (in things like age, blood pressure, BMI, etc)

***** Or in other words, how to be a reductionist: focus on everything in blue and ignore everything in green.

****** As risk factors, total-C, LDL-C and the total-C:HDL-C ratio would all predict that replacing SFA with MUFA would decrease the risk of CHD.  Unfortunately, only the Rose Corn Oil Trial tested this, as it had an olive oil group that had a slightly higher incidence of CHD events and mortality (not significant) [20].  In meta-analyses of observational studies, Jakobson, Skeaff and Chowdhury found no association between MUFA and CHD, whereas Mente found an inverse association between MUFA and CHD


  1. This is spectacular, you really hit it out of the park. I do still think that moving forward, observational evidence and mechanism will have to do, there are only so many mortality RCTs you can do with the funding afforded to nutrition (maybe when they kill homeopathy for good nutrition will get some of that, I can only dream), but as you implied, we actually have to be rigorous about it and look at the whole breadth of effects in context. Another good one, high dose vitamin C reduces LDL and reduces blood pressure, but not CVD risk. If you ask 1990s deer-in-the-headlights researchers they would end up calling this a "paradox" but experiments have shown that high dose antioxidant supplementation can impair the adaptation to exercise, not something you want if your goal is to avoid CVDs

    So...I guess, be informed by RCTs whenever possible but expand upon knowledge of pathophysiology and the real-world effects of foods and nutrients?

    1. Thanks Stabby. I agree that we will often have to rely on mechanisms and observational studies (can't wait for an RCT on everything), and like you said, we should think more broadly and discuss the results (are there confounding variables/other mechanisms that may influence the end result). A greater understanding of physiology will help this, because in your vitamin C example it's fairly well understood now that ROS have many important functions.

      I see the main problem is when something becomes dogma, because then that discussion or broader thinking is replaced by 'we know this to be true' and if a study arrives at an opposite conclusion then it's judged to be flawed

  2. The evidence regarding LDL-C satisfies Koch’s postulates of causality, which is one reason why it gets so much focus. Do you think that focusing only on LDL particle size also is a reductionist approach?

    I agree that the diet as a whole is the real issue. But then why do you disregard the Oslo studies, Lyon diet-heart etc. for being "multi-factorial", when it's the whole diet that's relevant?

    1. I reread the post again and realised that I may have been unclear. I have no problem with the lipid hypothesis. So long as LDL-C (or total-C:HDL-C, etc) continue to be associated with CHD, then it is reasonable to hypothesise that lowering LDL-C would reduce the incidence of CHD. Especially since LDL-C is also supported by all the mechanistic research suggesting that LDL-P is a factor in atherogenesis (of which LDL-C seems to be a fairly accurate surrogate marker of)

      What I object is when people make assumptions, such as ‘that because X lowers LDL-C, therefore X lowers CHD’. That is the reductionism I was talking about because it ignores all the other effects on CHD that X may have that are independent of LDL-C (i.e. regarding CHD risk, X is reduced to its effect on LDL-C).

      Furthermore, with the focus on cholesterol and CHD, sometimes the effects on non-CHD morbidity and mortality are ignored. For example: if something reduced CHD events and CHD mortality by 20% but has no effect on total mortality it’s likely to be celebrated as a success, but is it really? This is quite relevant for the lipid hypothesis because while LDL-C is associated with CHD, it also has a number of beneficial functions. This could also be considered another form of reductionism: regarding the health effects of X, it is reduced to its effect on CHD

      In addition, the lipid hypothesis is the kind of hypothesis that needs to be tested for each new thing that lowers LDL-C (for example). If you found a LDL-C lowering drug that reduced CHD, you shouldn’t generalise that to each dietary factor that lowers LDL-C, or even to other drugs (and vice versa)

      The Oslo Diet Heart Study, Lyon Diet Heart Study and others are important because they show that diet can have remarkable effect on the incidence of CHD and total mortality. However, what I’m focusing on at the moment is whether replacing SFA with PUFA will reduce the incidence of CHD events, CHD mortality and total mortality. Since there were many differences between the experimental and control diets in Oslo, it simply was not designed to test the effects of replacing SFA with PUFA. Similarly, you shouldn’t look at a Paleo diet trial (for example) and conclude that the greater improvements in the Paleo group is because they were eating fewer grains, as there are too many other dietary differences to make that conclusion. The experimental group in Lyon Diet Heart Study had a lower intake of both SFA and PUFA compared to the control group, so it’s clearly inappropriate for testing the effects of replacing SFA with PUFA

      I hope this answered your question and wasn't too lengthy of a response

    2. I think there's a misunderstanding of Koch's postulates in the question; Koch (crudely) thought causal factors could be identified if they were always associated with disease - of course CHD afflicts people with low LDL as well as high LDL, this is contrary to Koch, except in that everyone has LDL, but that can't be helped. If there is a slight tendency for people with higher LDL to be at more risk, could this be caused by one of the factors raising LDL (in the people with high LDL and CHD), and not another (in the people with high LDL and no CHD)? Could it be caused by a factor that changes the nature of LDL as it raises it?
      Further - long term LDL is predictive of CHD (very poorly, HDL is much better, VLDL, BP, countless other better predictors).
      But in the very short term, low LDL predicts heart attacks, because it shows the low grade inflammation that precedes them (or so I'm told by people trying to explain heart attacks in people with low LDL at admission - case-control analysis).
      So why, when LDL is tested, are we meant to worry about our long term risk, and and not at all about any immediate danger being flagged?

    3. I can narrow this down a bit as to what I think is going on
      - LDL in atherogenesis is part of a system. This system involves macrophages behaving differently, vascular endothelial cells behaving differently, LDL behaving differently ("modified" LDL) and an excess of TG-rich VLDL remnants.
      It's not as simple as elevated LDL inducing these changes. If this system is operative, then you'd expect that raising LDL would make it worse, lowering LDL make it better, but IMPROVE-IT lowered LDL by an extra 24% without any change in CVD mortality, all-cause mortality, or heart attacks. And this was the study trumpeted as proving the "LDL principle".

    4. I agree George. There are a lot of moving parts to atherosclerosis, but unfortunately LDL-C/response to retention is just about the only thing that's mentioned. Oxidative modification led to antioxidant trials and subsequently a fair bit of skepticism of that mechanism when they failed. There has been much less attention on methods to alter immune response, endothelial function, etc

      In any event, the LDL-C is quite often pushed very low in secondary prevention or high risk people, and yet they still have a high incidence of heart attacks (etc). So obviously there must be factors that aren't being sufficiently addressed​