Tuesday, March 31, 2015

Emphasising 'Randomised' and 'Controlled' in a Meta-Analysis of Randomised Controlled Trials regarding Saturated Fat and Coronary Heart Disease

A key feature of randomised controlled trials is that the groups in the trial are treated identically except for the experimental treatment.  With dietary interventions this is near impossible, although at the very least, clinical trials with a multifactorial diet intervention such as ODHS and STARS should not be considered adequately controlled for the purposes of drawing conclusions regarding the effects of replacing SFA with PUFA.  Arguably none of the trials should be considered well controlled for this purpose due to dietary advice to reduce TFA which only given to the experimental group.  That being said, it is still worthwhile to pool the results of these trials together and categorise them based on how well controlled they were, while acknowledging the issue of unequal advice to reduce TFA intake.

In this post the trials are categorised as ‘adequately controlled’ or ‘inadequately controlled’.   Clinical trials that are categorised as ‘adequately controlled’ are those that most closely approximate a true test of replacing SFA with PUFA, while the clinical trials categorised as ‘inadequately controlled’ have too many dietary and non-dietary differences between the groups to be considered close to a valid test of replacing SFA with PUFA.  Due to the uncertainty regarding TFA intake in SDHS, I’ll have a separate set of results that excludes it.

Below is a summary of the dietary and non-dietary differences between the groups, whether these differences are favourable or unfavourable to the experimental group and whether the clinical trial is assessed as adequately controlled


Dietary and non-dietary differences between the groups
Direction
Adequately Controlled
RCOT
Advice to reduce TFA in experimental group
Favourable
Yes
LAVAT
Higher TFA intake in control group
Lower and insufficient α-tocopherol intake in control group
Favourable
Favourable
No
MRCT
Advice to reduce TFA in experimental group
Favourable
Yes
ODHS
Highly multifactorial diet intervention
Very Favourable
No
SDHS
Advice to reduce TFA in experimental group
High intake of high TFA margarine in experimental group
Favourable
Unfavourable
Unknown
FMHS
Higher intake TFA intake in control group
Higher use of cardiotoxic medication in control group
Favourable
Favourable
No
MCS
Advice to reduce TFA in experimental group
Favourable
Yes
DART
Modestly multifactorial diet intervention
Advice to reduce TFA in experimental group
Slightly Favourable
Favourable
Yes
STARS
Highly multifactorial diet intervention
Very Favourable
No

A few things to note:

  • The results in this post came from the Review Manager V.5.1 software (RevMan), provided by the Cochrane Collaboration
  • E = number of CHD events (multiple events in one participant is counted each time)
  • P = number of participants who have had CHD events (multiple events in one participant is counted once)
  • I included data on CHD mortality from SDHS as CHD events (after all, a death from CHD is a CHD event)
  • For FMHS and MCS I’m using person years to calculate the RR and I’m using the same approach I discussed previously to enter the data from those trials into RevMan
  • I’ve only included the basic 6 forest plots in this blog post.  The ones excluding SDHS and FMHS are in a separate powerpoint

 Major CHD Events (E)

Major CHD Events (P)
 

Total CHD Events (E)

Total CHD Events (P)

CHD Mortality
 

Total Mortality

For a summary of the results see the table below.  Simply pooling the trials together suggests a ~10% reduction in CHD events and CHD mortality which doesn’t reach significance, with no effect on total mortality.  However, differentiating the trials based on whether they are adequately controlled or not tells a different story.  Pooling the results from the trials that are considered adequately controlled results in an RR that is consistently very slightly above 1.0 (not significant) and excluding SDHS from this category lowers the RR to approximately 1.0.  Meanwhile the pooling the results of the inadequately controlled trials results in a highly significant reduction in CHD events and CHD mortality of about 30%, but total mortality isn’t affected, even in this category, which is quite surprising.  Altogether this suggests that it’s extremely unlikely that replacing SFA with PUFA was responsible for the reduction in CHD in the inadequately controlled trials and that it was most likely due to the other differences that are summarised in the table above.

I didn’t do a separate analysis for both groups combined when excluding SDHS because that represents a very biased interpretation of these trials, one that criticises and excludes SDHS based on potential differences in TFA intake, while ignoring all the cases of higher TFA intake in the control group and other differences between the groups.  Unfortunately this is a common approach among those who promote conventional dietary advice.  A comment by Zahc sums up this attitude well “It seems that you are assuming that fat modification is beneficial, and therefore negative results must mean that the trial is flawed”


Adequately Controlled Trials
Inadequately Controlled Trials
Total
Adequately Controlled Trials – SDHS
Major CHD Events (E)
1.07 (0.87-1.32)
P = 0.53
0.68 (0.52-0.88)
P = 0.004
0.90 (0.74-1.10)
P = 0.31
0.99 (0.82-1.20)
P = 0.96
Major CHD Events (P)
1.07 (0.87-1.32)
P = 0.53
0.67 (0.49-0.92)
P = 0.01
0.92 (0.75-1.11)
P = 0.38
0.99 (0.82-1.20)
P = 0.96
Total CHD Events (E)
1.03 (0.85-1.25)
P = 0.75
0.69 (0.58-0.82)
P < 0.0001
0.86 (0.73-1.03)
P = 0.10
0.96 (0.81-1.13)
P = 0.62
Total CHD Events (P)
1.03 (0.85-1.25)
P = 0.75
0.71 (0.59-0.86)
P = 0.0004
0.88 (0.74-1.04)
P = 0.12
0.96 (0.81-1.13)
P = 0.62
CHD
Mortality
1.11 (0.92-1.33)
P = 0.28
0.66 (0.54-0.80)
P < 0.0001
0.88 (0.70-1.10)
P = 0.25
1.04 (0.85-1.26)
P = 0.72
Total
Mortality
1.06 (0.92-1.23)
P = 0.41
0.96 (0.85-1.08)
P = 0.46
1.00 (0.91-1.09)
P = 0.96
1.02 (0.90-1.17)
P = 0.72

Next, I wanted to see the effect of only including adequately randomised trials, thereby excluding FMHS.  I mentioned previously that FMHS strongly influences the result for CHD mortality, being both a large study and quite an outlier, and that excluding FMHS removes the favourable result for CHD mortality.  However, I underestimated its effect on CHD events as removing FMHS increased the RR by ~0.05-0.06 and removed any hint of significance for total CHD events 


Inadequately Controlled Trials
Inadequately Controlled Trials – FMHS
Total
Total - FMHS
Major CHD Events (E)
0.68 (0.52-0.88)
P = 0.004
0.76 (0.63-0.92)
P = 0.005
0.90 (0.74-1.10)
P = 0.31
0.96 (0.80-1.14)
P = 0.61
Major CHD Events (P)
0.67 (0.49-0.92)
P = 0.01
0.79 (0.63-0.98)
P = 0.03
0.92 (0.75-1.11)
P = 0.38
0.97 (0.82-1.15)
P = 0.72
Total CHD Events (E)
0.69 (0.58-0.82)
P < 0.0001
0.72 (0.56-0.92)
P = 0.009
0.86 (0.73-1.03)
P = 0.10
0.92 (0.77-1.11)
P = 0.38
Total CHD Events (P)
0.71 (0.59-0.86)
P = 0.0004
0.76 (0.59-0.98)
P = 0.03
0.88 (0.74-1.04)
P = 0.12
0.94 (0.79-1.11)
P = 0.45
CHD
Mortality
0.66 (0.54-0.80)
P < 0.0001
0.77 (0.59-1.00)
P = 0.05
0.88 (0.70-1.10)
P = 0.25
0.99 (0.83-1.20)
P = 0.95
Total
Mortality
0.96 (0.85-1.08)
P = 0.46
0.89 (0.71-1.12)
P = 0.33
1.00 (0.91-1.09)
P = 0.96
1.00 (0.89-1.13)
P = 0.96

4 comments:

  1. Bravo Steve.

    In the immortal words of Skeaff and Miller,

    " In this regard, the meta-analysis of trials in which serum
    cholesterol concentrations in the high P/S diet group
    were significantly lower than in the control group, revealed
    that a diet higher in PUFA and lower in SFA decreased
    the risk of fatal CHD; however, this was significant
    only after inclusion of results from the Oslo study
    which included smoking cessation as part of the treatment.
    The cholesterol-lowering effect of the high P/S diet
    is driven largely by the reduction in SFA intake as shown
    in the metabolic ward studies [Clarke et al., 1997]. The
    evidence from metabolic ward studies clearly shows that
    diets low in SFA reduce total cholesterol and should therefore
    reduce the risk of CHD. However, the meta-analysis
    of results from cohort studies – albeit from a limited
    number of studies – showed no association between SFA
    intake and CHD, demonstrating their unreliability.
    The observational evidence for an association between
    dietary PUFA and CHD risk is inconsistent and is unreliable."

    Against such faith, the gods themselves will struggle in vain.

    ReplyDelete
    Replies
    1. Thanks, and I agree. What's the point of doing research if you 'know' the answer regardless of the result?

      I don't remember ODHS having "smoking cessation as part of the treatment". They might be referring to another study in Oslo
      http://www.ncbi.nlm.nih.gov/pubmed/3903733

      I find it quite funny that if Skeaff & Miller included CHD mortality in LAVAT and used the correct figures for CHD mortality in ODHS and FMHS they may have got the significant result they were after

      Delete
  2. Brilliant again Steven! It's nice to have all this information in one place, and thanks for clarifying the discrepant figures seen in different analyses, it has really been helpful. I have to say, this website should be a lot more popular!

    ReplyDelete