Tuesday, March 31, 2015

Emphasising 'Randomised' and 'Controlled' in a Meta-Analysis of Randomised Controlled Trials regarding Saturated Fat and Coronary Heart Disease

A key feature of randomised controlled trials is that the groups in the trial are treated identically except for the experimental treatment.  With dietary interventions this is near impossible, although at the very least, clinical trials with a multifactorial diet intervention such as ODHS and STARS should not be considered adequately controlled for the purposes of drawing conclusions regarding the effects of replacing SFA with PUFA.  Arguably none of the trials should be considered well controlled for this purpose due to dietary advice to reduce TFA which only given to the experimental group.  That being said, it is still worthwhile to pool the results of these trials together and categorise them based on how well controlled they were, while acknowledging the issue of unequal advice to reduce TFA intake.

In this post the trials are categorised as ‘adequately controlled’ or ‘inadequately controlled’.   Clinical trials that are categorised as ‘adequately controlled’ are those that most closely approximate a true test of replacing SFA with PUFA, while the clinical trials categorised as ‘inadequately controlled’ have too many dietary and non-dietary differences between the groups to be considered close to a valid test of replacing SFA with PUFA.  Due to the uncertainty regarding TFA intake in SDHS, I’ll have a separate set of results that excludes it.

Below is a summary of the dietary and non-dietary differences between the groups, whether these differences are favourable or unfavourable to the experimental group and whether the clinical trial is assessed as adequately controlled


Dietary and non-dietary differences between the groups
Direction
Adequately Controlled
RCOT
Advice to reduce TFA in experimental group
Favourable
Yes
LAVAT
Higher withdrawal rate in experimental group
Lower and insufficient α-tocopherol intake in control group
Higher TFA intake in control group
Favourable
Favourable

Favourable
No
MRCT
Advice to reduce TFA in experimental group
Favourable
Yes
ODHS
Highly multifactorial diet intervention
Very Favourable
No
SDHS
Advice to reduce TFA in experimental group
High intake of high TFA margarine in experimental group
Favourable
Unfavourable
Unknown
FMHS
Higher intake TFA intake in control group
Higher use of cardiotoxic medication in control group
Favourable
Favourable
No
MCS
Advice to reduce TFA in experimental group
Favourable
Yes
DART
Modestly multifactorial diet intervention
Advice to reduce TFA in experimental group
Slightly Favourable
Favourable
Yes
STARS
Highly multifactorial diet intervention
Very Favourable
No

A few things to note:

  • The results in this post came from the Review Manager V.5.1 software (RevMan), provided by the Cochrane Collaboration
  • E = number of CHD events (multiple events in one participant is counted each time)
  • P = number of participants who have had CHD events (multiple events in one participant is counted once)
  • I included data on CHD mortality from SDHS as CHD events (after all, a death from CHD is a CHD event)
  • For FMHS and MCS I’m using person years to calculate the RR and I’m using the same approach I discussed previously to enter the data from those trials into RevMan
  • I’ve only included the basic 6 forest plots in this blog post.  The ones excluding SDHS and FMHS are in a separate powerpoint

 Major CHD Events (E)

Major CHD Events (P)
 

Total CHD Events (E)

Total CHD Events (P)

CHD Mortality
 

Total Mortality

For a summary of the results see the table below.  Simply pooling the trials together suggests a ~10% reduction in CHD events and CHD mortality which doesn’t reach significance, with no effect on total mortality.  However, differentiating the trials based on whether they are adequately controlled or not tells a different story.  Pooling the results from the trials that are considered adequately controlled results in an RR that is consistently very slightly above 1.0 (not significant) and excluding SDHS from this category lowers the RR to approximately 1.0.  Meanwhile the pooling the results of the inadequately controlled trials results in a highly significant reduction in CHD events and CHD mortality of about 30%, but total mortality isn’t affected, even in this category, which is quite surprising.  Altogether this suggests that it’s extremely unlikely that replacing SFA with PUFA was responsible for the reduction in CHD in the inadequately controlled trials and that it was most likely due to the other differences that are summarised in the table above.

I didn’t do a separate analysis for both groups combined when excluding SDHS because that represents a very biased interpretation of these trials, one that criticises and excludes SDHS based on potential differences in TFA intake, while ignoring all the cases of higher TFA intake in the control group and other differences between the groups.  Unfortunately this is a common approach among those who promote conventional dietary advice.  A comment by Zahc sums up this attitude well “It seems that you are assuming that fat modification is beneficial, and therefore negative results must mean that the trial is flawed”


Adequately Controlled Trials
Inadequately Controlled Trials
Total
Adequately Controlled Trials – SDHS
Major CHD Events (E)
1.07 (0.87-1.32)
P = 0.53
0.68 (0.52-0.88)
P = 0.004
0.90 (0.74-1.10)
P = 0.31
0.99 (0.82-1.20)
P = 0.96
Major CHD Events (P)
1.07 (0.87-1.32)
P = 0.53
0.67 (0.49-0.92)
P = 0.01
0.92 (0.75-1.11)
P = 0.38
0.99 (0.82-1.20)
P = 0.96
Total CHD Events (E)
1.03 (0.85-1.25)
P = 0.75
0.69 (0.58-0.82)
P < 0.0001
0.86 (0.73-1.03)
P = 0.10
0.96 (0.81-1.13)
P = 0.62
Total CHD Events (P)
1.03 (0.85-1.25)
P = 0.75
0.71 (0.59-0.86)
P = 0.0004
0.88 (0.74-1.04)
P = 0.12
0.96 (0.81-1.13)
P = 0.62
CHD
Mortality
1.11 (0.92-1.33)
P = 0.28
0.66 (0.54-0.80)
P < 0.0001
0.88 (0.70-1.10)
P = 0.25
1.04 (0.85-1.26)
P = 0.72
Total
Mortality
1.06 (0.92-1.23)
P = 0.41
0.96 (0.85-1.08)
P = 0.46
1.00 (0.91-1.09)
P = 0.96
1.02 (0.90-1.17)
P = 0.72

Next, I wanted to see the effect of only including adequately randomised trials, thereby excluding FMHS.  I mentioned previously that FMHS strongly influences the result for CHD mortality, being both a large study and quite an outlier, and that excluding FMHS removes the favourable result for CHD mortality.  However, I underestimated its effect on CHD events as removing FMHS increased the RR by ~0.05-0.06 and removed any hint of significance for total CHD events 


Inadequately Controlled Trials
Inadequately Controlled Trials – FMHS
Total
Total - FMHS
Major CHD Events (E)
0.68 (0.52-0.88)
P = 0.004
0.76 (0.63-0.92)
P = 0.005
0.90 (0.74-1.10)
P = 0.31
0.96 (0.80-1.14)
P = 0.61
Major CHD Events (P)
0.67 (0.49-0.92)
P = 0.01
0.79 (0.63-0.98)
P = 0.03
0.92 (0.75-1.11)
P = 0.38
0.97 (0.82-1.15)
P = 0.72
Total CHD Events (E)
0.69 (0.58-0.82)
P < 0.0001
0.72 (0.56-0.92)
P = 0.009
0.86 (0.73-1.03)
P = 0.10
0.92 (0.77-1.11)
P = 0.38
Total CHD Events (P)
0.71 (0.59-0.86)
P = 0.0004
0.76 (0.59-0.98)
P = 0.03
0.88 (0.74-1.04)
P = 0.12
0.94 (0.79-1.11)
P = 0.45
CHD
Mortality
0.66 (0.54-0.80)
P < 0.0001
0.77 (0.59-1.00)
P = 0.05
0.88 (0.70-1.10)
P = 0.25
0.99 (0.83-1.20)
P = 0.95
Total
Mortality
0.96 (0.85-1.08)
P = 0.46
0.89 (0.71-1.12)
P = 0.33
1.00 (0.91-1.09)
P = 0.96
1.00 (0.89-1.13)
P = 0.96

Monday, March 23, 2015

The Importance of Using Person Years to Calculate Relative Risk

One point I stressed a fair bit in the previous two posts is the importance of calculating the RR using person years rather than group size where appropriate*.  This was a major issue in FMHS, as the RR derived from age-adjusted person years was substantially different to the RR derived from group size.  While this should make little difference in large adequately randomised (or stratified) trials such as MCS, there were still many differences of ~4-5%.

FMHS (Men)
RR Using Group Size
RR Using Age-Adjusted Person Years
Relative Difference to RR Using Person Years
Major CHD Events
0.331
0.330
+0.3%
Total CHD Events
0.548
0.557
-1.6%
CHD Mortality
0.374
0.469
-20.3%
CVD Mortality
0.498
0.608
-18.1%
Total Mortality
0.724
0.882
-17.9%

FMHS (Women)
RR Using Group Size
RR Using Age-Adjusted Person Years
Relative Difference to RR Using Person Years
Major CHD Events
0.334
0.393
-15.0%
Total CHD Events
0.539
0.635
-15.1%
CHD Mortality
0.446
0.659
-32.3%
CVD Mortality
0.603
0.859
-29.8%
Total Mortality
0.703
1.064
-33.9%

MCS (Men)
RR Using Group Size
RR Using Person Years
Relative Difference to RR Using Person Years
Major CHD Events
0.932
0.895
+4.1%
CHD Mortality
1.147
1.102
+4.1%
CVD Mortality
0.978
0.940
+4.0%
Total Mortality
1.032
0.992
+4.0%

MCS (Women)
RR Using Group Size
RR Using Person Years
Relative Difference to RR Using Person Years
Major CHD Events
1.306
1.317
-0.8%
CHD Mortality
1.037
1.097
-5.5%
CVD Mortality
1.021
1.029
-0.8%
Total Mortality
1.156
1.164
-0.7%

To look at how this would affect the overall result in a meta-analysis I put two sets of data into the Review Manager V.5.1 software (RevMan), provided by the Cochrane Collaboration, one using person years for MCS and FMHS and one using group size

Unfortunately the RR in RevMan is calculated automatically using the number of events and group size for each group and can’t be changed.  One option I had would be to enter the number of person years (age-adjusted for FMHS) instead of the group size.  However, as the weighting for each trial is determined both by the number of events/deaths and the group size, that method would affect the weighting.  So I used some basic algebra to keep the total number of participants constant but changed the number of participants in each group so RevMan could automatically calculate the correct RR

The equation for the RR using person years is (E = events/death, and PY = person years):
RR = (Eexp / PYexp) / (Econ / PYcon)

To not affect the weighting the total number of person year needs to equal the total number of participants (N):
PYexp + PYcon = Nexp + Ncon

Therefore:
PYexp = (Nexp + Ncon) / (1 + (RR x Eexp / Econ))
PYcon = (Nexp + Ncon) / (1 + (Eexp / (Econ x RR)))

For ‘CHD events’ I’m using the number of major CHD events

CHD Events (Group Size)

CHD Events (Person Years)

CHD Mortality (Group Size)

CHD Mortality (Person Years)

Total Mortality (Group Size)

Total Mortality (Person Years)


RR Using Group Size
RR Using Person Years
CHD Events
0.90 (0.74-1.10)
0.90 (0.74-1.10)
CHD Mortality
0.83 (0.61-1.13)
0.88 (0.70-1.10)
Total Mortality
0.92 (0.79-1.07)
1.00 (0.91-1.09)

FMHS was the largest study and is quite an outlier regarding its results for CHD events and CHD mortality.  The inclusion/exclusion and the figures used for FMHS explains much of the difference between the meta-analyses by Hooper, et al, Mozaffarian, et al and Skeaff & Miller


CHD Events
CHD Mortality
Total Mortality
0.83 (0.69-1.00)
P = 0.073
0.84 (0.62-1.12)
P = 0.335
0.88 (0.76-1.02)
P = 0.005
0.81 (0.70-0.95)
P = 0.008
0.80 (0.65-0.98)
P = ???
0.98 (0.89-1.08)
P = ???
(Modified Fat)
0.82 (0.66-1.02)
P = 0.073
0.92 (0.73-1.15)
P = 0.46
1.02 (0.88-1.18)
P = 0.81
(Modified + Reduced Fat)
0.77 (0.57-1.03)
P = 0.077
0.98 (0.76-1.27)
P = 0.88
0.97 (0.76-1.23)
P = 0.78

The main differences between Hooper and Mozaffarian was that Mozaffarian included FMHS (RCOT is too small to count for much and Hooper didn’t have data for CHD mortality in SDHS).  Mozaffarian found a (likely) significant reduction in CHD mortality whereas Hooper didn’t and both found no difference in total mortality.  This because FMHS is a large trial and got a very favourable result for CHD mortality (even using person years), but found little difference in total mortality.  Simply excluding FMHS (if you wanted to only include adequately randomised trials (what a novel idea!)) single-handedly removes the favourable result for CHD mortality, changing the RR to 0.99 (0.83-1.20)


Despite the included trials being almost identical to Mozaffarian, Skeaff found no significant difference in CHD mortality and was the only meta-analysis to find a significant reduction in total mortality.  The lack of significance for CHD mortality was probably due to them using incorrect and lower values for CHD mortality from MRCT, ODHS and FMHS and not including CHD mortality from LAVAT and MCS.  With less than 300 CHD deaths and 57.17% of weighting going to DART (N = 194, RR = 1.00) it’s hardly surprising that their RR of 0.84 was so underpowered.  Regarding total mortality, Skeaff and Mozaffarian included exactly the same trials and had almost exactly the same numbers, but Mozaffarian got an RR of 0.98 and Skeaff got an RR of 0.88 (significant).  The reason why is that Mozaffarian calculated the RR using person years, whereas Skeaff did not

* The RR in LAVAT should also have been calculated using person years due to withdrawals, discharges and readmissions.  Fortunately discharges and readmissions were pretty similar between the groups, so those are unlikely to substantially affect the results.  However, there were twice as many withdrawals in the experimental group, and a rough adjustment for this would increase the RR by 1.147 (the figure is from the monograph)