The presence of small-study effects is a common threat to systematic reviews and meta-analyses, especially when
it is due to publication bias, which occurs when small primary studies are more likely to be reported (published)
if their findings were positive. The presence of small-study effects can be verified by visual inspection of
the funnel plot, where for each included study of the meta-analysis, the estimate of the reported effect size is
depicted against a measure of precision or sample size.
The premise is that the scatter of plots should reflect a funnel shape, if small-study
effects do not exist. However, when small studies are predominately in one direction (usually the
direction of larger effect sizes), asymmetry will ensue.
The fat function implements several tests for detecting funnel plot asymmetry,
which can be used when the presence of between-study heterogeneity in treatment effect is relatively low.
Arguments
- b
Vector with the effect size of each study. Examples are log odds ratio, log hazards ratio, log relative risk.
- b.se
Optional vector with the standard error of the effect size of each study
- n.total
Optional vector with the total sample size of each study
- d.total
Optional vector with the total number of observed events for each study
- d1
Optional vector with the total number of observed events in the exposed groups
- d2
Optional vector with the total number of observed events in the unexposed groups
- method
Method for testing funnel plot asymmetry, defaults to
"E-FIV"(Egger's test with multiplicative dispersion). Other options areE-UW,M-FIV,M-FPV,D-FIVandD-FAV. More info in "Details"
Value
a list containing the following entries:
- "pval"
A two-sided P-value indicating statistical significance of the funnel plot asymettry test. Values below the significance level (usually defined as 10%) support the presence of funnel plot asymmetry, and thus small-study effects.
- "model"
A fitted
glmobject, representing the estimated regression model used for testing funnel plot asymmetry.
Details
Egger regression method
A common approach to test the presence of small-study effects is to
estimate a regression model where the standardized effect estimate
(effect/SE) is regressed on a measure of precision (1/SE),
(method="E-UW", Egger 1997).
It is possible to allow for between-study heterogeneity by adopting a
multiplicative overdispersion parameter by which the variance in each
study is multiplied (method="E-FIV", Sterne 2000).
Unfortunately, it has been demonstrated that the aforementioned two tests are biased because: (i) the independent variable is subject to sampling variability; (ii) the standardized treatment effect is correlated with its estimated precision; and (iii) for binary data, the independent regression variable is a biased estimate of the true precision, with larger bias for smaller sample sizes (Macaskill et al. 2001).
Macaskill regression method
The standard approach estimates a regression model with the effect size as
a function of the study size (method="M-FIV", Macaskill et al.
2001). Each study is weighted by the precision of the treatment effect
estimate to allow for possible heteroscedasticity.
An alternative approach is to weight each study by a pooled' estimate of the
outcome proportion (method="M-FPV")
For studies with zero events, a continuity correction is applied by adding 0.5 to all cell counts.
Peters regression method
This approach (method="P-FPV") estimates a regression model with the
treatment effect as a function of the inverse of the total sample size
(Peters et al. 2006).
For studies with zero events, a continuity correction is applied by adding 0.5 to all cell counts.
Debray regression method
This approach was proposed for survival data, and uses the total
number of events as independent variable in the weighted regression model
(Debray et al. 2017). The study weights are based on the inverse variance
(method="D-FIV") or on an approximation thereof
(method="D-FAV").
References
Debray TPA, Moons KGM, Riley RD. Detecting small-study effects and funnel plot asymmetry in meta-analysis of
survival data: a comparison of new and existing tests. Res Syn Meth. 2018;9(1):41--50.
Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test.
BMJ. 1997;315(7109):629--34.
Macaskill P, Walter SD, Irwig L. A comparison of methods to detect publication bias in meta-analysis.
Stat Med. 2001;20(4):641--54.
Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L. Comparison of two methods to detect publication bias
in meta-analysis. JAMA. 2006 Feb 8;295(6):676--80.
Sterne JA, Gavaghan D, Egger M. Publication and related bias in meta-analysis: power of statistical tests
and prevalence in the literature. J Clin Epidemiol. 2000;53(11):1119--29.
Examples
data(Fibrinogen)
b <- log(Fibrinogen$HR)
b.se <- ((log(Fibrinogen$HR.975) - log(Fibrinogen$HR.025))/(2*qnorm(0.975)))
n.total <- Fibrinogen$N.total
d.total <- Fibrinogen$N.events
fat(b=b, b.se=b.se)
#> Call: fat(b = b, b.se = b.se)
#>
#> Fixed effect summary estimate: 0.4186
#>
#> test for funnel plot asymmetry: t =1.9021, df = 29, p = 0.0671
fat(b=b, b.se=b.se, d.total=d.total, method="D-FIV")
#> Call: fat(b = b, b.se = b.se, d.total = d.total, method = "D-FIV")
#>
#> Fixed effect summary estimate: 0.4186
#>
#> test for funnel plot asymmetry: t =1.6847, df = 29, p = 0.1028
# Note that many tests are also available via metafor
require(metafor)
#> Loading required package: metafor
#> Loading required package: Matrix
#> Loading required package: metadat
#> Loading required package: numDeriv
#>
#> Loading the 'metafor' package (version 4.4-0). For an
#> introduction to the package please type: help(metafor)
#>
#> Attaching package: ‘metafor’
#> The following object is masked from ‘package:metamisc’:
#>
#> forest
fat(b=b, b.se=b.se, n.total=n.total, method="M-FIV")
#> Call: fat(b = b, b.se = b.se, n.total = n.total, method = "M-FIV")
#>
#> Fixed effect summary estimate: 0.4186
#>
#> test for funnel plot asymmetry: t =-1.4275, df = 29, p = 0.1641
regtest(x=b, sei=b.se, ni=n.total, model="lm", predictor="ni")
#>
#> Regression Test for Funnel Plot Asymmetry
#>
#> Model: weighted regression with multiplicative dispersion
#> Predictor: sample size
#>
#> Test for Funnel Plot Asymmetry: t = -1.4275, df = 29, p = 0.1641
#>