Indian Journal of Medical Microbiology  Home 
Metaanalysis in microbiology Correspondence Address: Abstract
Full Text Introduction Exponential growth of research outputs makes it increasingly difficult to discern usable knowledge in a flood of information. It is not surprising that primary studies addressing the same issue sometimes produce contradictory results. Conflicting findings are not easy to reconcile and in some cases, an abundance of primary studies often hinders meaningful integration of results using traditional narrative methods. Clearly, it is essential to have some means of systematically comparing and contrasting the results among relevant studies. Used extensively to compile results of individual studies assessing risk factors in epidemiological studies and effectiveness of healthcare interventions, metaanalysis is useful in decisionmaking that directs public health policy. Historical application of metaanalysis in public health and microbiology may not be as extensive as in evidencebased medicine. Yet, metaanalysis has addressed key research questions in microbiology such as prevalence of pathogens [1],[2] and prevalence of infectious disease. [3] This review is directed at laboratory investigators and clinicians for a better understanding of metaanalysis research in microbiology. For purposes of clarifying our points in explaining this research methodology, we use an example from a recent paper that addresses the relationship between Enteroaggregative Escherichia coli (EAEC) colonisation and acute diarrhoea in SouthAsian children. [4] What is MetaAnalysis? Metaanalysis is a logically formal and objective technique as well as quantitative mode of summarising research findings, helping to identify genuine associations. Considered at the top of the hierarchy of evidence, [5] this statistical methodology integrates results of independent but related studies to synthesise summaries. Why do MetaAnalysis? The reasons for performing a metaanalysis have to do with sample sizes of the studies, when they are large but results conflict, or when they are small, but their positive findings are not consistent. [6] Primary studies often do not have enough statistical power to assess relationships between risks (interventions) and outcomes. Being most useful when individual studies are too small to yield valid conclusions, metaanalysis increases power, reduces risk of error and facilitates exploratory analysis to generate hypotheses for future research. [7] Performing MetaAnalysis Literature search and data abstraction A publishable metaanalysis starts with a wellformulated and answerable question considering the time, cost and available resources. In microbiology studies, this includes availability of a good number of primary studies that address associations of key risk factors (e.g. pathogens, genetic susceptibility) with infectious disease. The published Preferred Reporting Items for Systematic Reviews and MetaAnalyses (PRISMA) statement recommends that a full electronic search strategy for at least one major database be presented, [8] although such an approach had been deemed insufficient by some. [9] Still a typical search strategy should involve electronic retrieval of all available literature, which includes digital sources such as PubMed (http://www.ncbi.nlm.nih.gov/pubmed/), ScienceDirect (www.sciencedirect.com), Institute of Scientific Information (ISI) Web of Knowledge (http://www.isiwebofknowledge.com) and Google Scholar (http://scholar.google.com). For greater precision in this step, additional measures to exhaustively identify eligible studies include manual searching of relevant journals, references lists and personal contact with researchers. [Figure 1] is a representative flowdiagram depicting the steps taken in searching for relevant literature using key words such as "diarrhoea children Asia" and "Escherichia coli0". It maps out the number of records identified, included and excluded, and the reasons for exclusions. [Table 1] shows the next step which is abstraction of both qualitative (e.g. last name of first author) and quantitative (e.g. sample sizes and statistical power) data from the collection of eligible studies. [4]{Figure 1} {Table 1} Summary effects calculations Metaanalysis reports findings in terms of effect sizes, which provides information about how much change is evident across all studies and for subsets of studies. In the metaanalyses example, results of each study are graphically presented in a forest plot [Figure 2] which uses the odds ratio (OR) and 95% confidence interval (CI) metric. The OR has convenient mathematical properties, which allow for ease in combining data and testing the overall effect for significance. [10] The forest plot [Figure 2] in this example, generated from the metaanalysis software, Review Manager (RevMan) is composed of five columns, the leftmost with qualitative data (study, named by last name of first author) and the remaining four with quantitative data. Between columns three and four is the forest plot with a solid vertical line (labelled 1 on the xaxis) which corresponds to the null effect. The area to the left of the vertical line indicates decreased risk and to its right, increased risk. The two leftmost columns show the raw data (cases/controls) from which the OR and 95% CIs (rightmost column) are calculated. In this figure, ORs of the 18 studies are each represented by a blue square and a black horizontal line, representing the point estimate and 95% CIs, respectively. Of the 18 studies, six do not cross the vertical line, two [11],[12] of which are in the decreased risk area and the remaining four [13],[14],[15],[16] in the increased risk area. The 95% CIs of all these studies do not cross 1.0 (null effect) indicating significant effects (P ≤ 0.05) and those of the remaining 12 studies cross the vertical line indicating that the effect estimates were nonsignificant (P > 0.05). Weight of the study in the metaanalysis is indicated in column four of [Figure 2] presented as proportion of the study that contributes to the pooled OR. The diamond (♦) at the bottom of the forest plot represents the pooled summary effect (OR 1.51, 95% CI 1.122.04) which shows that EAEC is significantly associated with acute diarrhoea in Asian children (P = 0.008).{Figure 2} Modifier analyses A summary or pooled effect needs to be tested further to ensure rigor of this methodology. In the metaanalysis example, three tests were undertaken; first, subgroup analysis used categories such as geography, span as well as power of the studies and the assays used for case definition [Table 1]. Findings showed that all subgroups had more imprecise ORs compared with the overall pooled OR and that only the statistically powerful studies (>80%) and those that used the "gold standard" HEp2 assay reflected the significant overall effect [Figure 3]a]. Second, using the Galbraith plot method, [17] five outliers were detected, showing three studies above [13],[16],[18] and two below [11],[12] the confidence limits [Figure 4]. The outcome of outlier analysis increased the number of significant ORs, narrowed all CIs, in the overall and subgroups conferring greater precision on the pooled ORs [Figure 3]b].{Figure 3}{Figure 4} SigmaPlot compared with RevMan [Figure 3] [Figure 4], [Figure 5] [Figure 6] in this review were generated from SigmaPlot 11.0 which displays the versatility of this software in graphically expressing metaanalysis results. SigmaPlot has a number of advantages over RevMan. (i) In the forest plots, [Figure 3]a and b] SigmaPlot allows control of graphically weighting the point estimates according to sample size (larger squares indic ate larger sample size). (ii) Furthermore, the squares can be filled or not. Filled black squares (■) indicates significance whose CIs do not cross the null effects line, while CIs that do are indicated by unfilled white squares (□) that represent nonsignificance. (iii) In a simple interface with Microsoft Excel, SigmaPlot provides a graphical summary of heterogeneity [Figure 3].{Figure 5}{Figure 6} Biases in MetaAnalysis Heterogeneity Heterogeneity is the methodological, epidemiological and clinical dissimilarity across various studies, and metaanalysts spend considerable effort in addressing this issue. When component studies in a metaanalysis are similar to each other, the fixedeffects method of analysis [19] is applied based on the assumption that associations are the same across studies and recognising that the collection of eligible literature is not heterogeneous. When they are not, the random effects analysis model is used, [20] which assumes variability across populations usually resulting in a wider CI. [10] Statistically, heterogeneity is estimated using a Chisquarebased Q test [21] and quantified with the I 2 metric which shows what proportion of the total variation across studies is beyond chance. [22] Values of I2 lie between 0 and 100% where a value >75% may be considered substantial heterogeneity. [22] In [Figure 2], the Chisquare value (92.1, P < 0.0001) and I 2 value of 82% indicate high heterogeneity. Graphically, the Galbraith plot is used to detect which component studies (outliers) contribute to heterogeneity. [17] Exclusion of these outliers either reduced or removed heterogeneity of the original findings [Figure 3]a] and is summarised in [Figure 3]b. In [Figure 3]A and 3B, red boxes indicate presence of heterogeneity and green, its absence and show either loss or reduction of heterogeneity with application of outlier analysis. Publication bias Publication bias [23] is an issue where significant findings receive priority in published literature over those whose results are nonsignificant. This bias is evaluated graphically with the funnel plot [Figure 5]. Here, the effect estimate from each study in the metaanalysis is scattered against a measure of its precision, usually 1/SE (standard error). This figure shows a symmetrical distribution of the points with small studies scattered along the length of the xaxis but still centred on the OR estimates from large, more precise studies. This is consistent with absence of bias. To confirm such absence, Egger's regression asymmetry [10] and Begg's and Mazumdar's rank correlation [24] tests were applied. In the metaanalysis example, these tests generated P values of 0.20 and 0.22, respectively, [4] indicating that conclusions were not altered because of this issue. For comparison, [Figure 6] shows a simulated funnel plot indicating presence of publication bias which shows an asymmetrical distribution of the points. Value of MetaAnalysis in Microbiology Interest in the role of pathogens in infectious disease has grown over the past decade. Rapid advancements in identifying key pathogens using molecular techniques have resulted in large amounts of published epidemiological evidence on infectiondisease associations. In the metaanalysis example, one aspect of subgroup treatment included the evolving use of assays in identifying EAEC over a period of 22 years (19892011). Thus, in [Table 1], we see a shift from cellbased (HEp2HeLa) to molecularbased (pCVD432) assays. [4] Metaanalysis is an attractive and cheaper alternative to the primary study, which when large, is bound to be expensive and logistically problematic. Done rigorously, metaanalysis effects merit higher confidence and greater statistical precision. Its findings can unmask largescale patterns not obvious in primary studies. Metaanalysis can then facilitate critical transfer of knowledge from researcher to clinician enabling analyses of important patient subgroups, delineation of high risk factors for infection enough for information to be useful for public health advice in risk for infection. Consequently, metaanalysis lends rigor to better assist health authorities in directing therapeutic decisions to target populations, urgency for health education and control measures. Indeed, in the public health domain, a number of difficult issues that had been repeatedly studied were either resolved or clarified by the application of metaanalysis techniques. [25] This has led some government guidelines to recommend metaanalysis as the preferred method of summarising evidence of effectiveness and safety of health technologies in the face of multiple study results. [26] Survey of MetaAnalyses Publications in Microbiology To obtain a sense and estimate extent of the use of metaanalysis in microbiology, we surveyed PubMed literature (April 11, 2013) using the search terms, "metaanalysis" in the title box and "microbiology" in the all fields box. This resulted in 655 citations, 48 of which were excluded because they were not metaanalyses. We then categorised the remaining 607 according to country and three periods (19921999, 20002006 and 20072013). We counted the number of published metaanalyses from two, 13 and 11 countries grouped under North America, Europe and Asia, respectively and divided the totals by the number of years in each period, producing a metaanalysis output per year value. [Figure 7] shows that use of metaanalysis research has grown dramatically in the last 6 years with Europe slightly ahead of the other two continents. The remarkable aspect of this growth is that of Asia, from a miniscule output in the 90s to a level comparable with that of North America 15 years later. The Asian and North American 20072013 values were attributed to China and the USA, respectively, both pegged at 77%, demonstrating the dominance of these countries in term of metaanalysis publications.{Figure 7} Conclusions Microbiology research outputs, as in other biomedical disciplines, are increasing at an exponential rate. Because of the critical importance of microbiology findings across populations and geographical regions, objective evaluation of these primary studies may facilitate decisionmaking that impacts upon public health policy. Clinicians and researchers in this field would likely benefit from the use and interpretation of this statistical technique. The survey in this review delineates the increasing use of metaanalysis in microbiology in North America and Asia, where China asserted its growing capability to utilise output from primary studies microbiology research only recently. In the EAEC metaanalysis example we cited in this review, we sought to examine the question of whether children with diarrhoea were more likely than those without diarrhoea to carry EAEC in stool (as detected by HEp2HeLa and pCVD432 assays). The reason we thought this necessary was that the existing studies had such conflicting results that varied in geography, methodology, and sample sizeleading to perceived controversy or uncertainty about the pathogenic nature of this organism. As our results show, the application of metaanalytic statistical approaches allowed us to confirm that EAEC is in fact a cause of acute diarrhoea in childhood, which was long considered controversial due to conflicting results from individual studies. Similar metaanalyses in microbiology may have the ability to advance our understanding of other emerging and established pathogens. References


