![Cochrane Handbook For Systematic Reviews Of Interventions Version Cochrane Handbook For Systematic Reviews Of Interventions Version](/uploads/1/2/5/8/125871324/438865013.png)
Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. This page provides detailed information about this publication. Please click on any. Corrections and changes in version 4.2.1 (December 2003) of the Handbook. 1.1 Systematic reviews and the Cochrane Handbook.
Published online 2011 Dec 24. doi: 10.1007/s00264-011-1460-y
PMID: 22198362
This article has been cited by other articles in PMC.
Abstract
In the last decades of the twentieth century it became obvious that modern medical care is replete with data and information, but in need of reliable evidence. This has led to an increased effort to systematically synthesise medical research and make it more useful for practitioners. Systematic reviews use an approach to research synthesis that minimises the risk of misinterpreting a body of evidence due to incomprehensive search or subjective opinion. Carrying out a systematic review is a rigorous procedure which corresponds to standard methodological steps in primary research studies. It involves posing a well-defined question, developing a robust search strategy, screening for relevant primary studies, critical appraisal of included studies, data extraction and processing, analysis and interpretation of results. In some, but not all systematic reviews it is appropriate to conduct a meta-analysis, which is a statistical procedure that integrates the results of several independent studies. Results of meta-analysis are graphically presented in forest plots, with pooled point estimate and its confidence interval represented as a rhombus, usually called a “diamond”. Methodological quality of systematic reviews should not be judged by the quality of primary studies included, but by a distinct set of criteria specified in assessment tools such as AMSTAR. Systematic reviews and meta-analyses should be reported according to the PRISMA checklist. A major contribution to the development of methodological standards has been given by The Cochrane Collaboration, whose Handbook of Systematic Reviews of Interventions is the primary reference for all authors and referees of systematic reviews in health care.
Introduction
Orthopaedic surgery shares a common feature with all other medical specialties: it is full of questions. A patient’s simple complaint of a painful knee presents the physician with a series of question marks. Which investigation will lead to a correct diagnosis? What therapy should be recommended—drugs, physiotherapy, surgery or a combination? What is the proper dosage and duration of therapy? Which surgical approach should be taken? What are the adverse effects and what is the risk of therapeutic failure? Which therapy is most cost-effective?
Called to come up with correct answers, the physician stands before the patient as the ancient king Oedipus stood before the Sphinx. A life may be at stake. What is there to help the physician in resolving the riddle?
In the age of evidence-based medicine [], medical research is propagated as an invaluable aid in making health care decisions. However, busy clinicians find it difficult to keep up with the burgeoning medical literature. For example, the Thomson Reuters Journal Citation Reports (http://scientific.thomson.com/products/jcr) lists 61 journals in the “Orthopaedics” subject category alone. The sheer quantity of published material is not the sole problem—the quality of research and its relevance for practice is variable and it requires critical appraisal skills to sift the “wheat” from the “chaff”.
In the last decades of the twentieth century it became obvious that modern medical care is replete with data and information, but in need of reliable evidence. This has led to an increased effort to systematically synthesise medical research and make it more useful for practitioners. Actually, research has always been synthesised—either formally or informally—and weighted by experts, local practice, marketing, journal impact factor or number of citations. Such synthesis, however, was prone to overlook important sources of data and come to biased conclusions []. Evidence-based medicine proponents therefore advocated a systematic approach to research synthesis, which would minimise the risk of misinterpreting a body of evidence due to incomprehensive search or subjective opinion.
In recent years, the popularity of systematic reviews has risen and they are now commonly published in medical journals []. International Orthopaedics is no exception: a Web of Knowledge search using the keywords “systematic review” or “meta-analysis” in the title yielded 33 items in this journal (as of 22 August 2011), five of which were not true systematic reviews or meta-analyses. The first article of this type was published in 1996 and their number has increased particularly in recent years (Fig. 1).
Number of systematic reviews published in International Orthopaedics between 1996 and 2011 (data for 2011 up to 22 August)
A properly conducted systematic review can serve one or more of the following purposes: (1) reduce the influence of any flaws or errors in a single study; (2) resolve the confusion when several studies report conflicting findings; (3) yield new insights by combining findings from different studies; (4) highlight when there is not enough evidence and further research is needed to resolve a clinical question; and (5) show when enough evidence has been produced and no further research is needed or justifiable [4].
Clinicians can better appreciate the value of systematic reviews if they understand that producing such a piece of research requires a rigorous procedure corresponding to standard steps used in primary research studies.
Posing the right question
As with any other research, a systematic review begins with formulation of one or more answerable questions. The review questions should be born out of real-life clinical dilemmas, so that the review findings have a potential to influence medical practice or policy. Ideally, consumers should be involved in this endeavour, for example by pointing out the relevant questions or outcomes that need to be addressed [, ].
The review questions are preferably written in the form of PICO, which is an acronym for “patients/population”, “intervention”, “comparison intervention” and “outcome” [7]. The PICO format was first developed for intervention reviews, but it can be adapted to other types of reviews and questions, such as for diagnostic accuracy or prognosis studies. PICO is basically a statement on the inclusion and exclusion criteria. Too broad a question may result in an unmanageable number of studies to be included and analysed, and too narrow a question may yield no studies at all. In both cases the review findings may not be useful in practice. Therefore a preliminary search of the literature is recommendable to inform decisions on the inclusion and exclusion criteria.
Besides the four key elements in the PICO format, there is another important criterion that needs to be defined when planning a review question: design of studies to be included. Some reviews are limited to randomised controlled trials (RCT) and others include non-randomised study designs also. For most questions on the effectiveness of clinical intervention it makes sense to include only RCTs, as this design provides the highest level of evidence in primary research studies [8, ]. Non-randomised study designs are appropriate for other types of questions, such as long-term safety and cost-effectiveness, and for some types of intervention, especially the more complex variety. The decision about inclusion of non-randomised trials has implications for the analysis phase of the review, as simply combining RCTs and non-randomised studies in a meta-analysis is not recommendable [10].
An example of well-defined inclusion and exclusion criteria can be found in the systematic review of closed suction surgical wound drainage after hip fracture surgery []. From the first paragraph of the methods section it is clear that the review looked at patients who underwent hip fracture surgery, the intervention of interest was closed suction drainage, the comparison was no drainage and outcomes were wound infection, wound haematoma, complications directly relating to drains and transfusion rate. The review included randomised and quasi-randomised controlled trials, irrespective of the quality of allocation concealment [].
Developing a search strategy
One of the crucial differences between a systematic review and a traditional, narrative, non-systematic review is in the comprehensiveness of the literature search. An objective answer to the review question can only be given if all primary studies that explored this question are found and considered.
The search begins with electronic databases and PubMed is usually the first database to be explored. However, it is not sufficient to limit the search only to PubMed, as other databases may contain relevant studies not covered by PubMed []. Some of the important electronic databases for clinical studies are the Cochrane Central Register of Controlled Trials, EMBASE and SCOPUS. If at all possible, the search should include non-English databases such as LILACs and SciELO for studies in Spanish and Portuguese, and the Chinese biomedical database Cnki.net.
The search string typically includes terms related to the health condition or problem, interventions and study design, joined by Boolean operators (AND, OR). Truncation and wildcards are used to capture synonyms, related terms and variant spellings [13]. Assistance from an experienced librarian may be very helpful at this stage of review production [].
Relevant studies may also be found by going through the reference lists of included studies (“backward citation search”) or by checking the articles that have cited the included studies (“forward citation search”) [15].
Depending on the review question, authors may decide to search for reports that are not published in scientific journals and indexed in electronic databases. This kind of material is known as grey literature and includes technical reports, working papers and white papers produced by government agencies or research groups. Systematic reviews in orthopaedics mostly address clinical questions that are best answered by properly conducted clinical trials, so grey literature may not be a sufficiently reliable source of data. However, even clinical trials sometimes remain unpublished and review authors need to deal with this fact. An important recent development that allows searching for unpublished studies is the establishment of publicly available registers of clinical trials and the move to make the trial registration obligatory []. Some unpublished studies may be detected by contacting authors of published studies and industry representatives. Failing to identify unpublished trials may lead reviewers to biased conclusions, as published trials are more likely to report significant effects than unpublished ones [].
Transparent reporting of search strategy and results in systematic reviews is achieved by appending the search string to the published review and presenting a study flow chart with the number of unique records: (1) identified by the searches, (2) excluded after preliminary screening of titles and abstracts, (3) retrieved in full text, (4) excluded after reviewing the full text, with reasons for exclusion, and (5) number of studies finally included in the review (Fig. 2). A good example is the review of surgical vs non-surgical treatment of chronic low back pain [], where five different electronic databases were searched, using a set of search terms describing the health problem (low back pain), interventions (physical therapy, physiotherapy, rehabilitation, cognitive therapy, surgery, spinal fusion, spinal stabilisation) and study design (RCT, random allocation, randomisation, clinical trial). A study flow chart was provided, although the full search string was not appended to the review [].
Flow diagram of studies in a systematic review []. Available via http://www.prisma-statement.org
Importance of review protocol
Authors of systematic reviews are well aware of problems and potential biases that arise when protocols of clinical trials are not registered in a timely manner or are not available for public scrutiny. It is only logical to apply the same standards for the systematic reviews, whose detailed protocols should be published prior to the beginning of the search and screening. Indeed, protocols of systematic reviews are regularly published in The Cochrane Library and, as of February 2011, the new PROSPERO database (http://www.crd.york.ac.uk/prospero) is available to everyone who wishes to prospectively register their systematic reviews in health and social care.
The benefits of the prospective registration of systematic reviews are twofold: it enables comparison of reported review findings with what was planned in the protocol and prevents the duplication of review effort by two or more groups of authors who may simultaneously work on the same review question, without knowledge of each other. Additionally, if protocols are subjected to peer refereeing, as in the case of systematic reviews produced within The Cochrane Collaboration, some important amendments can be introduced before the review work has actually begun [19].
The protocols should contain a detailed description of review objectives, criteria for inclusion and exclusion of studies, comparisons and outcomes of interest, search strategy, and data collection and analysis, including planned methods for the quality assessment of the included studies. Any deviation from the protocol can introduce bias [, ], and must be addressed and justified in the final review.
Screening and data extraction
The search results need to be sorted out and duplicate records removed, either manually or by use of reference management software. The search may yield several hundreds or even thousands of titles and/or abstracts that need to be screened. Screening is a tedious job and the first instance in the review process where at least two authors need to perform the work independently. The reason is that a single person can overlook a relevant item, thus making a double check necessary [22]. It may be practical to use a checklist for assessing the eligibility of records []. If the relevance of a report is unclear, the full text should be obtained and studied. As the screening involves a degree of arbitrariness, one can expect some disagreements which should be resolved by discussion and consensus among review authors.
Once the final set of included studies is decided upon, data extraction can begin. This step should also be done by two authors independently to avoid inadvertent errors []. Ideally, the data extraction form should be predesigned in a word processing or spreadsheet software, accompanied by a written guide, and piloted before use to ensure clarity of items and consistency of data entry. Data extraction sheets can take different forms and authors should look for different examples to find the form that suits them best. It is wise to extract the data as comprehensively as possible. Every detail reported in the methods and results section of a study article should be systematically recorded, as it may be useful in the analysis phase of the review. Introduction and discussion sections of included articles usually do not contain information relevant for the analysis, but they provide a context for the overall consideration of the study.
Data extraction is not always a simple and straightforward task. The quality of reports may vary considerably and many important details may be missing, especially in older studies published before reporting guidelines were introduced and widely accepted []. There are different conversion rules that use available data to calculate the statistics required for meta-analyses. For example, standard deviations can be calculated based on the reported range, standard error of mean or 95% confidence interval [26].
In some cases, inadequate reporting makes it impossible to calculate the necessary data or statistics from the published results, so the review authors may be forced to exclude such reports from the analyses []. A possible solution is to contact the primary study authors and ask them to provide the raw data, but this approach may not always be successful.
Critical appraisal and assessing risk of bias
Individual studies need to be critically appraised before any valid conclusions can be drawn about a body of evidence. Critical appraisal can never be fully objective, so it is usually done by at least two reviewers to achieve an agreement on the quality of included studies. There are many different scales and checklists that have been used to assess the quality of studies [] and authors of systematic reviews should be able to justify their choice of assessment tool. There is a difference between the quality of research and quality of reporting, whereby the former relates to the risk of bias in the actual design and conduct of a study, and the latter refers to the adequacy and completeness of reporting [29]. Some scales and checklists confuse these separate issues [], and this should be taken into account when choosing the assessment tool for a systematic review.
An increasing number of systematic reviews, especially those including only RCTs, use the Cochrane Risk of Bias Tool to assess the quality of included studies [29]. This tool differs from most existing scales and checklists in that it does not attempt to provide a summary score, but aims to qualitatively assess and transparently describe the key study domains which can be related to different types of biases. Specifically, risk of selection bias is assessed by the quality of sequence generation and concealment, performance bias by blinding of participants and personnel, detection bias by blinding of outcome assessors, attrition bias by completeness of outcome data and reporting bias by completeness of reporting. All other potential sources of bias are assessed under the final domain of the tool. It is advisable to use the latest and complete version of the Cochrane Risk of Bias Tool, though some review authors have decided to modify the tool for their purposes [].
It is hardly disputable that low-quality studies included in a systematic review cannot have the same weight as high-quality studies. Consequently, studies should not only be critically appraised, but the results of this appraisal should be integrated into the analysis and interpretation of review findings. A possible way to do that is to conduct a sensitivity analysis, for example by including only high-quality (or low risk of bias) studies in the primary meta-analysis and then exploring how inclusion of low-quality (or high risk of bias) studies would affect the summary estimates.
Meta-analysis
Results of primary studies included in a systematic review can be analysed in two ways: narratively, by providing a summary and discussion of findings, and quantitatively, by using a statistical procedure called meta-analysis. In the early days of evidence-based medicine, the terms ‘systematic review’ and ‘meta-analysis’ were commonly used interchangeably, which caused some confusion in the understanding of these concepts. An illustrative example is that the first reporting guidelines for systematic reviews in medicine were called the Quality of Reporting of Meta-analyses (QUOROM), which addressed aspects such as search strategy or validity assessment that were clearly not related to statistical procedure of meta-analysis, but to the systematic review as a study design []. This was corrected in the revised version of the reporting guidelines, now called Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [, ].
Meta-analysis is much better than narrative analysis in providing answers to the three most important questions in a systematic review: (1) what is the direction of effect, (2) what is the size of the effect and (3) is the effect consistent across studies. As meta-analysis uses weighted results of more than one independent study, it can provide more precise estimates of the effects than those derived from individual studies. It is possible to conduct meta-analysis for some comparisons and outcomes in a systematic review, and not for others.
Meta-analysis may not be appropriate if studies or outcomes are too diverse or if included studies are at high risk of bias. In such cases, a thorough narrative summary with a thoughtful discussion of primary studies’ findings is a more meaningful approach.
Results of meta-analyses are graphically presented in forest plots []. For example, in the systematic review comparing gamma nail and dynamic hip screw in treating peritrochanteric fractures, there are five forest plots presenting comparisons of the studied interventions for five different outcomes: wound infection rate, mortality, postoperative femoral shaft fracture rate, reoperation rate and patients walking independently []. Mortality was assessed in six of seven studies included in the review and their results are shown as squares centred on the mortality point estimate for each study (Fig. 3). Horizontal lines that run through the squares indicate 95% confidence intervals of the estimates. The overall estimate—which is the result of meta-analysis—is at the bottom, represented as a diamond. The centre of the diamond is the pooled point estimate, and its horizontal tips indicate the confidence interval. As the vertical line of no effect in the forest plot cuts through the diamond, it is obvious that there is no statistically significant difference in mortality between the two compared interventions [].
An example of a forest plot as a graphical presentation of meta-analysis (from [], with permission)
Assuring quality in systematic reviews
Not all systematic reviews are equally well conducted [, ]. However, it is a mistake to judge the quality of a systematic review by the quality of studies included. A properly conducted systematic review explores how the quality of primary studies affects our confidence in the estimates of effectiveness, for example by sensitivity analysis. If most of the included studies are low quality, meta-analysis may be judged inappropriate, with a valid conclusion that there is no reliable evidence regarding the studied question.
Systematic review is a distinct research enterprise and it ought to be appraised by a particular set of criteria. There are several dozen appraisal tools for systematic reviews available in the literature [38], but most of them are not widely used. As with the primary research reports, one should make a distinction between the quality of review design and conduct, and the quality of reporting. AMSTAR is a convenient and validated 11-item tool for assessing the methodological quality of systematic reviews [, ]. The PRISMA statement lists a minimum set of items that should be reported in systematic reviews and meta-analyses [, ], so it can be used as a tool for assessing the quality of reporting. For a more comprehensive report on standards for the conduct and reporting of systematic reviews, readers are advised to consult Finding What Works in Health Care: Standards for Systematic Reviews, a recent publication of the US Institute of Medicine of the National Academies [41].
A major contribution to the development of methodological standards for systematic reviews has been given by The Cochrane Collaboration (www.cochrane.org). This international network of more than 28,000 dedicated people from over 100 countries was established in 1993, with a mission to help health care providers, policymakers, patients, their advocates and carers make well-informed decisions about health care, based on the best available research evidence, by preparing, updating and promoting the accessibility of systematic reviews. Several thousand Cochrane reviews, covering all areas of medicine and health care, have been published in the Cochrane Database of Systematic Reviews and even more studies relevant to the methods of systematic reviews of health care and social interventions have been collected in the Cochrane Methodology Register []. There are 15 Cochrane Methods Groups that gather experts interested in different methodological aspects of systematic reviews. The Cochrane Handbook of Systematic Reviews of Interventions is the flagship methodological publication of The Cochrane Collaboration and the primary reference book for all authors and referees of systematic reviews in health care [43].
In conclusion, systematic reviews are research studies that use rigorous, evidence-based, transparent and reproducible methods, which make them very different from traditional narrative reviews []. The importance of team work in the undertaking of systematic reviews cannot be overestimated and author teams should ideally include content experts (clinicians), a methodologist and/or statistician and a librarian. This article should help readers to better understand and appreciate the work behind systematic reviews and inform the assessors of future systematic reviews submitted to International Orthopaedics.
Acknowledgments
Dario Sambunjak is financially supported by the Croatian Ministry of Science, Education and Sports (grant No. 216-1080314-0245 to Matko Marusic).
References
1. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312:71–72. doi: 10.1136/bmj.312.7023.71.[PMC free article] [PubMed] [CrossRef] [Google Scholar]
2. Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA. 1992;268:240–248. doi: 10.1001/jama.1992.03490020088036. [PubMed] [CrossRef] [Google Scholar]
3. Ernst E, Pittler MH. Assessment of therapeutic safety in systematic reviews: literature review. BMJ. 2001;323:546. doi: 10.1136/bmj.323.7312.546.[PMC free article] [PubMed] [CrossRef] [Google Scholar]
4. Lane S (2009) Sense about systematic reviews. Available via http://www.senseaboutscience.org/data/files/resources/52/Sense-About-Systematic-Reviews.pdf
5. Wale J, Colombo C, Belizan M, Nadel J. International health consumers in the Cochrane Collaboration: fifteen years on. J Ambul Care Manage. 2010;33:182–189. [PubMed] [Google Scholar]
6. Wale JL, Belizán M, Nadel J, Jeffrey C, Vij SL (2011) The Cochrane Library review titles that are important to users of health care, a Cochrane Consumer Network project. Health Expect [Epub ahead of print]. doi: 10.1111/j.1369-7625.2011.00723.x [PMC free article] [PubMed]
7. Straus SE, Richardson WS, Glasziou P, Haynes RB. Evidence-based medicine: how to practice and teach EBM. 3. Edinburgh: Churchill Livingstone; 2005. [Google Scholar]
8. Oxford Centre for Evidence-based Medicine (2001) Levels of evidence. http://www.cebm.net/index.aspx?o=1047. Accessed 14 Nov 2011
9. Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64:401–406. doi: 10.1016/j.jclinepi.2010.07.015. [PubMed] [CrossRef] [Google Scholar]
10. Reeves BC, Deeks JJ, Higgins JPT, Wells GA, on behalf of the Cochrane Non-Randomised Studies Methods Group (2011) Chapter 13: Including non-randomized studies. In: Higgins JPT, Green S (eds) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration. Available via www.cochrane-handbook.org. Accessed 14 Nov 2011
11. Clifton R, Haleem S, McKee A, Parker MJ. Closed suction surgical wound drainage after hip fracture surgery: a systematic review and meta-analysis of randomised controlled trials. Int Orthop. 2008;32:723–727. doi: 10.1007/s00264-007-0420-z.[PMC free article] [PubMed] [CrossRef] [Google Scholar]
12. Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, Ramos-Remus C. Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials. 2000;21:476–487. doi: 10.1016/S0197-2456(00)00067-2. [PubMed] [CrossRef] [Google Scholar]
13. Lefebvre C, Manheimer E, Glanville J, on behalf of the Cochrane Information Retrieval Methods Group (2011) Chapter 6: Searching for studies. In: Higgins JPT, Green S (eds) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration. Available via www.cochrane-handbook.org. Accessed 14 Nov 2011
14. Mead TL, Richards DT. Librarian participation in meta-analysis projects. Bull Med Libr Assoc. 1995;83:461–464.[PMC free article] [PubMed] [Google Scholar]
15. Bates MJ. The design of browsing and berrypicking techniques for the online search interface. Online Review. 1989;13:407–424. doi: 10.1108/eb024320. [CrossRef] [Google Scholar]
16. Laine C, Horton R, DeAngelis CD, Drazen JM, Frizelle FA, Godlee F, et al. Clinical trial registration: looking back and moving ahead. JAMA. 2007;298:93–94. doi: 10.1001/jama.298.1.jed70037. [PubMed] [CrossRef] [Google Scholar]
17. Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K (2009) Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev 1:MR000006 [PubMed]
18. Ibrahim T, Tleyjeh IM, Gabbar O. Surgical versus non-surgical treatment of chronic low back pain: a meta-analysis of randomised trials. Int Orthop. 2008;32:107–113. doi: 10.1007/s00264-006-0269-6.[PMC free article] [PubMed] [CrossRef] [Google Scholar]
19. Bero LA, Grilli R, Grimshaw J, Harvey E, Oxman AD (1997) External refereeing of protocols for systematic reviews [abstract]. Prague, Czech Republic, Third International Congress on Biomedical Peer Review and Global Communications
20. Silagy CA, Middleton P, Hopewell S. Publishing protocols of systematic reviews. Comparing what was done to what was planned. JAMA. 2002;287:2831–2834. doi: 10.1001/jama.287.21.2831. [PubMed] [CrossRef] [Google Scholar]
21. Kirkham JJ, Altman DG, Williamson PR. Bias due to changes in specified outcomes during the systematic review process. PLoS One. 2010;5(3):e9810. doi: 10.1371/journal.pone.0009810.[PMC free article] [PubMed] [CrossRef] [Google Scholar]
22. Glasziou P, Sanders S, Pirozzo S, Doust J, Pietrzak E (2002) Abstract screening—the value of two reviewers [abstract]. Pushing the Boundaries, Oxford, United Kingdom, Fourth Symposium on Systematic Reviews
![Systematic Systematic](/uploads/1/2/5/8/125871324/404510846.jpg)
23. Stavlas P, Roberts CS, Xypnitos FN, Giannoudis PV. The role of reduction and internal fixation of Lisfranc fracture-dislocations: a systematic review of the literature. Int Orthop. 2010;34:1083–1091. doi: 10.1007/s00264-010-1101-x.[PMC free article] [PubMed] [CrossRef] [Google Scholar]
24. Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP. Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol. 2006;59:697–703. doi: 10.1016/j.jclinepi.2005.11.010. [PubMed] [CrossRef] [Google Scholar]
25. Plint AC, Moher D, Morrison A, Schulz K, Altman DG, Hill C, et al. Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. Med J Aust. 2006;185:263–267. [PubMed] [Google Scholar]
26. Higgins JPT, Deeks JJ (2011) Chapter 7: Selecting studies and collecting data. In: Higgins JPT, Green S (eds) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration
27. Cheng T, Feng JG, Liu T, Zhang XL. Minimally invasive total hip arthroplasty: a systematic review. Int Orthop. 2009;33:1473–1481. doi: 10.1007/s00264-009-0743-z.[PMC free article] [PubMed] [CrossRef] [Google Scholar]
28. Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials. 1995;16:62–73. doi: 10.1016/0197-2456(94)00031-W. [PubMed] [CrossRef] [Google Scholar]
29. Higgins JPT, Altman DG, Sterne JAC on behalf of the Cochrane Statistical Methods Group and the Cochrane Bias Methods Group (2011) Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S (eds) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration. Available via www.cochrane-handbook.org. Accessed 14 Nov 2011
30. Xue D, Zheng Q, Li H, Qian S, Zhang B, Pan Z. Selective COX-2 inhibitor versus nonselective COX-1 and COX-2 inhibitor in the prevention of heterotopic ossification after total hip arthroplasty: a meta-analysis of randomised trials. Int Orthop. 2011;35:3–8. doi: 10.1007/s00264-009-0886-y.[PMC free article] [PubMed] [CrossRef] [Google Scholar]
31. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF, for the QUOROM Group Improving the quality of reports of meta-analyses of randomised controlled trials: the QUORUM statement. Lancet. 1999;354:1896–1900. doi: 10.1016/S0140-6736(99)04149-5. [PubMed] [CrossRef] [Google Scholar]
32. Moher D, Liberati A, Tetzlaff J, Altman DG for the PRISMA Group (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6: e1000097 [PMC free article] [PubMed]
33. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100. doi: 10.1371/journal.pmed.1000100.[PMC free article] [PubMed] [CrossRef] [Google Scholar]
34. Lewis S, Clarke M. Forest plots: trying to see the wood and the trees. BMJ. 2001;322:1479–1480. doi: 10.1136/bmj.322.7300.1479.[PMC free article] [PubMed] [CrossRef] [Google Scholar]
35. Liu M, Yang Z, Pei F, Huang F, Chen S, Xiang Z. A meta-analysis of the Gamma nail and dynamic hip screw in treating peritrochanteric fractures. Int Orthop. 2010;34:323–328. doi: 10.1007/s00264-009-0783-4.[PMC free article] [PubMed] [CrossRef] [Google Scholar]
36. Furlan A, Clarke J, Esmail R, Sinclair S, Irvin E, Bombardier C. A critical review of reviews on the treatment of chronic low back pain. Spine. 2001;26:E155–E162. doi: 10.1097/00007632-200104010-00018. [PubMed] [CrossRef] [Google Scholar]
37. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4:e78. doi: 10.1371/journal.pmed.0040078.[PMC free article] [PubMed] [CrossRef] [Google Scholar]
38. Shea BJ, Dubé C, Moher D. Assessing the quality of reports of systematic reviews: the QUOROM statement compared to other tools. In: Egger M, Smith GD, Altman DG, editors. Systematic reviews in health care: meta-analysis in context. London: BMJ Books; 2001. pp. 122–139. [Google Scholar]
39. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10. doi: 10.1186/1471-2288-7-10.[PMC free article] [PubMed] [CrossRef] [Google Scholar]
40. Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62:1013–1020. doi: 10.1016/j.jclinepi.2008.10.009. [PubMed] [CrossRef] [Google Scholar]
41. Eden J, Levit L, Berg A, Morton S, editors. Finding what works in health care: standards for systematic reviews. Washington, DC: National Academies Press; 2011. [Google Scholar]
42. Starr M, Chalmers I, Clarke M, Oxman AD. The origins, evolution, and future of The Cochrane Database of Systematic Reviews. Int J Technol Assess Health Care. 2009;25(1):182–195. doi: 10.1017/S026646230909062X. [PubMed] [CrossRef] [Google Scholar]
43. Higgins JPT, Green S, eds (2011) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011], The Cochrane Collaboration. Available via www.cochrane-handbook.org. Accessed 14 Nov 2011
44. Lang TA. The value of systematic reviews as research activities in medical education. Acad Med. 2004;79:1067–1072. doi: 10.1097/00001888-200411000-00011. [PubMed] [CrossRef] [Google Scholar]
Articles from International Orthopaedics are provided here courtesy of Springer-Verlag