Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

Systematic Review Process: best practices

Manage & Analyse your Literature

A citation management program will save you a lot of time when doing your evidence synthesis.  Programs like Endnote, Mendeley or Zotero will store and organize the citations collected during your screening, de-duplicate the results and automatically format in-text citations and bibliographies in your manuscript.

With Mendeley Reference Manager you can store, organize and search all your references from just one library, seamlessly insert references and bibliographies into documents, cite, read, highlight and annotate PDFs. Download Mendeley

Zotero is a free and open-source reference management software to manage bibliographic data and related research materials.  Download Zotero.

EndNote enables you to easily view, edit, organize, and share research material.  You can Insert in-text citations while simultaneously creating a bibliography, easily collaborate, read, review, annotate and search PDFs in your library.  You can download the corporate licensed version of ENDNOTE here.

The purpose of article screening is to remove studies that are clearly not related to your topic.  This critical appraisal will help you evaluate the quality of relevant studies, as not all will have a sufficiently rigorous methodology to avoid biased results.   Use your inclusion/exclusion criteria to first screen the title and abstracts of your studies and determine whether they are relevant to your research question. Once titles and abstracts have been screened, the full text must be retrieved and screened to definitely decide whether the study fits the eligibility criteria of your synthesis.

It is highly recommended that two independent reviewers screen all studies, resolving areas of disagreement by consensus or by a third party who is an expert in the field. Listed below are tools that can be used for article screening.

Excel

Lists of references can be exported from reference management tools such as EndNote into Excel for screening. This is the most basic tool for the job. For a more fully-featured product see Rayyan below.

Rayyan

Rayyan is a free online tool that can be used for screening and coding of studies. It will pre-populate inclusion and exclusion criteria, although these can be customised. You can also tag and filter to code and organize references. Works well for title and abstract screening, for full-text screening revert to EndNote (you can export references from Rayyan back to EndNote for this step). Supports collaboration and blind reviewing to avoid bias. Has an app for use on mobiles and tablets.

Covidence

This is an online systematic review management tool.  Koç University does not have a licence for it, but individual licences can be purchased for small teams.

SRDR

SRDR (Systematic Review Data Repository) is a Web-based tool for the extraction and management of data for systematic review or meta-analysis. It is also an open and searchable archive of systematic reviews and their data. Access the "Create an Extraction Form" section for more information.

The Systematic Review Toolbox

The SR Toolbox is a community-driven, searchable, web-based catalogue of tools that support the systematic review process across multiple domains. Use the advanced search option to restrict to tools specific to data extraction. 

“My sources are unreliable, but their information is fascinating”                                         Ashleigh Brilliant

Unreliable sources of fascinating information can be great fun but, of course, high quality information is vital to sound research. Before deciding whether or not to incorporate what you have found into your literature review, you need to evaluate the resources to make sure that they contain information which is valuable and pertinent.

Accuracy        

  • Is the information reliable?
  • Is the information error-free?
  • Is the information based on proven facts?
  • Can the information be verified against other reliable sources?

Authority       

  • Who is the author?
  • Does he or she have the qualifications to speak/write on that topic?
  • Is the author affiliated with a reputable university or organization in this subject field?
  • Is the source peer reviewed or refereed?

Objectivity     

  • What is the intended purpose of the information?
  • Is the information fact or opinion?
  • Is the information biased?

Currency        

  • When was the information published?
  • Is the information current or out-dated?
  • Does currency matter in this topic?

Coverage        

  • Does the information covered meet your information needs?
  • Does it provide basic or in depth coverage

See the following links for further help on critically appraising the literature:

Data Extraction

You will need to extract data from relevant studies in order to examine and compare results. While the data is being extracted, it is very important to employ good data management practices. Proper data management should begin as soon as you start extracting data, and may even dictate which types of data you decide to retain. Some reviews require the use of software to help with extracting data.

What should I extract?

You should plan to extract data that is relevant to answering the question posed in your systematic review.  As mentioned previously, it may help to consult other similar systematic reviews to identify what data to collect.  You should use your key questions and your eligibility criteria as a starting point.

Helpful data may include:

  • Information about the study (author(s), year of publication, title, DOI)
  • Demographics (age, sex, ethnicity, diseases / conditions, other characteristics related to the intervention / outcome)
  • Methodology (study type, participant recruitment / selection / allocation, level of evidence, study quality)
  • Intervention (quantity, dosage, route of administration, format, duration, time frame, setting)
  • Outcomes (quantitative and / or qualitative)

If you plan to synthesize data, you will want to collect additional information such as sample sizes, effect sizes, dependent variables, reliability measures, pre-test data, post-test data, follow-up data, and statistical tests used.

Data Extraction Tools

     

You can use spreadsheet or database software to create custom extraction forms. Spreadsheet functions such as drop-down menus and range checks can speed up the process and help prevent data entry errors. Relational databases (such as Microsoft Access) can help you extract information from different categories like citation details, demographics, participant selection, intervention, outcomes, etc.

            

RevMan is free software used to manage Cochrane reviews. For more information on RevMan, including an explanation of how it may be used to extract and analyze data, watch Introduction to RevMan - a guided tour.

  

Survey or form tools can help you create custom forms with many different question types, such as multiple choice, drop downs, ranking, and more.  Content from these tools can often be exported to spreadsheet or database software as well. Here at KU we have access to the survey/form software Qualtrics

   

SRDR (Systematic Review Data Repository) is a Web-based tool for the extraction and management of data for systematic review or meta-analysis. It is also an open and searchable archive of systematic reviews and their data. Access the "Create an Extraction Form" section for more information.

                                                        

 

 

 

The SR Toolbox is a community-driven, searchable, web-based catalogue of tools that support the systematic review process across multiple domains. Use the advanced search option to restrict to tools specific to data extraction. 

                                                                             

 

DistillerSR is a systematic review management software program, similar to Covidence. It guides reviewers in creating project-specific forms, extracting, and analyzing data.