Public Health

Library of the Health Sciences

Take notes!

Your Librarian

Profile Photo
Devon Olson
Contact:
Hours (virtual):
Monday through Friday: 9:00 a.m. - 5:00 p.m.
Available by appointment

School of Medicine and Health Sciences
1301 N Columbia Road
701-777-4828

Tools

  • Covidence is an online systematic review program developed by, and for, systematic reviewers. It can import citations from reference managers like EndNote, facilitate the screening of abstracts and full-text, populate risk of bias tables, assist with data extraction, and export to all common formats.
    • Covidence is not a free tool, though you can sign up to receive a free trial
  • Rayyan is a web application to help systematic review authors create systematic reviews, collaborate on them, maintain them over time and get suggestions for article inclusion.
    • Rayyan is free
  • SR Toolbox is a community-driven, searchable, web-based catalogue of tools that support various tasks within the systematic review and wider evidence synthesis process.

Guidelines for reporting your review

The rules for reporting systematic reviews are not as stringent as with systematic reviews, though use of the PRISMA scoping reviews extension (PRISMA-SR) is beginning to become the norm in some disciplines

  • PRISMA-SR scoping review extension
    • Statement/ explanatory paper:
      • Tricco, AC, Lillie, E, Zarin, W, O'Brien, KK, Colquhoun, H, Levac, D, Moher, D, Peters, MD, Horsley, T, Weeks, L, Hempel, S et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018,169(7):467-473. doi:10.7326/M18-0850.
    • Fillable Checklist - PDF | Word

Scoping Reviews

Scoping reviews are similar to systematic reviews in that they collect and analyze a large body of literature. There is even a PRISMA flow chart designed especially for scoping reviews. However, there are some important differences between systematic and scoping reviews:

  • The purpose of a scoping review is to identify gaps in the research by exploring the nature and extent of what has already been done, rather than assess the quality or efficacy of that previous research
    • findings usually include themes identified by the authors
  • Scoping reviews aren't expected to gather all of the available research, simply to gather enough evidence to get an idea of the scope of the larger body of research
  • Scoping reviews are a form of secondary research literature, because they gather together and analyze the results of primary (and also sometimes secondary) research

Is a scoping review for you?

indications for systematic reviews:

  • uncover the international evidence
  • confirm current, variations on, and new practice
  • identify and inform areas for future research
  • identify and investigate conflicting results
  • produce statements to guide decision-making

 

indications for scoping reviews

  • identify the types of available evidence in a given field
  • clarify key concepts and definitions in the literature
  • examine how research is conducted within a certain topic or field
  • identify key characteristics or factors related to a concept
  • as a precursor to a systematic review
  • identify and analyze knowledge gaps

 

  • the above "indications" are from:
    Munn, Z., Peters, M. D. J., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18(1), 143. https://doi.org/10.1186/s12874-018-0611-x
     

and 12 more types of reviews:

  • Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26(2), 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

The Librarian's role

How can librarians support the review?

  • Translating a research question into an effective search strategy
  • Locating existing systematic reviews and protocols to give you background information for your work
  • Maximizing the balance between search precision and retrieval 
  • Selecting databases for systematic searching
  • Advising on grey literature search strategies
  • Sharing strategies and tools for saving, managing and exporting search results and citations

Levels of librarian support and how to credit

Much of the work a librarian does to assist researchers falls under their everyday job responsibilities. However, systematic reviews, scoping reviews, and meta-analyses, often require greater librarian involvement and include the librarian as a partner in the research project. When these levels of assistance lead to a published paper, it is appropriate to credit the librarian, either as a coauthor or acknowledged contributor.

Basic guidelines on what to expect as usual support from your librarian collaborator:

  • Uncredited - regular librarian work
    • Overview of review process and best practices
    • Identifying keywords and databases
      • but not the construction of structured search phrases
    • Recommending tools
    • Single consultation
  • Acknowledgement or Contributor note - regular librarian work but significant support
    • Troubleshooting search strategy created mostly by researchers
    • Data management consultation
    • More than one consultation
  • Authorship (one or more of the following)
    • study uses a search strategy or filter developed by librarian
    • registering a protocol
    • citation retrieval and management
    • hand searching for articles
    • Assisting with paper's methods section
    • Section edits

Best Practices

The Institute of Medicine, the  Cochrane Collaboration, the Centre for Reviews and Dissemination (University of York), and other organizations each have their own recommendations on conducting a systematic or scoping review.  

Best practices from Cochrane:

  • Include a medical librarian on your team
    • Cochrane recommends the team include "an experienced medical/healthcare librarian or information specialist to provide support for the search process." (Cochrane 2023 4-1)
  • Use Boolean to structure your searches
    • "Search strategies should avoid using too many different search concepts but a wide variety of search terms should be combined with OR within each included concept." (Cochrane 2023 4-2-2)
  • Use subject headings in your searches
    • "Both free-text and subject headings (e.g. Medical Subject Headings (MeSH) and Emtree) should be used." (Cochrane 2023 4-2-2)
  • Minimize selection bias
    • Manage the efficiency and thoroughness of the search in a way that minimizes language and publication bias (Cochrane 2023 4-2-2)
  • Do not have 2 people independently search for articles
    • "Unlike for tasks such as study selection or data extraction, it is not considered necessary (or even desirable) for two people to conduct independent searches in parallel. It is strongly recommended, however, that all search strategies should be peer reviewed, before being run, by a suitably qualified and experienced medical/healthcare librarian or information specialist" (Cochrane 2023 section 4-2-2)
  • Search all relevant databases
    • "It is highly desirable that searches be conducted of appropriate national, regional and subject specific bibliographic databases" (Cohrane 2023 4-3-1-4)
  • Select a minimum of 2 people to independently code your articles
    • "As a minimum, information that involves subjective interpretation and information that is critical to the interpretation of results (e.g. outcome data) should be extracted independently by at least two people" (Cochrane 2023 5-5-2)

Steps

The steps for conducting a systematic review are the same as that of a scoping review. The difference is in your scope, you aren't gathering all of the research, only enough to analyze its scope:

  1. Form your research team
    1. structured reviews are often so much work that a team of researchers is needed
      1. Guideline: Cochrane recommends the team include "an experienced medical/healthcare librarian or information specialist to provide support for the search process." (Cochrane 2023 4-1)
    2. Make sure you have enough people on your team to allow 2 people to independently review all articles collected
      1. Guideline: "As a minimum, information that involves subjective interpretation and information that is critical to the interpretation of results (e.g. outcome data) should be extracted independently by at least two people" (Cochrane 2023 5-5-2)
  2. Form an answerable clinical question
  3. Identify the type of review appropriate to your question
  4. Identify possible journals for publication
  5. Refine research question
  6. Check protocols and guides
    1. Read guidelines for your review:

      1. The Institute of Medicine Guidelines

      2. Cochrane Collaboration Guidelines

      3. The Centre for Reviews and Dissemination (University of York) Guidelines

    2. Check registers of protocols of currently underway reviews to be sure that another team is not currently working on a review on your same research question

      1. PROSPERO - International Prospective Register of Systematic Reviews

        1. here is an article analyzing PROSPERO: Booth A, Mitchell AS, Mott A, James S, Cockayne S, Gascoyne S, McDaid C. An assessment of the extent to which the contents of PROSPERO records meet the systematic review protocol reporting items in PRISMA-P. F1000Res. 2020 Jul 27;9:773. doi: 10.12688/f1000research.25181.2.

      2. Open Science Framework (OSF)

    3. PRISMA is a checklist that helps you ensure that you report the correct info when you write up your review. It is important to consult PRISMA from the start to ensure that you are gathering data in the correct way to facilitate accurate reporting in your publication later.

      1. PRISMA Flow Diagram is the template used by authors of scoping reviews to illustrate their article search process

      2. PRISMA Flow Diagram Generator

      3. Statement/ explanatory paper

      4. Fillable Checklist - PDF | Word

  7. Start a draft of your own research protocol
  8. Identify databases for structured search and journals/databases for hand search
    1. Systematic and scoping reviews typically search a minimum of 3 academic journal article databases which cover the disciplines represented by your research question. If there are more nuanced concepts in your question, you may need to search even more databases.
      1. for example, for a research question on Indigenous maternal mental health, a librarian might recommend you search PubMed (medicine), CINAHL (health sciences), PsycINFO (psychology), and one or two non-paywalled Indigenous-focused databases like iPortal or Native Health Database.
    2. Hand-searching is a manual method of scanning select journals from cover to cover, page-for-page for relevant articles in case they were missed during indexing. According to the Cochrane Handbook"...involves a manual page-by-page examination of the entire contents of a journal issue or conference proceedings to identify all eligible reports of trials.

      "Handsearching may include checking the reference lists of journal articles, a technique called snowballing. In 2013, Craane et al found that "...hand search[ing] plays a valuable role in identifying randomised controlled trials" beyond Medline and Embase.

      Craane B., Dikstra PU. (2012 Feb) Methodological quality of a systematic review on physical therapy for temporomandibular disorders: influence of hand search and quality scales.  Clinical Oral Investigations 16(1) 295-303

      1. Some researchers also place a search of google scholar or non-profit websites under the conceptual heading of their hand search within their methodology, even though they use a structured search phrase to search those websites. This is because, unlike academic article databases, Google does not support article exporting, and it is not often feasible to manually copy-paste all google results into your article database. Instead, articles will need to be hand-selected by the researchers, which is the traditional hand search methodology.

    3. Guideline: "It is highly desirable that searches be conducted of appropriate national, regional and subject specific bibliographic databases" (Cohrane 2023 4-3-1-4)
    4. Guideline: Manage the efficiency and thoroughness of the search in a way that minimizes language and publication bias (Cochrane 2023 4-2-2)
  9. Develop a systematic search strategy
    1. form a master search which contains all of the concepts within your clinical question combined appropriately with boolean search commands
  10. translate your master search concepts into separate search phrases for each database to ensure that your search works the same despite different database "dialects"
    1. Use Boolean
      • Guideline: "Search strategies should avoid using too many different search concepts but a wide variety of search terms should be combined with OR within each included concept." (Cochrane 2023 4-2-2)
    2. Use subject headings in addition to keywords
      • Guideline: "Both free-text and subject headings (e.g. Medical Subject Headings (MeSH) and Emtree) should be used." (Cochrane 2023 4-2-2)
  11. Finalize and register your research protocol (including search strategy) in a community forum such as the Open Science Framework (OSF),
    1. or asking a specialist such as a Medical Librarian to peer review your search strategy(ies)
  12. Conduct searches
    1. Guideline: "Unlike for tasks such as study selection or data extraction, it is not considered necessary (or even desirable) for two people to conduct independent searches in parallel. It is strongly recommended, however, that all search strategies should be peer reviewed, before being run, by a suitably qualified and experienced medical/healthcare librarian or information specialist" (Cochrane 2023 section 4-2-2)
    2. track or use a tool that will record the dates of your searches, your exact search phrases, and any limiters or filters used
  13. Export search results
    1. tools such as Covidence and Rayyan can eliminate some file management labor
  14. Select studies for inclusion
    1. exclude articles which do not fit within your criteria
  15. Critically appraise included articles
  16. Extract and synthesize the data
    1. Systematic reviews typically analyze efficacy and other structural and methodological aspects of research studies, but it is also possible to extract more theoretical data.
  17. Write and publish your review

Resources and Examples:

Devon (your librarian) reviews a scoping review:

Maggio, LA, Larsen, K, Thomas, A, Costello, JA, Artino, AR. Scoping reviews in medical education: A scoping review. Med Educ. 2021; 55: 689– 700. https://doi.org/10.1111/medu.14431

  • Positives:
    • Methods section
      • Correctly cited a source which guides the construction of the methodology of scoping reviews
        • Arksey and OMalley, and Levac et al update
      • appropriately identified research questions
      • correctly attribute the construction of the search to the librarian
        • librarian is also appropriately included as an author in the article
      • their supplement with the search phrases does provide every search phrase used in its entirety together with limits used on the searches
        • However, No detail is given with regard to the search phrases in the body of the published article, readers need to consult a supplemental document
      • 2 authors reviewed each article, a third facilitated resolution of disagreements
      • Also had a shared learning process at the beginning of coding to establish coding consistency
      • Reference PRISMA in the creating of their extraction instrument, which they refer to as a “charting tool”
      • Also piloted “charting tool”, and modified tool based on experiences in the pilot
      • Again, a third author served to resolve disagreements
    • Results section
      • They provide reference numbers (citations) to each article included in each data point
      • Include a figure of all of the codes in their extraction instrument and the number of articles as a percentage as well as citations coded for each item
  • Negatives:
    • Methods
      • Limited search to 14 medical education journals
        • They do argue that they did this because the indexing of medical education is not comprehensive, so they “erred on the side of search strategy accuracy” rather than scope or depth
      • No detail is given with regard to the search phrases in the body of the published article, readers need to consult a supplemental
        • However, as a positive, their supplement with the search phrases does provide every search phrase used in its entirety together with limits used on the searches