Literature Reviews

Since you are situating your research within the larger scholarly conversation rather than summarizing everything that has been written about a topic, here are some questions that can help you decide what to include in your literature review:

• Who are the important voices in the conversation?
  • Authors who wrote about your topic or a similar topic
  • Most frequently cited works/authors on your topic
• Who identified the research gap that you seek to address?
• What conflicts are there about your topic?
• What has been most recently written about your topic?

Both systematic and semi-systematic literature reviews require you to establish a search methodology before conducting your literature review. You will need to identify:

• Where you will search (which databases, journals, other resources)
• How you will search (search terms, search strategies)
• What you will include/exclude from your results (specific criteria)

You will also want to take detailed notes as you search. See Library Books and Documenting and Organizing for resources to help guide you in your search.

It is integral to spend time honing and defining your research question before searching the literature. Here are a couple tools used for by particular science and social science disciplines to help you define your research question:

• PICOT / PICO (quantitative evidence-based research/synthesis) and
• SPIDER (qualitative evidence-based research/synthesis) 

PICO (Quantitative) and SPIDER (Qualitative)

Cooke, Smith, & Booth (2012).

*The "T" (PICOT) is left out of the above study. It represents Time, or the duration of data collection (Riva, Malik, Burnie, Endicott, & Busse, 2012)

Engineering PICO*

P = Population, Problem, Process

I = Intervention, Inquiry, Investigation, Improvement

C = Comparison (current practice or opposing viewpoints)

O = Outcomes (measuring what worked best)

*Read more about it on Arizona State University Library's "Engineering -- Formulating questions w/PICO" guide: https://libguides.asu.edu/engineering/PICO

Your research topic and type of literature review will help you determine where to look.

For literature reviews within a paper, you will likely at least want to search an important subject database and a citation tracking database.

• Subject Research Guides can help you identity important subject databases
• Web of Science and Google Scholar are citation tracking databases

For systematic/semi-systematic literature reviews, you will likely be more comprehensive in your search. In addition to the databases mentioned above, you may want to:

• Use the Library Search and "Expand Beyond Library" to search everything indexed by UND library databases and additional sources, such as open access materials
• Search WorldCat or/and Google Books, particularly for humanities disciplines
• Search government documents or other gray literature resources relevant for your discipline

Dissertations and Theses can also help you with a literature review, as these tend to include thorough literature reviews on a topic. Take a look at their literature review section and citations.

Search Strategies

Conventional Subject Searching in Databases

Subject database searching generally includes developing a search strategy around subject terms, reflecting aspects of the research question. You may want to use Booleans (AND/OR/NOT) and wild card operators (*/!) to help you create a thorough and precise search strategy. Searches are often restricted by language and date, and sometimes geographic region, through the use of database limiters.

Example research question and search strategy

Research question: Is there a correlation between fast food advertising and childhood obesity?

Prelude to developing a search strategy: How could that correlation be shown? Perhaps the number of ads by fast food companies over time and childhood obesity over time? How can I tell whether those ads target children? Perhaps if the ads include cartoons or toys or character mascots they can be considered to target children; perhaps previous research will help me identify additional methods, as well. What words could be used to describe "fast food," "advertising," "children" and "obesity"?

Initial search strategy: (kid* or child*) AND (market* OR advertis*) AND "fast food" AND (obesity OR weight OR fat)

Updated search strategy after initial search: (kid* or child*) AND (market* OR advertis*) AND ("fast food" OR "quick service")

Citation Chaining & Citation Searching (Backward & Forward Snowballing)

These techniques refer to checking reference lists and citing articles (articles that have cited the article that you are currently looking at). Citation chaining involves checking references on all included papers identified by various search methods so that relevant references not yet identified can be added to the pool of included studies. It also includes checking articles that cited an included paper. Many research databases link citing articles to each article record. Databases that are useful for citation searching include Google Scholar, Web of Science, CINAHL Complete, Wiley Online, and others. Access Chester Fritz Library's most used databases by visiting our home page and clicking on QuickLinks or the complete list by visiting our A-Z Databases page.

Traditional vs. Comprehensive Pearl Growing

Traditional Pearl Growing (TPG) begins with one or more target articles, judged to be such due to their relevancy to the research topic. The target article is called a pearl. It's a step beyond the citation chaining and searching methods. The researcher then identifies keywords to add to their search from aspects of the article (e.g., abstract, subject terms, author, etc.). Hawkins and Wagers (1982) coined this process as "growing more pearls" (as cited in Schlosser, Wendt, Bhavnani, & Nail‐Chiwetalu, 2006).

Comprehensive Pearl Growing (CPG) involves the following process: (1) Start with a compilation of studies from a relevant review or a topical bibliography; (2) determine relevant databases for these studies; (3) determine how these studies are indexed in database 1 in terms of keywords and quality filters; (4) find other relevant articles in database 1 (or as many are relevant) using the index terms in a Building Block query; and (5) end when articles retrieved provide diminishing relevance. Thus, rather than beginning with only one pearl, CPG requires of the searcher to begin with a compilation of studies from a relevant narrative review or a topical bibliography. Like TPG, CPG makes use of existing studies to determine the keywords and quality filters under which they are indexed in order to retrieve more articles of the same kind (Schlosser, Wendt, Bhavnani, & Nail‐Chiwetalu, 2006).

Although pearl growing techniques are effective across disciplines, they may be particularly strategic for interdisciplinary research questions in which multiple controlled vocabularies (e.g., thesauri, database subject terms, discipline-specific terminology), are integral to pulling together  sources across research databases (Schlosser, Wendt, Bhavnani, & Nail‐Chiwetalu, 2006).

Text-Mining

In Software Engineering, various text-mining (TM) techniques are used more and more to implement systematic literature review processes, however further research is needed--read Feng, Chiam, and Lo (2017) linked below for more information.

Document Your Literature Search

Use paper and pen, the below excel file, or online tools or applications like Trello to set up a system for documenting your search strategy. This contributes to research transparency and gives you a mechanism to provide quick and accurate documentation of your search strategies when pre-registering systematic review protocol or being questioned about how you searched the literature (and what you may have missed) by supervisors, colleagues, or reviewers.

For systematic reviews or meta-analyses, use the PRISMA or MOOSE checklists to evaluate each included resource for inclusion.

Organize Your Literature

Where Will You Look?

The literature you gather greatly depends upon the sources that you look in. Studies appearing in peer-reviewed journals are easy to locate but will likely over-represent significant and novel results, while certain types of grey literature (e.g., dissertations and theses; self-published manuscripts; unpublished studies; conference abstracts, presentations, and proceedings; regulatory data; unpublished trial data; government publications; reports such as white papers, working papers, and internal documentation; patents; and policies & procedures) may be more difficult to find and access in full text--for example, you may need to contact authors or organizations directly. It is good practice to use listserv and distribution lists for this type of material along with direct personal contacts, keeping in mind that the latter may bias the results towards those in support of a particular contact's central beliefs and research results (Cooper, 2010).

Obviously, this means that limiting your search to journals in databases may skew results towards statistically significant findings, biasing your pool of studies which would be lacking in null, or inconclusive, results. You can also search for grey literature in institutional repositories like UND Scholarly Commons, government/professional organizations and conference websites, Open Data Repositories, open preprint repositories, theses and dissertation databases, online Researcher Communities, and journals that publish Registered Reports or null and inconclusive findings like PLOS ONE.

Author's Versions & Grey Literature Database Examples: