Skip to Main Content

Introduction to Literature Search

This guide is intended to provide you with the basic skills necessary to search for literature. The skills are presented broadly and can be applied across all databases.

Analyzing and Appraising the Literature 

Not all research is created equal or trustworthy. You want to ensure that your research paper includes the best possible evidence. You must put your collected evidence through a process of critical appraisal before including it in your final work.

Appraisal of Evidence

Type of Evidence

 Description
Systematic Review

Authors have systematically searched for, appraised, and summarized all of the literature for a specific topic

A meta-analysis is a systematic review that uses quantitative methods to summarize the results
Critically Appraised Topics Authors evaluate and synthesize multiple  research studies
Critically Appraised Articles Authors evaluate and synthesize individual research studies 
Randomized Controlled Trials Features a randomized group of patients in both an experimental group and a control group. The groups are followed up to assess variables/outcomes of interest
Cohort Study Identities two groups (cohorts) of patients, one which received the exposure of interest, and one which did not, and follows these cohorts forward for the outcome of interest
Case Control Study Identities patients who have the outcome of interest (cases)and control patients without the same outcome, and examines for exposure of interest
Background Information/Expert Opinion A summary of information that is not necessarily back by research studies. The author(s) is usually a credentialed expert on the particular subject/topic. Can include point-of-care resources, textbooks, and encyclopedias. These provide a good foundation/introduction/overview on a specific conditions, drug, disease, etc. ​

Adapted from Evidence Based Practice Toolkit from Winona State University.

Accuracy 
Just because information is published in a book, journal or on the web, does not guarantee that it is accurate.

Questions to Ask:

  • Is the information referenced so that you can verify the accuracy? Are citations and sources listed?
  • Is the information reliable or free from error?
  • Is there evidence to support conclusions?
  • Was the information reviewed by editors or subject experts before it was published?
Authority
Authority is the source of the information. A good authority will provide information to verify who they are and include sufficient credentials or contact information.
Questions to Ask:
  • Who is the author? What are the credentials of the author? With whom are they affiliated with? Does their affiliation affect their credibility?
  • Is the author an expert on the subject?
  • Does the author have contact information available?
  • Who is the publisher? Is the publisher reputable?
  • Is the publisher scholarly? Or is the publisher commercial, a self-publisher or government? What else has the publisher published?
Objectivity 
It is important to keep in mind that all sources have the potential to have a bias no matter what format (book, article or website) and no matter who is responsible for the material (author, publisher or funding source),
Questions to Ask:
•What is the purpose? What are the authors trying to accomplish?
•What topics are covered? 
•Is the information fact or opinion?
•Is there a particular position on the subject? Can you detect a bias? 

Currency

The currency of a resource is important because information from older sources may be outdated, inaccurate or superseded by more recent publications.

Questions to Ask:

  • When was the information created or last updated? Is there a date for when the information was last revised?
  • Is the information still valid? Has it been superseded by newer research? 
  • On websites, do the links still work? If not, it may be an indication that the information has not been updated recently.

Coverage 

Think about if the information adequately covers the subject. Look for a statement describing the purpose of the source.

Questions to Ask:

  • Is the information relevant to your assignment?
  • Who was it written for? Was it written for a general audience, consumers or people in a specific profession?
  • Is the information complete? How in depth is the information?

CRAAP is an acronym for the general categories of criteria that can be used to evaluate information you find. Developed at the Meriam Library at California State University-Chico, the CRAAP Test is a list of questions to keep in mind as you evaluate your sources.

Currency: the timeliness of the information

Relevance: the importance of the information for your needs

Authority: The source of the information

Accuracy: The reliability, truthfulness, and correctness of the content

Purpose: The reason the information exists

Quality Assessment

For any study that you wish to include in your paper, you must first assess its quality and validity. This critical evaluation will allow you to:

  • assess the benefits and strengths of the research against its flaws and weakness
  • determine if the study has been undertaken in a method that produced reliable findings
  • the results make sense -- the math adds up
  • know what these results mean in the context of the decision/conclusions that are being proposed
  • assess the usefulness of the evidence for informing future clinical decisions 

There are three questions that you should apply to any study:

  1. What are the results?
  2. What is the risk of bias in the study?
  3. Will the results help me in caring for my patients?

There are a number of checklists that you can use to assist with determining the quality of a study. Below are some recommended resources.

Validity Questions to consider for Systematic Reviews 

1. Did the review explicitly address a sensible question?

2. Was the search for relevant studies detailed and exhaustive?

3. Was the risk fo bias of the primary studies addressed?

4. Was the selection and assessment of studies reproducible?

Adapted from: Guyatt, GH. Users' guides to the medical literature: a manual for evidence-based clinical practice. 3rd edition. McGraw-Hill, 2015.

Validity Questions to consider for Therapy articles

1. Were the patients randomized?

2. Was group allocation concealed?

3. Were patients in the treatment and control groups similar with respect to known prognostic factors?

4. Were 5 important groups (patients, care givers, collectors of outcome data, adjudicators of outcome, and data analysts) aware of group allocation?

5. Outside of experimental intervention, were the groups treated equally?

6. Was follow-up complete?

7. Were patients analyzed in the groups to which they were randomized?

Key issues for therapy studies -- FRISBE

  • Follow up
  • Randomized/concealed allocation
  • Intention to treat
  • Similar at baseline
  • Blinding
  • Equal treatment

Adapted from: Guyatt, GH. Users' guides to the medical literature: a manual for evidence-based clinical practice. 3rd edition. McGraw-Hill, 2015.

Validity Questions to consider for Diagnostic studies

1. Did the clinicians face diagnostic uncertainty?

2. Did investigators compare the test to an appropriate, independent reference standard?

3. Were those interpreting the test and reference standard blind to the other result?

4. Did the investigators perform the same reference standard to all the patients regardless of the results of the test under investigation?

Adapted from: Guyatt, GH. Users' guides to the medical literature: a manual for evidence-based clinical practice. 3rd edition. McGraw-Hill, 2015.

Validity Questions to consider for Prognosis studies

1. Was the sample of patents representative at a well-defined point in the course of the disease or disorder?

2. Were the patients sufficiently similar with respect to prognostic factors?

3. Was the follow up complete? 

4. Were outcomes criteria objective and unbiased?

Adapted from: Guyatt, GH. Users' guides to the medical literature: a manual for evidence-based clinical practice. 3rd edition. McGraw-Hill, 2015.

Validity Questions to consider for Harm/Etiology studies

Includes what to consider for a COHORT STUDY  and a CASE CONTROL STUDY

1. In a COHORT STUDY did the exposed and control group start and finish with the same risk for outcome?

  • Were the patients similar for prognostic factors that are known to be associated with the outcome?
  • Were the circumstances and methods for detecting the outcome similar?
  • Was the follow up sufficiently complete?

2. In a CASE CONTROL STUDY  did the cases and control group have the same risk (chance) of being exposed in the past?

  • Were the cases and controls similar with respect to the indication or circumstances that would lead to exposure?
  • Were the circumstances and methods for determining exposure similar for cases and controls?

Calculate the odds ratio (OR) -- a measure of association between an exposure and an outcome. Represents the odds that a person with the outcome was exposed compared to the odds that a person without the outcome was exposed.

Adapted from: Guyatt, GH. Users' guides to the medical literature: a manual for evidence-based clinical practice. 3rd edition. McGraw-Hill, 2015.

For THERAPY STUDIES

Experimental Event Rates (EER): the outcome present/total in an experimental group

Control Event Rate (CER): the outcome present/total in control group

Absolute Risk Reduction (ARR): an absolute comparison of risks. Tells you how much lower the modified risk is than the starting risk. Subtracts the event rates. control/placebo group (starting risk) - experimental group (modified risk)

Absolute Risk Increase (ARI): the arithmetic difference the rates of events in the experimental and control groups. EER - CER

Relative Risk Reduction (RRR): a relative comparison of risks. Tells you how much lower the modified risk is relative to the starting risk. Absolute risk reduction / control group (starting risk)

Relative Risk Increase (RRI): the proportional difference in risk between the rates of events in the control group and the experimental group. RRI is usually a larger number than the ARI . CER - EER / CER

Number Needed to Treat (NNT):  the number of patients that need to be treated with the experimental treatment compared to the control treatment for 10 years in order ot have one patient experience a good outcome. 1 / absolute risk reduction (ARR)

Numbers Needed to Harm (NNH): the number of patients that a clinician would have to treat with the experimental treatment over the specific period of time to report one additional patient with an adverse outcome. Numbers Needed to Harm (NNH) = 1 / absolute risk increase (ARI)

For COHORT STUDIES

Calculate the risk ratio/relative risk (RR) -- the ratio of the probability of an outcome occurring in an exposed group compared ot the probability of the outcome occurring in a non-exposed group

For CASE CONTROL STUDIES

Calculate the odds ratio (OR) -- a measure of association between an exposure and an outcome. Represents the odds that a person with the outcome was exposed compared to the odds that a person without the outcome was exposed.

Adapted from: Guyatt, GH. Users' guides to the medical literature: a manual for evidence-based clinical practice. 3rd edition. McGraw-Hill, 2015.

Is it original research?

How do you tell if an article is considered original research? Signs that is include...

  • it is the report of a study written by the researchers who actually conducted the study
  • the researchers describe their hypothesis or research question and the purpose of the study
  • the researchers outline their research methods
  • the results are reported
  • the researchers interpret their results and discuss possible implications

Other key indicators that will help you determine if your article is based on original research include the inclusion of...

  • Methods section
  • Results section
  • Discussion

A quick skim of an article will reveal if these sections are present. Sometimes the article's abstracts will include summaries that correspond to these key sections.