For any study that you wish to include in your paper, you must first assess its quality and validity. This critical evaluation will allow you to:
There are three questions that you should apply to any study:
There are a number of checklists that you can use to assist with determining the quality of a study. Below are some recommended resources.
Users' Guides to the Medical Literature: a Manual for Evidence-Based Clinical Practice, 3E by
Publisher's Note: Products purchased from Third Party sellers are not guaranteed by the publisher for quality, authenticity, or access to any online entitlements included with the product. The leading guide to the principles and clinical applications of evidence-based medicine--updated with new examples and new chapters A Doody's Core Title for 2019! Revised and updated to reflect the latest in medical research and evidence-based resources Practical focus and organization to guide clinicians through the fundamentals of using the medical literature to the more advanced strategies and skills for use in patient care, using the key questions to assess any new research: What are the results? How serious is the risk of bias? How do I apply the results to patient care? Updated real-world examples drawn from the medical literature New chapters that reflect the changing complexity of medical research and literature, including genetic association studies, systematic reviews and meta-analyses, network meta-analysis, noninferiority trials, quality improvement, and evidence-based medicine and the theory of knowledge New emphasis on understanding the role of patient preferences and preappraised resources that provide updated evidence and evidence-based recommendations for practice
1. Did the review explicitly address a sensible question?
2. Was the search for relevant studies detailed and exhaustive?
3. Was the risk fo bias of the primary studies addressed?
4. Was the selection and assessment of studies reproducible?
1. Were the patients randomized?
2. Was group allocation concealed?
3. Were patients in the treatment and control groups similar with respect to known prognostic factors?
4. Were 5 important groups (patients, care givers, collectors of outcome data, adjudicators of outcome, and data analysts) aware of group allocation?
5. Outside of experimental intervention, were the groups treated equally?
6. Was follow-up complete?
7. Were patients analyzed in the groups to which they were randomized?
Key issues for therapy studies -- FRISBE
1. Did the clinicians face diagnostic uncertainty?
2. Did investigators compare the test to an appropriate, independent reference standard?
3. Were those interpreting the test and reference standard blind to the other result?
4. Did the investigators perform the same reference standard to all the patients regardless of the results of the test under investigation?
1. Was the sample of patents representative at a well-defined point in the course of the disease or disorder?
2. Were the patients sufficiently similar with respect to prognostic factors?
3. Was the follow up complete?
4. Were outcomes criteria objective and unbiased?
Includes what to consider for a COHORT STUDY and a CASE CONTROL STUDY
1. In a COHORT STUDY did the exposed and control group start and finish with the same risk for outcome?
2. In a CASE CONTROL STUDY did the cases and control group have the same risk (chance) of being exposed in the past?
Calculate the odds ratio (OR) -- a measure of association between an exposure and an outcome. Represents the odds that a person with the outcome was exposed compared to the odds that a person without the outcome was exposed.
Experimental Event Rates (EER): the outcome present/total in an experimental group
Control Event Rate (CER): the outcome present/total in control group
Absolute Risk Reduction (ARR): an absolute comparison of risks. Tells you how much lower the modified risk is than the starting risk. Subtracts the event rates. control/placebo group (starting risk) - experimental group (modified risk)
Absolute Risk Increase (ARI): the arithmetic difference the rates of events in the experimental and control groups. EER - CER
Relative Risk Reduction (RRR): a relative comparison of risks. Tells you how much lower the modified risk is relative to the starting risk. Absolute risk reduction / control group (starting risk)
Relative Risk Increase (RRI): the proportional difference in risk between the rates of events in the control group and the experimental group. RRI is usually a larger number than the ARI . CER - EER / CER
Number Needed to Treat (NNT): the number of patients that need to be treated with the experimental treatment compared to the control treatment for 10 years in order ot have one patient experience a good outcome. 1 / absolute risk reduction (ARR)
Numbers Needed to Harm (NNH): the number of patients that a clinician would have to treat with the experimental treatment over the specific period of time to report one additional patient with an adverse outcome. Numbers Needed to Harm (NNH) = 1 / absolute risk increase (ARI)
Calculate the risk ratio/relative risk (RR) -- the ratio of the probability of an outcome occurring in an exposed group compared ot the probability of the outcome occurring in a non-exposed group
Calculate the odds ratio (OR) -- a measure of association between an exposure and an outcome. Represents the odds that a person with the outcome was exposed compared to the odds that a person without the outcome was exposed.
Adapted from: Guyatt, GH. Users' guides to the medical literature: a manual for evidence-based clinical practice. 3rd edition. McGraw-Hill, 2015.
