Stumped? 5 steps to find the latest evidence
Shortcuts and online resources make searches less confusing, easier to apply to practice
When grappling with difficult cases, clinicians often wonder, “What does the evidence say, and how can I find it?” Thanks to evidence-based medicine (EBM), study results are easier to track down and apply to patient care.
From our experience, we tell how to use EBM’s 5 steps and offer tips and shortcuts to help you quickly find the evidence you need.
SPEEDING RESEARCH TO PRACTICE
Health care is rich in evidence-based innovations, but successful innovations are often disseminated slowly—if at all—to practitioners. 1 Studies that demonstrate effective treatments for mental disorders may take considerable time to find their way into clinical practice. 2 Similar delays have been noted in other specialties, depriving patients of the most up-to-date medical treatments. Academic medical center surveys show that up to 40% of clinical decisions are not supported by the literature. 3,4
Evidence-based psychiatry: Teaching tool makes leap to clinical practice
Faculty at McMaster University in Canada developed evidence-based medicine as a medical education model by which physicians would rely on the literature—rather than textbooks or tradition—to solve patient problems. 7 EBM’s influence on psychiatry has been greatest in the United Kingdom through efforts of its National Health Service, the Centre for Evidence-Based Mental Health at the University of Oxford, and the journal, Evidence-Based Mental Health. 5
The U.S. experience. Given managed care’s influence on American medicine, one might expect EBM to have had a similar effect in the United States. This has not been the case, however, perhaps because for-profit HMOs have focused more on controlling costs rather than improving quality.
The U.S. Agency for Healthcare Research and Quality has funded evidence-based practice centers that produce systematic reviews but has done relatively little to promote EBM teaching and practice. Instead, professional organizations such as the American College of Physicians and journals such as JAMA have taken the lead in teaching physicians about EBM.
Residents learning EBM. Most EBM instruction occurs in medical school primary care departments. This is changing, however, now that the Accreditation Council for Graduate Medical Education (ACGME) requires all residents to show competence in EBM methods. 8 ACGME requires residency programs in all specialties—including psychiatry—to ensure that graduates achieve EBM core competencies, including being able to use the process described in this article.
Two problems that daunt psychiatrists and other physicians—information overload and uncritical acceptance of information—contribute to less-than-optimal care. 5
Information overload. With thousands of medical journals and millions of articles being published, no clinician can keep up with all developments in his or her field. Furthermore, study results often appear contradictory.
Review articles summarize the literature, but most are “journalistic” or “narrative”—not systematic reviews—and thus are subject to author bias in the studies cited and methods used to summarize conflicting results. 4 Textbook chapters have the added problem of being almost immediately out-of-date. 6
Two approaches could narrow the gap between research and practice:
- clinical practice guidelines and pathways, a “top-down” approach favored by administrators
- evidence-based medicine, a “bottom-up” approach favored—and developed by—medical educators (Box 1).5,7,8
EBM and patient care. EBM has been described as “partly a philosophy, partly a skill, and partly…a set of tools.” 9 David Sackett—often considered the father of EBM—has defined it as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.” 10
Although some critics disagree, clinical expertise plays an important role in EBM, as the clinician must integrate research evidence, patient preferences, and the patient’s clinical condition when making decisions. 11
EBM’S 5 ACTION STEPS
With EBM, evidence from the medical literature is applied to patient care through 5 action steps:
- Formulate the question.
- Search for answers.
- Appraise the evidence.
- Apply the evidence to the patient.
- Assess the outcome. 5,6,12
Step 1: Formulate the question. EBM begins with a clinical question related to the diagnosis, treatment, prognosis, or cause of a patient’s illness. The question is formatted to include the patient’s problem or diagnosis, the intervention of interest and any comparison intervention, and the outcome of interest.
Take, for example, this question: “In patients with bipolar disorder, is lamotrigine as effective as lithium in preventing relapse?” In this format:
- bipolar disorder is the diagnosis
- lamotrigine is the intervention of interest
- lithium is the comparison intervention
- relapse is the outcome of interest.
Step 2: Search for answers. Because answering different types of questions requires different types of evidence, this step involves determining the most appropriate type of evidence and searching for it. 13
Evidence hierarchies (Table 1) 5,6,12,14 indicate that certain types of evidence are considered more credible than others. The higher the level of evidence, the more likely it is to provide valid, unbiased estimates of an intervention’s effect.
Hierarchy for studies of therapy or harm
Type of evidence
Systematic review of randomized controlled trials (RCTs)
Individual RCT with narrow confidence interval
Systematic review of cohort studies
Individual cohort study RCT with <80% follow-up
Outcomes research Ecological study
Systematic review of case-control studies
Individual case-control study
* Highest- to lowest-quality evidence
Source: Adapted from references 5,6,12, and 14
For example, observational studies may give misleading results, compared with randomized, controlled, clinical trials. 15 Expert opinion—which does not necessarily reflect the best evidence in the literature—is considered the lowest evidence level. 16
Beginning the literature search with Medline—as many clinicians do—is relatively inefficient. Searching the massive National Library of Medicine database often identifies a large number of articles, which you then must appraise for validity.
More efficient is Haynes’ “4S” strategy of:
It gives priority to sources of high-quality, pre-appraised information ( Table 2), so that you may omit Step 3—appraisal—in the EBM process.
“4S” approach to answering clinical questions
Systems (comprehensive sources)
National electronic Library for Mental Health
APA Practice Guidelines
National Guideline Clearinghouse
Synopses (structured abstracts)
Evidence-Based Mental Health
ACP Journal Club
Syntheses (systematic reviews)
Cochrane Database of Systematic Reviews
Database of Abstracts of Reviews of Effectiveness (DARE)
Health Technology Assessment (HTA) database
Studies (original articles)
Source: Adapted from references 12 and 17.
Systems. A system is an information source that covers a variety of diagnoses, summarizes the results of high-quality systematic reviews, is frequently updated, and is linked to the original studies. Three examples are:
- Clinical Evidence, a journal published semi-annually in the United Kingdom that is also available online (www.clinicalevidence.com). Its ease of use, frequent updates, and clear links to the best evidence make it a good starting point.
- National electronic Library for Mental Health (NeLMH) (www.nelmh.org), created by the U.K.’s Centre for Evidence-Based Mental Health. 18 It provides information only about depression, schizophrenia, and suicide but may become the model for evidence-based psychiatry.
- Evidence-based clinical practice guidelines developed by the American Psychiatric Association 9 and other organizations. Many of these are included in the National Guideline Clearinghouse (www.guideline.gov), produced by the U.S. Department of Health and Human Service’s Agency for Healthcare Research and Quality.
Synopses. If a system does not answer a clinical question, look for synopses—structured abstracts of high-quality systematic reviews or original articles. Synopses are brief and pre-appraised for quality, allowing you to quickly get the point without reading a lengthy article. For psychiatry, Evidence-Based Mental Health and ACP Journal Club are the best synopses sources.
Syntheses. If you don’t find a synopsis to answer your question, then search for a synthesis—a high-quality systematic review. The best source is the Cochrane Database of Systematic Reviews; 20 others include:
- Database of Abstracts of Reviews of Effectiveness (DARE)
- Health Technology Assessment (HTA) database.
Web sites for learning more about EBM
Centre for Evidence-Based Medicine (Oxford)
Centre for Evidence-Based Medicine (Toronto)
Centre for Evidence-Based Mental Health
EBM Education Center of Excellence
Evidence-based Medicine Resource Center
Netting the Evidence
University of Sheffield Health Informatics
Users’ Guides to the Medical Literature
Studies. Only if the first three “S’s” fail to yield an answer would you then search Medline or similar databases for applicable studies.
The PubMed Clinical Queries interface lets you specify the type of question (therapy, diagnosis, etiology, or prognosis) and whether the search is to be sensitive or specific. You can also use filters built into this interface when searching for systematic reviews.
Step 3: Appraise the evidence. After you find an article, appraise its validity and importance. Checklists for appraising studies may be found in EBM texts 4,6,12 or downloaded from Web sites (Table 3). Specific appraisal questions to ask depend on the study design and your clinical question.
Step 4: Apply the results to your patient. Assuming the evidence is valid, important, and feasible in your setting, consider your patient’s preferences 21 and apply the results.
Step 5: Assess the outcome. Evaluate your performance in searching the literature, and assess the patient’s response.
STREAMLINING THE EBM PROCESS
Studies in academic settings have shown that the full 5-step model can be incorporated into daily practice. 6,12 In nonacademic settings, however, practitioners complain about lack of time and information resources, as well as inadequate EBM skills. 11,12 Two shortcuts can streamline the process:
Use the 5-step process selectively; it is not required for every patient encounter. 6,12 After you research a question for one patient with a particular diagnosis, you can apply the answer to similar patients. Because most psychiatrists’ patients fall into relatively few diagnostic categories, only the exceptional patient would trigger the full 5-step process.
Use pre-appraised information sources. With online databases such as Clinical Evidence and Evidence-Based Mental Health, you can find information quickly, often in the time it takes to use textbooks. 5,12,17
Excellent EBM texts 4,6,12 and online resources may be useful (Table 3). Courses are listed on the EBM Education Center of Excellence Web site, and the Centre for Evidence-Based Mental Health at Oxford University offers an outstanding course for psychiatrists.
- Gray GE. Concise guide to evidence-based psychiatry. Washington, DC: American Psychiatric Publishing, 2004.
- Greenhalgh T. How to read a paper: the basics of evidence-based medicine (2nd ed). London, BMJ Books, 2001.
- Guyatt G, Rennie D (eds). Users’ guides to the medical literature: a manual for evidence-based clinical care. Chicago: AMA Press, 2002.
- Sackett DL, Straus SE, Richardson WS, et al. Evidence-based medicine: how to practice and teach EBM (2nd ed). New York: Churchill Livingstone, 2000.
Supported in part by a grant from the National Institute of Mental Health (5-R24-MH61456-03).
1. Berwick DM. Disseminating innovations in health care. JAMA 2003;289:1969-75
2. Drake RE, Goldman HH, Leff HS, et al. Implementing evidence-based practices in routine mental health settings. Psychiatr Serv 2001;52:179-82
3. Geddes JR, Game D, Jenkins NE, et al. What proportion of primary psychiatric interventions are based on evidence from randomised controlled trials? Qual Health Care 1996;5:215-17.
4. Greenhalgh T. How to read a paper: the basics of evidence based medicine (2nd ed). London: BMJ Books, 2001.
5. Gray GE. Evidence-based medicine: an introduction for psychiatrists. J Psychiatr Pract 2002;8:5-13.
6. Sackett DL, Straus SE, Richardson WS, et al. Evidence-based medicine: How to practice and teach EBM (2nd ed). New York: Churchill Livingstone, 2000.
7. Evidence-Based Medicine Working Group. Evidence-based medicine: a new approach to the teaching of medicine. JAMA 1992;268:2420-5.
8. Accreditation Council for Graduate Medical Education General competencies [ACGME Outcome Project Web site], 2001. Available at: http://www.acgme.org/outcome/. Accessed May 5, 2004.
9. Dawes M. Preface. In: Dawes M, Davies P, Gray A, et al (eds). Evidence-based practice: a primer for health professionals New York: Churchill Livingstone, 1999.
10. Sackett DL, Rosenberg WMC, Gray JAM, et al. Evidence-based medicine: what it is and what it isn’t. BMJ 1996;312:71-2.
11. Straus SE, McAlister FA. Evidence-based medicine: a commentary on common criticisms. Can Med Assoc J 2000;163:837-41.
12. Gray GE. Concise guide to evidence-based psychiatry Washington, DC: American Psychiatric Publishing, 2004.
13. Glasziou P, Vandenbroucke J, Chalmers I. Assessing the quality of research. BMJ 2004;328:39-41.
14. Phillips B, Ball C, Sackett D, et al. Centre for Evidence-Based Medicine: Levels of evidence and grades of recommendations, May 2001. Available at: http://www.cebm.net/levels_of_evidence.asp. Accessed Sept. 18, 2004.
15. Lacchetti C, Guyatt G. Therapy and validity: surprising results of randomized controlled trials. In: Guyatt G, Rennie D (eds). Users’ guides to the medical literature: a manual for evidence-based clinical care. Chicago: AMA Press, 2002.
16. Antman EM, Lau J, Kupelnick B, et al. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts: treatments for myocardial infarction. JAMA 1992;268:240-8.
17. Haynes RB. Of studies, summaries, synopses, and systems: the “4S” evolution of services for finding current best evidence. Evid Based Ment Health 2001;4:37-9.
18. Dearness KL, Tomlin A. Development of the National Electronic Library for Mental Health: providing evidence-based information for all. Health Info Libr J 2001;18:167-74.
19. American Psychiatric Association. Practice guidelines for the treatment of psychiatric disorders: compendium 2002. Washington, DC: American Psychiatric Publishing, 2002.
20. Antes G. Oxman AD for the Cochrane Collaboration. The Cochrane Collaboration in the 20th Century. In: Egger M, Smith GD, Altman DG (eds). Systematic reviews in health care: meta-analysis in context London: BMJBooks, 2001.
21. Say RE, Thomson R. The importance of patient preferences in treatment decisions—challenges for doctors. BMJ 2003;327:542-5.