Evaluating Databases & Sources

Critically evaluating the databases you are searching and the sources you are finding is an extremely important part of the research process. This page will provide you with skill for, and examples of, effectively evaluating scholarly information.

Question Everything (In 3 "Easy" Steps)

When you first search a database, whether it's Google, the library website, or a scholarly database, it's good to always ask the following 3 questions:

• What do you see?
• What is being privileged?
• What do you not see?

To understand this better, I did a Google search for the English novelist Jane Austen. After entering in her name, this is what I see on the results page:

• Over 44 million results
• An advertisement for an "Austen Dress"
• Links to wikipedia.org, biography.com, and other popular websites
• A list of frequently asked questions
• News stories about recent movie adaptations
• Images of illustrations, book covers, etc.

By outlining what we see we are able to the start to understand what Google privileges, which more times than not consists of links to very popular websites, advertisements and shopping options, media such as images and videos, news and popular culture, and very basic and quick contextual information.

Furthermore, by understanding what Google privileges, we might begin to make sense of the contours of what we don't see. For example, Google returned over 44 million results, but how do we make sense of that amount of information? Also, we don't know why Google is showing what it shows us or why certain results show up first while others do not. Indeed, Google works by a proprietary algorithm which the public does not have access to and its search engine is a money-making product that serves advertisements. All of this makes Google good for certain things, but it is clearly not the best place for scholarly research.

Google vs. the Library Website

Now that we've explored Google a bit, let's evaluate the library website by again asking our three questions. To start, after searching for "Jane Austen," what do we see?

• Over 52 thousand results
• Links to articles and books
• Facets (the categories on the left of the screen)

It becomes quickly apparent that the library website is privileging certain things, such as scholarly literature, a more manageable amount of results, and many ways for users to sort and make sense of those results. We also notice from the facets that each of the links/resources has been robustly described, making it easier for us to find out more about them, locate related resources, and browse and search in very smart complex ways.

What do we not see? While search results are displayed via algorithm, the collection of resources being searched were selected by librarians and reviewed by scholars. This means that expertise and consideration of research needs of the Northwestern community went into curating these resources, however, libraries are not neutral, so we do not see the biases of the people who made these decisions. In the end, neither of these are perfect resources, but the library website is a powerful tool tailored to doing scholarly research.

ACT UP

Whether evaluating a thread on Twitter or an article in a scholarly journal, following the steps of ACT UP will help get you started. (Adapted from "ACT UP: Evaluating Resources for Social Justice" by Dawn Stahura.)

• Authority
  _ Who (person, organization, company) created the source?
  _ What are their credentials/affiliation or experience that qualify them on this topic?
  _ What editorial process was used to disseminate this resource?
   
• Currency
  _ Does your topic require current information?
  _ When was the source written and published?
  _ Has the information been revised or updated recently?
   
• Truth
  _ What type of evidence is used to support claims or arguments?
  _ How accurate is the information? Can you verify the claims in other resources?
  _ Has the information been reviewed? Is there a bibliography?
   
• Unbiased

  _ Is the information presented to impact your emotions or your reason?
  _ Is the purpose to sell? To teach? To persuade? To entertain?
  _ Does the point of view appear objective or unbiased?
   

• Privilege
  _ Check the privilege of the author(s). Who is missing from the conversation?
  _ Critically evaluate the subject terms associated with the source: How are they described? What are their biases?
  _ What is the purpose of the publication? To sell, teach, persuade or entertain?

Peer-Review vs. Popular Publications

As mentioned above, Google privileges more popular resources, while on the library website many of the resources you will encounter will be academic, including books, articles, and more that have gone through "peer-review." Essentially, a book or article that has been peer-reviewed means that the author's work has gone through several layers of critical analysis by editors, publishers, and other scholars in their disciplines and related fields. More often than not, when looking for "academic sources" as part of your paper or project, you'll want to be sure they are peer-reviewed, and you can even sort your library website search results (see the "facets" to the left on the results page) that way to help you out!

For just one example of peer-review, check out the submission guidelines of the PMLA, an influential journal published by the Modern Language Association (MLA).

Citations & Serendipity

There are ways of searching and evaluating sources that are much different than doing a Google search or entering in keywords to the library website. One excellent tactic is to browse the sources and citations in the articles and books you find. Have you found a good article? Who is the author citing? What other resources can you find by browsing the footnotes in the text and the "references" section at the end of an a book or article? Not only will this help you locate additional relevant sources, but it will help you evaluate the sources you find by understanding who the author is in conversation with and how deeply they engage with the literature of their fields and disciplines.

Finally, never forget that some of the best things happen when you are least expecting it! If you find a book at the library, take time to look around at the other books on the shelves near it. Often times you'll end up leaving the library with several books that will help you better understand your topic, challenge your preconceptions, and enhance your argument.