Skip to Main Content

DataBank: Article Level Metrics & Citation Analysis

Recommended For

All Disciplines     

Concerns

"Manipulating Google Scholar Citations and Google Scholar Metrics: simple, easy and tempting" Delgado-López-Cózar, Emilio; Cabezas-Clavijo, Álvaro. 2012.

“Google Scholar Metrics: an unreliable tool for assessing scientific journals” Delgado-López-Cózar, Emilio; Cabezas-Clavijo, Álvaro. El profesional de la información, 2012.

"Google's Book Search: A Disaster for Scholars" Geoffrey Nunberg, Chronicle of Higher Ed, 2009.

Introduction - Google Scholar

Since it is difficult to know which publications and dates are in Google Scholar, and because of serious errors in its counts (see "Limitations" below) results in Google Scholar must be carefully examined and it should only be used as a supplement to other tools.

Look for the link to "Cited By" which will identify works that have cited the document originally retrieved in the search. Be certain to examine the cited works themselves to eliminate duplicate counts (some cited by citations are listed and counted more than once) and other errors.

Tips - Google Scholar

To search for particular publications:
(2) Enter author's last name and, in "exact phrase" box, the title of the work
(3) Works producing zero results should be re-searched via author's name in quotes plus a publication year limitation.
Cut/pasting and searching by author/title provides quality control and the results are easier to review.

To search for all of the works by an author:
(2) Enter author's last name and first initial in quotes in author box.
(3) On the results page, change two of the three limitations boxes to read "Article excluding Patents" and "since year."
Viewing the results year by year makes the results more manageable in terms of duplication and author verification.  Nonetheless, searchs per title (above) are more reliable than by author.


Alternatively, Google Scholar Citations allows authors to set up their own profile, which is automatically updated, and which is more manageable and possibly less labor than searching by author or article title.

Limitations - Google Scholar

There are arguments in the literature as to whether or not Google Scholar is a better citation tracker than ISI Thomson Reuters products, e.g., Web of Science. This guide concludes that it is not, that it is less accurate and less reliable. For those looking for a cost-free alternative to ISI products, this guides recommends Microsoft Academic Search.

NOTE: Due to its popularity, it is important to note that there are potentially important problems with using Google Scholar as a citation analysis tool. Here are a few:

  • it can mistake a reviewer's name to be the author of a work
  • it can mistake the title of a journal to be the title of an article
  • it can mistake a partial book title to be an article title
  • it cannot always distinguish a page number from a date
  • we do not know the universe, but we know that not all scholarly journals or books are included
  • many non-scholarly works are included, such as student handbooks, administrative pieces, study guides, syllabi, conference programs, repositories, unpublished items, and more
  • it does not cover older materials as well as Web of Science data
  • and as a result of the above, it makes calculation errors. Google Scholar results should be carefully examined and it should be used only in conjunction with other tools.

Many of these problems and more are replicated at the level of journal evaluation.

Introduction - Google Books

To get a count of journal articles that appear in books is unusual among citation tools and particularly relevant to Humanities and Social Sciences.  Google Books can be used to find citations within books to articles in journals. This may very usefully supplement counts of article citations that appear in other articles. 

Use the “Advanced Book Search” and enter the full title of the article in “with the exact phrase.” Across from “Search” keep the default at “all books.” Change the default of “Content” to “books” only. 

Limitations - Google Books

  • it may undercount by not finding citations in books that are in-copyright, because the full-text of these books is not accessible in the database. Although the Google Book Project reportedly has as its aim to scan almost everything, that does not mean that everything it scans is available for viewing. Google Books has available for free viewing the full-text of out-of-copyright books from over 20 major research libraries. It also provides partial text of in-copyright books. It is not at all clear how or how much of the citations in all scanned books are being counted or counted correctly.
  • results need to be reviewed because it will include citations within ephemeral and other material