Scholarly impact is essentially the influence a researcher’s work has on their field and beyond. It measures how much their research shapes knowledge, informs other studies, or is used in practice. It might reflect the influence a researcher has had on the development of a field; non-academic influence that relates to policies, industry, or public understanding; recognition by peers and professional societies; or most commonly, metrics like citation scores.
But how can scholarly impact be measured?
There are many ways. Some focus on citation metrics, others on journals. They could examine collaborative networks, look at different awards and recognitions received.
Really, there's no one way to measure the impact of a scholar's impact on the world. But there are many tools out there that can help give an idea of the ways that academic scholarship has a lasting influence.
The guide below will explore three below, and help to explain what they are measuring and what limitations each might have.
One tool used to assess a scholar's impact is Web of Science, a database that Foley Library makes available to the Gonzaga community. In addition to indexing literature across science, the social sciences, arts, and humanities. It allows users to explore connections between articles, topics, and scholars, as well as track citations, discover trends, and measure research impact.
Using Web of Science, we can dive deeper into President Passerini's impact as a scholar over the course of her career. The first thing we can see are the fields that her Web of Science indexed publications fall into:
These are the top five fields that President Passerini has published in but looking at the Web of Science report linked above, you can see that she has also published in other fields like Communication, Multidisciplinary Psychology, Education and Educational Research, and more.
Web of Science can also show a scholar's impact in citation metrics.
Dr. Passerini has published in a wide variety of topics and fields throughout her academic career. Of her 19 listed publications on Web of Science, she has received a total of 375 citations, averaging over 19 citations per item, and has an H-Index of 9.
An H-Index is a metric used to assess the productivity and citation impact of a researcher. The index indicates that of a scholars' publications, H papers have received at least H citations. President Passerini's H-index of 9 from Web of Science indicates that at least 9 of her papers indexed by the service have at least 9 citations each.
Visualized, we can see how even though President Passerini's publishing activity stays pretty steady, her citations keep increasing, indicating that her work continues to gain influence and visibility over time, reaching a wider audience and broadening her impact.
But Web of Science, while helpful, has some limitations. It will only count the publications that are included in an institutional subscription. So while it lists 19 publications for President Passerini, it doesn't give us a full view of her scholarship or impact over the course of her career.
For a more complete assessment of President Passerini's impact as a scholar, let's look at Google Scholar.
Google Scholar provides both research profiles and the ability to search scholarly literature across numerous fields and disciplines. Like Web of Science, it is another tool that can help you assess someone's impact as an academic researcher. It provides similar information to Web of Science, but not always in the same way.
Instead of assessing the scholarship and determining the fields that a researcher has published in, Google Scholar lets each researcher list their areas of interest in their profile, as President Passerini has done in hers, listing knowledge management, information systems, and more.
Then, clicking on the links will direct users to the researcher profiles with the highest number of citations in that area.
CitationsWhile Web of Science only listed 19 of President Passerini's publications, Google Scholar has indexed almost 146 unique titles with almost 5000 citations, more than 1500 in the last five years alone.
Google Scholar lists her h-index as 25, meaning that 25 publications have at least 25 citations each. But additionally, Google Scholar has also listed an i10-index, which is a metric created by them to represent the number of publications that have received at least 10 citations.
But Google Scholar also highlights the number of citations each publication has received. Clicking on the title of the publication will take users to information on individual item, which includes the basic bibliographic information, a link to the item on the web (if available), and the ability to see the citation history since the time of publication. This allows for deep dives into the individual citation metrics for each published piece, and helps to evaluate the reach and ongoing influence of a scholar’s work, track patterns of engagement over time, and identify which publications have had the greatest impact within the field.
ImpactOne way that Google Scholar helps to assess a scholar's impact is by listing co-authors that a scholar has published with throughout their career. Looking at a scholar’s coauthors helps show their impact in ways that citation counts alone cannot. Coauthor patterns reveal how widely a researcher collaborates, whether they work across different fields, and how connected they are within their academic community. They can also help to highlight the different roles a scholar might play, like mentorships with students or partnerships with early-career researchers, and show ties to well-known or highly cited researchers. Mapping their research network as well as exploring their individual work can help give a more complete understanding of a scholar’s influence.
While Google Scholar might offer a more complete picture of a scholar's impact, it's not perfect either. Not all of a scholar's works might be available on the web, which is how Google Scholar indexes them. Its indexing parameters are also very broad, meaning that it doesn't always discern between scholarly works and non-scholarly works like blog posts, presentations, etc. It's metrics, too, can have problems. Google Scholar will often count every mention of a work as a citation, whether that's an accurate representation of how the item was referenced or not. Additionally, there's a lack of transparency with Google Scholar and it's algorithms and indexing criteria.
Finally, let's look at Pure, which is a researcher profile and research information management system (RIMS) provided to Gonzaga faculty by Foley Library.
Pure is a platform from Elsevier that has many uses, including the creation and dissemination of researcher profiles for Gonzaga faculty. It helps our institution explore the impact of the faculty individually and at-large. Where works have been published under an open access license or made available to the public, records include links to the full-text items.
Like Google Scholar, the discipline keywords are chosen by the user. But Pure also offers a way for scholars to contextualize their research with a research statement (or teaching philosophy, biography, etc.). See President Passerini's Pure Researcher Profile page and expand the section to read more about her academic career.
This helps gives users a more personalized understanding of a scholar's interests but also the way they draw connections between different areas of their inquiry, the foundational concepts that ground their research, and more.
Scopus is a large, multidisciplinary database of scholarly research that is also owned by Elsevier. Like Web of Science, it provides tools to track, analyze, and visualize scholarship. At this time, Gonzaga does not subscribe to the Scopus database, however Pure integrates some features and tools of Scopus into its workflows.
Researcher profiles can be populated by linking faculty members' Scopus ID, which allows Pure to pull in any publications linked to that ID via an API. While Pure does not specifically focus on citation counts as a metric of impact, where available, publication records in Pure will include a link to a preview of a document's Scopus page. That preview page will list the number of citations that Scopus has indexed for the title, as well as the number of references in the publication itself.
Pure explores a scholar's impact in two ways. Like Google Scholar, the first way is by making their collaboration network visible to users. But Pure does this in a way that is much more visually engaging with digital mapping.
Users can drill down into locations on the map to view the institutions and organizations that a scholar’s collaborators are affiliated with. By emphasizing the institutional identities within a collaboration network, the visualization helps highlight where disciplinary centers of inquiry are concentrated.
The second way that Pure helps to measure a scholar's impact is by indicating where their work aligns with the United Nations Sustainable Development Goals (SDGs). This is a set of 17 global goals that the UN adopted in 2015 to serve as a blueprint for addressing global challenges like poverty, inequality, climate change, and more.
Using the 17 SDGs as a set of controlled keywords, Pure tags each publication with the relevant SDGs. It also tags researchers based on the primary SDGs that are reflected in their listed publications.
This is a unique way to explore the impact that a scholar has beyond just academia. They help to understand how a scholar's work aligns with global priorities, examining not just how much a scholar publishes or how many times they're cited, but the societal impact of their work, how their scholarly activities contribute to solving real-life global problems.
Like Web of Science and Google Scholar, Pure doesn't necessarily capture a scholar's complete history of research output. Importing content from Scopus, for example, is limited to publications in journals or databases that are indexed by Scopus. Other integrations are possible, as is manually entering content, but these methods are not as seamless or easy as the Elsevier-owned Scopus pathway.
While it can include datasets, exhibitions, creative works, and other types of non-traditional scholarly outputs, Pure is optimized for more traditional publications like chapters, articles, books, and conference presentations.
Ultimately, there are many different ways and tools that can be used to explore the impact of a scholar, both in the academic sphere and in the real world. Like many research processes, determining the best tools to use depends on the question you are asking. And the Foley Librarians can help you discern which platforms or metrics will help you assess impact for the metrics you are interested in.