Should Google Scholar be employed as a benchmarking tool for evaluating the performance of professors in education?
Google Scholar
In today's academic landscape, an individual's research rankings and outputs have a significant impact on their career security and progression. Academics often compare their performance to that of superior colleagues to demonstrate their standing within their discipline when seeking employment and promotions. This paper critically examines the use of Google Scholar (GS) as a benchmarking tool for education professors, considering the reliability of data and issues of participation. The analysis focuses on the GS profiles of full professors at top-ranked universities in Australia, the United Kingdom, and the United States to explore the extent of GS usage in the education professoriate. The paper establishes quartiles of impact based on the H-index and investigates the gender distribution within these quartiles. The limitations of using GS data are highlighted, and the legitimacy of using GS as a benchmarking tool for education professors is questioned. In an environment where metrics play an increasingly important role in job security and promotion, it is crucial to critically evaluate the reliability of metrics with questionable quality and uneven participation.
The H-index
The H-index and citation rates are commonly used metrics in academia to assess the value of an academic's work and facilitate benchmarking. The H-index measures the number of papers an academic has published that have received at least the same number of citations. While the H-index is popular for its simplicity, it has been criticized for potentially rewarding mediocrity and neglecting underlying inequities. Factors such as dubious citation farming practices and gender disparities in self-citation patterns further complicate the interpretation of the H-index.
Using Metrics in Education Benchmarking
Benchmarking research outputs in education using metrics is a controversial practice. These metrics, driven by neoliberal forces, contribute to the commodification of academic work and the researchers themselves. The reliability of the H-index as a performance measure has been challenged, as it can be manipulated and may reward unethical practices. Moreover, the H-index fails to address the inequities that affect individual performance, such as gender disparities and other external factors.
Tools for H-index Calculation
Various tools are available for calculating citation rates and the H-index, each with its limitations in terms of database coverage and accessibility. Google Scholar (GS) offers advantages in terms of cost and breadth of coverage but may include fringe materials. However, GS may not capture the full range of research outputs in the discipline of education, where books and book chapters are considered important publications. Tools like Scopus and Web of Science, while selective, may not accurately evaluate the performance of education professors who often publish in different formats.
Conclusion
Benchmarking research outputs using metrics has become common in academia, but the reliability and utility of these metrics, particularly when using Google Scholar as a benchmarking tool in education, need to be critically examined. The H-index, despite its popularity, has been criticized for its potential to reward mediocrity and overlook underlying inequities. Google Scholar's widespread use in the education professoriate needs further investigation to determine the extent of its usage and its influence on benchmarking practices. The limitations of using Google Scholar data, including quality confounders and issues of participation, must be carefully considered. This paper aims to explore the use of Google Scholar for benchmarking in education and addresses the methodological limitations associated with Google Scholar profiles.
