In my previous post advocating for Breaking Down the Barriers- the unjust world of academic paywalls I reflected on the ‘ethics of public access to public-funded research.’
I have never been inside the academic tent and I was keen to see what the barriers are for researchers/academics. In my efforts to understand how the universities are currently evaluated I asked an expert to explain the system to me and what a ‘fit for purpose” system could look like to foster an open access mindset
I learnt that academics and researchers, particularly in Australia, often find themselves navigating a complex and demanding system that evaluates their performance across multiple metrics. While these metrics are designed to measure productivity and impact, they can inadvertently place undue pressure on researchers.
Smiling – To find an image for this post I asked my image repository to find an image that shows a “Professor at desk overwhelmed”. Every single picture of the professor was male and elderly.
This is what the expert shared with me to get an appreciation of the challenges we have, to move to an open access system and what needs to change.
“Australian researchers, like those in many other countries, are often evaluated based on a variety of metrics, and their ranking can be influenced by several factors. Here are some key aspects:
- Publication Count: The number of research papers an academic writes is a significant factor. Generally, more publications can indicate higher productivity and a larger contribution to the field. However, quantity is not the only measure; the quality and impact of these publications are also crucial.
- Citation Metrics: Researchers are often assessed by how often their work is cited by others. Metrics like H-index and i10-index are commonly used. The H-index measures both the productivity and citation impact of a researcher’s publications, while the i10-index counts the number of publications with at least ten citations.
- Journal Quality: Where the research is published matters. Papers in high-impact journals are usually seen as more prestigious and can significantly enhance a researcher’s reputation.
- Research Impact and Quality: Beyond citations, the actual impact and quality of research are important. This can be harder to quantify but includes factors like the relevance and applicability of the research, as well as its contribution to advancing the field.
- Grants and Funding: Success in obtaining competitive research grants, especially from prominent funding bodies, is another measure of a researcher’s standing.
- Awards and Recognitions: Awards from reputable organisations can also contribute to a researcher’s ranking.
- Peer Review and Editorial Roles: Participation in peer review and editorial duties for journals can also reflect a researcher’s expertise and reputation in their field.
- Collaboration and Networking: The extent and nature of a researcher’s collaborations, both nationally and internationally, can also influence their standing.
It’s important to note that while the number of research papers is a significant factor, it’s the combination of these elements that typically contributes to how Australian researchers are ranked. There’s a growing recognition in the academic community that quality, impact, and broader contributions to the field and society are as important, if not more so, than the sheer number of publications.
How the system is changing
Measuring the quality, impact, and broader contributions of academic research, beyond just the count of publications, is a complex task. Here are several methods and metrics that are increasingly being used to assess these aspects:
- Altmetrics: This refers to alternative metrics that assess the impact of research in ways beyond traditional citation counts. Altmetrics include social media mentions, article views or downloads, media coverage, policy document references, and other web-based mentions. These metrics can provide a more immediate and diverse measure of the impact and reach of research.
- Citation Analysis Beyond Counts: Instead of just counting citations, deeper analysis can be conducted. This includes looking at who is citing the work (to understand its influence across different fields), examining the context of citations, and understanding the longevity of the impact (how long a paper continues to be cited).
- Qualitative Reviews: Peer reviews and expert evaluations can provide insights into the quality and significance of research. This can involve assessments by fellow academics, industry experts, or interdisciplinary panels.
- Research Impact Narratives: Researchers are increasingly asked to provide narratives or case studies that describe the impact of their work on society, policy, the environment, or the economy. These narratives allow researchers to explain the significance of their work in a broader context.
- Public Engagement and Communication: Evaluating how researchers engage with the public, policymakers, and other stakeholders can be an important measure of their broader contributions. This can include public lectures, policy briefs, blogs, or media appearances.
- Grant and Award Recognition: Success in obtaining prestigious grants and awards can be a proxy for the quality and relevance of a researcher’s work, as these often involve rigorous peer review processes.
- Patents and Commercialisation: For research with commercial applications, the number of patents filed or successful commercial products can be an indicator of impact.
- Interdisciplinary and Collaborative Work: The ability to engage in interdisciplinary research and to collaborate effectively across different fields or with industry can also be a measure of a researcher’s broader contribution.
- Teaching and Mentorship: The role of researchers in teaching and mentoring the next generation of scholars and practitioners can also be considered, reflecting their contribution to the academic community and society.
- Post-Publication Peer Review and Discussion: Platforms that facilitate post-publication peer review and discussion, such as PubPeer or ResearchGate, can provide insights into the ongoing impact and relevance of research.
Incorporating these diverse metrics and methods helps provide a more holistic view of a researcher’s contributions. However, it’s crucial to use these measures thoughtfully, recognising the unique context of each research field and the individual career paths of researchers.”
#OpenAccess #AcademicFreedom #PublicFundedResearch #AcademicPressure #ResearchEthics #OpenScience #KnowledgeSharing #AcademicPublishing #ResearchAccessibility #EducationForAll



