Research Evaluation Needs to Change with the Times
The focus on a narrow set of metrics leads to a lack of diversity in the types of leader and institution that win funding.
Send us a link
What a Professor Considers when Reading Recommendation Letters
What a Professor Considers when Reading Recommendation Letters
The role they play in evaluations for graduate school admissions, fellowships and jobs can be baffling.
Key Role in Evaluation Procedure: the Evaluation Panels and Their Members
Key Role in Evaluation Procedure: the Evaluation Panels and Their Members
The SNSF's National Research Council decides whether or not to fund applications. The 89 evaluation panels handle the preparatory work on which it bases its decisions, assessing several thousand applications each year.
Creating What We Seek to Measure - How to Understand the Performative Aspect of Impact Evaluation?
Creating What We Seek to Measure - How to Understand the Performative Aspect of Impact Evaluation?
This post draws on a recent analysis of different impact evaluation tools to explore how they constitute and direct conceptions of research impact.
'Good Progress' on Research Excellence Framework Panel Diversity
'Good Progress' on Research Excellence Framework Panel Diversity
Female representation now proportionate to UK academia as a whole, even if ethnic minorities still fall short.
Responsible Research Assessment Faces the Acid Test
The University of Liverpool is planning to make lay-offs on the basis of controversial measures. How should the global movement for responsible research respond?
European Research Council Bans Journal Impact Factor from Bids
For the first time prestigious funder has explicitly told academics they must not include metric when applying for grants.
Not What You Know, but Whom You Know? Study of ERC Stirs Old Scientific Controversy
Not What You Know, but Whom You Know? Study of ERC Stirs Old Scientific Controversy
A new study of grants awarded to early-career researchers by Europe's premier science agency is reviving an old controversy over the way governments decide which scientists get research money, and which do not.
Pedagogical Research Needs Clearer Standards of Excellence
Wider take-up by academics requires both relevance to specific disciplines and accessibility across disciplines, says Camille Kandiko Howson.
German Professors' Motivation to Act As Peer Reviewers
Acting as a reviewer is considered a substantial part of the role-bundle of the academic profession. However, little is known about academics' motivation to act as reviewers.
Bias Is a Big Problem. But So Is 'Noise.'
When it comes to mistaken judgments, there is more than one kind of error.
The Changing Role of Funders in Responsible Research Assessment: Progress, Obstacles and the Way Ahead
The Changing Role of Funders in Responsible Research Assessment: Progress, Obstacles and the Way Ahead
A responsible research assessment would incentivise, reflect and reward the plural characteristics of high-quality research, in support of diverse and inclusive research cultures.
COVID is Amplifying the Inadequacy of Research-Evaluation Processes
COVID is Amplifying the Inadequacy of Research-Evaluation Processes
Systems for assessing scientists' work must properly account for a lost year of research - especially for female researchers.
When More is More: Broad Calls for Multilingualism and Evaluation Reform
When More is More: Broad Calls for Multilingualism and Evaluation Reform
In recent years, numerous initiatives have highlighted linguistic biases embedded in current evaluation processes and have called for change. The DORA-hosted community discussion on multilingualism in scholarly evaluation was inspired by actions others have taken to address these issues.
AI-based Citation Evaluation Tools: Good, Bad or Ugly?
What are the pitfalls of using AI as a citation evaluation tool?
Academia in Motion: a Different Form of Recognition and Reward
Academia in Motion: a Different Form of Recognition and Reward
A better balance between teaching and research duties, greater recognition of team performances and the elimination of simplistic assessment criteria would improve the systems of recognition and rewards in academia.
How Do Researchers Evaluate Research?
The Official PLOS Blog studies how researchers evaluate both the credibility and impact of research outputs.
Rethinking Research Assessment
Part one of a four part series on major barriers to equitable decision-making in hiring, review, promotion, and tenure processes that commonly result from biased thinking in academia. Part one delves into objective comparisons.
What is Meaningful Research and How Should We Measure It?
Purely metric-based research evaluation schemes potentially lead to a dystopian academic reality, leaving no space for creativity and intellectual initiative, claims a new study.
Gender and Other Potential Biases in Peer Review: Cross-sectional Analysis of 38 250 External Peer Review Reports
Gender and Other Potential Biases in Peer Review: Cross-sectional Analysis of 38 250 External Peer Review Reports
The Swiss National Science Foundation (SNSF) set out to examine whether the gender of applicants and peer reviewers and other factors influence peer review of grant proposals submitted to a national funding agency.
Beyond Publication - Increasing Opportunities For Recognizing All Research Contributions
Beyond Publication - Increasing Opportunities For Recognizing All Research Contributions
Recognizing the many ways that researchers (and others) contribute to science and scholarship has historically been challenging but we now have options, including CRediT and ORCID.
Changing How We Evaluate Research is Difficult, but Not Impossible
DORA has evolved into an active initiative that gives practical advice to institutions on new ways to assess and evaluate research. This article outlines a framework for driving institutional change.
Five Better Ways to Assess Science
Hong Kong Principles seek to replace 'publish or perish' culture.
Improving Research Assessment in the Triage Phase of Review
Improving Research Assessment in the Triage Phase of Review
Universities and research funders are increasingly reconsidering the relevance and importance of researchers' contributions when assessing them for hiring, promotion or funding.
Value People As Well As Papers to Improve Research Culture
As scientists, we try to make sure our research is rigorous so that we can avoid costly errors. We should take the same approach to tackle issues in research culture, says Professor Christopher Jackson.
NIH Peer Review: Criterion Scores Completely Account for Racial Disparities in Overall Impact Scores
NIH Peer Review: Criterion Scores Completely Account for Racial Disparities in Overall Impact Scores
Study found that preliminary criterion scores fully account for racial disparities - yet do not explain all of the variability - in preliminary overall impact scores.
Empathy and Grit - Not Just Publication Records - Should Be Considered in Researcher Assessment
Empathy and Grit - Not Just Publication Records - Should Be Considered in Researcher Assessment
Critics of current methods for evaluating researchers’ work say a system that relies on bibliometric parameters favours a ‘quantity over quality’ approach, and undervalues achievements such as social impact and leadership.
How China's New Policy May Change Researchers' Publishing Behavior
How China's New Policy May Change Researchers' Publishing Behavior
A researcher from the Wuhan University of China offers a view of how Chinese researchers are reacting and are likely to alter their behavior in response to new policies governing research evaluation.
×