Short abstract

The paper will summarize the main lessons learnt about how the introduction of incentives (pay for publications) in South Africa over the past twenty years have produce a vast array of unintended, unexpected and especially questionable gaming practices by South African academics

Long abstract

The introduction of a new Research Output Policy in 2005 by the South African Department of Higher Education has impacted hugely on the publication strategies of academics at public universities in the country. CREST (our research centre) has been studying and assessing the impact of this policy since its inception and produced four main reports over this period. The incentive system is nearly unique in the world as individual academics can - depending on university-specific rules, actually receive supplements to their annual salary for the number of publications produced. We will highlight in this presentation the five most problematic and even devastating effects of this system, viz.

(1) Increase in publication in predatory journals and conference proceedings

(2) The establishment of publication cartels

(3) Increase in publication of poor quality journals

(4) Increase in ghost author affiliations

The second part of the talk will report briefly on a national research programme that we lead to address these unethical and even fraudulent behaviours and identify the key strategies that should be accepted and implemented by the DHET in a revision of the policy in the near future. A revision of the policy is urgently needed as public trust in this subsidy-system has already been eroded and the rationale behind this publicly funded instrument questioned.

Short abstract

The present study provides results of a research carried out in the Sao Paulo Research Foundation employing shadow experiments over thousands of reviews to understand the extent to which the selection is meeting the expected standards of excellence.

Long abstract

There is growing interest among research funding agencies worldwide in examining their selection processes to better understand whether these procedures meet the expected standards of excellence. This includes evaluating the outputs and outcomes of peer review, as well as investigating the potential presence of bias. Some funding agencies have pioneered small-scale experimental initiatives aimed at assessing the impact of alternative selection methods, in addition to traditional peer review, and determining whether such processes may introduce or mitigate selection bias. The present study provides both a review on these attempts and on the methodology and the results of a study carried out in the Sao Paulo Research Foundation (FAPESP). We conducted a shadow experiment to examine the differences between FAPESP’s historical peer review selection processes and alternative prioritization methodologies. The experiment was based on thousands of peer reviews, focusing on the possibilities of alternativa selection methods on the gray zone between the highest-ranked rejected proposals and the lowest-ranked accepted ones. In addition, we applied statistical methods to analyze the presence of potential biases in FAPESP’s selection procedures and to understand the parameters that influence policy making towards funding.

Short abstract

Kulczycki will argue, using over 100 years of experience from Eastern Europe (especially Poland), that metascience has been used as a crucial instrument for national integration and states’ modernization.

Long abstract

Kulczycki will argue, using over 100 years of experience from Eastern Europe (especially Poland), that metascience has been used as a crucial instrument for national integration and states’ modernization. He will highlight how the implementation of metascience after the First World War and 80 years later during the post-communist transition played a key role in reforming the scientific sector and aligning it with broader socio-economic objectives.

Short abstract

The presentation focuses on the development and establishment of an African STI Leaders’ Forum, which has been designed to serve as a potentially powerful platform for the dissemination and use of metascience knowledge(s) in strategic STI visioning and policymaking in Africa.

Long abstract

The presentation focuses on the development and establishment of an African STI Leaders’ Forum, which has been designed to serve as a potentially powerful platform for the dissemination and use of metascience knowledge(s) in strategic STI visioning and policymaking in Africa. The context for and complex dynamics of convening the leadership of Africa’s broader STI ecosystem illustrate the need for metascience scholarship and its potential impact on advancing African STI in ways that are responsive to the challenges and opportunities that African scientific communities and the institutions of which they are a part now confront.

Short abstract

This paper will focus on outlining the challenge for metascience posed by the need to create robust and actionable knowledge about the uptake of social science knowledge to policy.

Long abstract

While, accounting for the societal impact of research is normally associated with funders concerns about accountability or value for money, the uptake of social science research into policy is a more longstanding research effort to grapple with one aspect of research impact. Lindblom and Cohen (1979), Weiss (1979) and more recently, Nelson et al (2023) have dedicated significant attention to developing methods for tracing policy uptake. Although these studies have been focused on the USA, similar, if not identical studies have been done in Europe (LSE 2011), Rosli and Rossi (2016). The outcomes of these studies are varied and the insights on which they converge are often too general in character to do anything other than evidence a need to dig deeper.

Is research on research about the uptake of social science research in policy amenable to large scale studies that could yield robust and generalizable insights? Or is this research area more suitable for an artisanal approach based on single case studies with deep, but locally actionable results? The challenge for metascience in delivering in this narrow but important area is the diversity of policy systems. They vary even within nation states and by virtue of this diversity, the ability to do anything but single case studies on policy uptake is limited. This paper focuses on some potential pathways to find a route to creating meso level studies of knowledge uptake to policy.