One of the most frustrating aspects of being a researcher is applying for funding – or, more specifically, applying and getting rejected. Many of us spend a lot of time and resources on proposals which are more likely to be rejected than not. With the ascent of LLMs the situation has become even worse. While many different things need to change to improve the situation – restrictions on the number of proposals a given individual is allowed to submit per call, better base funding of research institutions, among other things – my personal favourite is randomization in the awarding of research funds.

Based on my experience with applying for research funding (having contributed to and coordinated multiple proposals of different types/sizes for various funding bodies including Horizon Europe, BMFTR, DFG, Biodiversa+, Volkswagen Foundation), the problem is the following: Because everyone knows that (i) the funding shares are extremely and increasingly low for most programmes with relevant amounts of money involved and (ii) the people reviewing the proposals are, well, people, and therefore subject to all kinds of biases, we tend to invest a lot of time and effort into polishing our grandiose proposals. The polishing often focuses a lot on secondary aspects that do not have much to do with content or actual research ideas – visualizations, structure, CVs, language etc. Also, it leads to systematic overpromising – everyone knows that what a proposal promises probably won’t be delivered in full because it’s not quite realistic. So we’re playing a game of pushing the boundaries of „a bit unrealistic, but not completely bonkers“ and „somewhat vague to allow for deviations, but not completely free of substance“. Since everyone believes (probably for good reasons, see above) that this is needed to give your proposal the „edge“ needed to just-barely outcompete other proposals, we’re trapped in a rather sad social dilemma situation where more effort is invested than would be needed if evaluation happened according to clear, objective criteria only.

One could say, and some people do: Let’s just make the evaluation process better and more objective. Well, I guess I don’t believe in objectivity too much – I may have read too much behavioural economics. In my eyes, we (or, more specifically, the funding bodies) are kidding ourselves by thinking that the winning proposals are objectively better than the „losers“ in any meaningful sense. Objective evaluation at the margin, i.e. between proposal that are quite close in terms of quality, is impossible – because of the aforementioned biases to which reviewers are prone, but also and maybe even more because of the multidimensional nature of „quality“ of a research proposal. In that sense, the biases operate at two different levels – they affect how a given proposal is evaluated according to each individual dimension (e.g. novelty or feasibility or stakeholder relevance – which, by the way, are multidimensional composites themselves!), but also how exactly the (sub-)dimensions are weighted.

So, let’s make the evaluation process as objective as possible, even if we can’t reach the first-best, full objectivity? I’d say this is a case where the theory of second best applies: „if there is introduced into a general equilibrium system a constraint which prevents the attainment of one of the Paretian conditions [i.e. conditions required to reach a Pareto optimum], the other Paretian conditions, although still attainable, are, in general, no longer desirable“. (For another economic argument against the „contest model“ of awarding research funds, see here.)

The bottomline is: When you are a „close loser“, the reasons why your proposal was rejected and another one selected for funding feel pretty arbitrary, random. And they effectively are. So instead of kidding ourselves that this can be improved substantially, let’s just accept this element of randomness – but in a systematic and honest way. Roll a dice. Flip a coin. Or, more likely, ask a random numbers generator.

More specifically, the idea would be to filter out „bad“ proposals and keep all that are good enough (I know, there is arbitrariness and subjectivity involved in this, especially with respect to the threshold separating the two – I still think that this is a price worth paying). And among the latter, select the proposal(s) to be funded randomly. The benefits of such an approach include:

• a substantial simplification of the review process, which is much more basic (less resources invested in the evaluation);
• much weaker incentives for perfectionist polishing and exaggeration/overclaiming (less resources invested in the preparation of a proposal);
• a more level playing field where access to resources (be it just to invest more time into the proposal or hire a professional company to support you [which in Horizon Europe has become quite a thing]) and simple accumulated experience with how proposals are written don’t play a big role anymore;
• less frustration in the case of unsuccessful proposals (because of less resources being „wasted“ and because the randomness of the selection is explicit – no outrage at evaluation reports anymore).

Is this a perfect solution? Nope. Is it a silver bullet? Certainly not. Is it technically feasible? Yes, as pilots along those lines in Switzerland or by the Volkswagen Foundation show. Is it likely to be accepted at a larger scale anytime soon? I don’t know. Probably not. But the idea may be worth entertaining even if it takes a while to be taken seriously.

P.S. By the way, all the (inherently biased) evaluation mentioned above is about the quality of the proposal! While I am not aware of any research into that, I am pretty sure that the quality of the proposal is a pretty imperfect predictor of the quality of the ensuing research, which in my eyes strengthens the argument for randomization.