Researchers in every age have lamented the politics and structural problems with how science is conducted, and it would be naïve to suppose that the current state of science represents any kind of historical low. Nonetheless, there are a number of specific problems with the system of scientific research today that could be addressed and that cause either important inaccuracies in what we report or cause the pace of progress to slow. The problems are of particular concern in medical research, where their practical consequences are measured in thousands or millions of lives, and where the structural problems in the research system are particularly acute. Here are a few of the key problems:
As I detail in a blog post here, putting more pressure on individuals or organizations to perform well can have perverse consequences, particularly when (a) we measure performance through imperfect proxies, such as number of articles published, and (b) these metrics can be manipulated at the expense of the underlying goal. The current science funding system puts enormous pressure on researchers to perform at the highest levels, particularly in medical research where many individuals are in “soft money” positions – i.e., where their salary depends on them securing large grants. Most researchers love their research and want to do it well, but when you threaten to take away their jobs even well-meaning researchers will start to “play the game,” publishing in ways that advantage their CVs more than the state of science. The impact of each such decision is small, but the cumulative impact is enormous on both the trustworthiness of published articles and on a culture of intellectual investigation. Some performance incentive is essential, but it should be much less than in the current system. An improved system would have (a) a very high bar to start a career as an independent researcher; (b) a relatively low bar and streamlined procedure to obtain basic yearly operating funds and salaries (Canada’s NSERC is a good model, though the yearly amounts are a bit too low); (c) a relatively high bar to obtain larger grants; and (d) the elimination of incentives to bring in grant dollars, such as benefits to the institution tied to the value of the grants.
Most students completing a Ph.D will never work as a professor in their chosen field, and many will not have any form of employment that allows them to use their specialized training. Both universities and professors have strong incentives to crank out as many students as possible (good for their bottom lines and CVs, respectively). This is a disservice to the students we train, who will struggle in a hard job market. Post-doctoral fellowships lasting 5-7 years before obtaining a job as assistant professor are becoming the norm in some fields. Many young people pursue Ph.Ds because they are not sure what to do or have no other job – an awful reason to use public tax money for specialized training in an overpopulated field. The solution to this is to make Ph.Ds free for accepted candidates (i.e., no tuition or fees, and a reasonable fellowship), but ensure that the bar for acceptance is very high based on field-by-field job market analyses. Researchers should be prevented from circumventing this system by funding Ph.D students with grant money.
Many researchers have written about this, but briefly, the peer-review process, while helpful, is too long. Journals publish articles that will help them be profitable or prestigious, even if they do not always represent the best science. It is hard to publish negative or ambiguous results. Many solutions have been proposed, but the problem is still getting worse rather than better.
If I bring in multi-million dollar grants and do mediocre science, my university will love me and my CV will look excellent. If I bring in almost no funding, sit alone in my office, and publish 20 high-quality articles per year that change the paradigms of various fields, I will receive little recognition from my university. All the incentives are for researchers to pursue the former type of research, not the latter, even though the latter is a much better investment of public funds. Not only are we not rewarded for being frugal, we are actively rewarded for being profligate! This means that either we could finance many more researchers or finance current research much more cheaply if we just encouraged researchers to be budget conscious. The solution is simple: funding organizations should evaluate not just the quality of research proposals (as is currently the case) but also the cost-effectiveness of proposals.
An Australian study estimated that the cost of submitting a grant proposal is about $17,000. Funding rates in Canada at CIHR are about 15-20%. This means that to get a grant that may be worth $200-500,000, an investment of ~$100,000 is necessary. While “profitable” in a certain sense, this certainly implies that too much of the money invested in research goes to looking for more money. Would you give to a charity that would use 1/3 of your donations on fundraising? A related problem is that, because grants are too specific and less visionary, they are both a lot of work to compile and subject to critique based on very minor methodological points. The solution again is simpler, broader grants that give researchers latitude, such as NSERC’s Discovery grants program and (potentially) CIHR’s Foundation scheme.