Watching cancer research go under?

On January 30th, my first research paper was published in The BMJ. Although I am immensely proud of this paper and grateful to everyone who helped it along, our findings paint a grim picture of academic research – and, by extension, of the career I am walking into. Below, I try to make sense of what we found and what it says about the modern research enterprise.

I dived into metascience and research integrity around March 2024 at AusHSI*, in Brisbane, Australia. At the time I had no idea what paper mills were – and, frankly, it had never occurred to me that fake research was a real thing. In fact, few people knew. I’ve often been told “Before we started, I had never imagined it would be that bad”. These days it’s the opposite; I’ve become slightly paranoid and see fake papers everywhere – and most of them probably are. Fun fact: I’ve likely read hundreds of fake papers and only a handful of real ones.

It all started with cancer research, as it was becoming obvious that someone was manufacturing research papers on a large scale in this field. Evidence was pointing to templated text, reused and manipulated images, fake data, fake reagents, implausible results and so on. This wasn’t about a few papers here and there, but about thousands of them – and probably many more (shout-out to all the people manually checking these papers). I quickly realised that cancer research was hosting a cancer of its own: fraudulent studies that grow and multiply much faster than authentic ones. Spread across thousands of journals, the tumour becomes a flood, and genuine research is barely keeping its head above water.

*The Australian Centre for Health Services Innovation

Don’t write scientific papers – buy them

When I learnt that scientific papers could be bought and sold, I immediately thought of Fallout. The running joke of Fallout is that everything is a business, even the most unimaginable things – like the end of the world. In the TV series (yes, I know the video games are quite different), a megacorporation called Vault-Tec built futuristic underground vaults and sold premium access slots, anticipating the nuclear apocalypse. However, when peace negotiations threatened to kill demand, the company decided to drop the bombs itself, hence Fallout. Needless to say, nothing went as planned.

Paper mills found the demand already waiting for them. No bombs required. They turned academia’s pressure to publish into a million-dollar market for research that was never done. And all of a sudden, fake research papers sell like hotcakes, and seller and buyer alike thrive at the expense of the scientific record. As we speak, you can go on Facebook or Telegram and, for the modest sum of a few hundred dollars, buy your way onto a paper. I mean, why spend two years writing a paper when you can get one in under two minutes? In hindsight, I may have gone about this the wrong way: I’ve only got one paper – I could have twenty.

Like any respectable industry, paper mills have to balance quantity and quality. The economics are attractive: assuming an average slot goes for $800 and an average paper carries six authors, a single manuscript could be worth around $5,000. However, the delicate part is fabricating the science itself. Some mills – arguably the low-end ones – will not bother: plagiarised text, manipulated images, invented data, to a point where it eventually becomes obvious. They work fast, but they are the ones we catch (so I kind of like these ones, I can show off in front of colleagues and go “this one’s fake”).

The worrying ones are the others: those that slowly and carefully assemble a study that looks perfectly well-conducted, where the only trace of fraud is a trial registration number that leads nowhere. These are the too-good-to-be-true papers, and they’re becoming increasingly common. You get the idea, there is a whole range of products on this market, and the price scales with the quality of the forgery and the ambition of the target journal.

You’ve read fake papers. You just don’t know yet

One of the main objectives of our project was to find easy ways of detecting paper mill papers. A week into it, I was convinced that if such ways existed, they would not be easy. Part of the problem lies in the fact that the scientific literature is a whole bloody mess. Different templates, different publication types, different conventions from one field to the next – finding a single actionable indicator of fraud is like looking for a needle not in a haystack, but in the entire field.

And yet, most of the suspicious papers I read looked like one another – so much so that I stumbled on two near-duplicates by chance (both later retracted). The template hypothesis then became our best lead. Fresh out of data science school and riding the LLM wave, I decided to use BERT – which I had discovered three months earlier – to detect paper mill papers. We used retracted papers of suspected paper mill origin listed in the Retraction Watch Database to train our model, and the resulting accuracy was surprisingly good. So good that we decided to screen the entire cancer research literature. Here is what we found, straight from the paper:

Read: Machine learning based screening of potential paper mill publications in cancer research: methodological and cross sectional study

The model achieved an accuracy of 0.91. When applied to the cancer research literature, it flagged 261,245 of 2,647,471 papers (9.87%, 95% confidence interval 9.83 to 9.90) and revealed a large increase in flagged papers from 1999 to 2024, both across the entire corpus and in the top 10% of journals by impact factor. More than 170,000 papers affiliated with Chinese institutions were flagged, accounting for 36% of Chinese cancer research articles. Most publishers had published substantial numbers of flagged papers. Flagged papers were overrepresented in fundamental research and in gastric, bone, and liver cancer.

Roughly one cancer paper in ten. Not proven fraudulent – flagged as bearing the textual signature of templated production. But one in ten.

The take-home message is that there are hundreds of thousands of potential paper mill papers in the literature. Keep in mind that this is only cancer research, and that paper mills are also actively targeting conference proceedings in computer science and mathematics, and many other fields besides. Furthermore, this is only templated paper milling – what about all the subtler mills, and all the other fraudulent practices? Our model was trained on a small sample with limited representativeness, so if anything, these numbers are likely to be underestimates. The last concern is AI, which will only accelerate paper mills and push cancer research further towards the edge. Note that we already discussed false positives and prevalence estimates here:

Read: Reply to Elson and Cummins: Uncertainty at the heart

Nobody’s fault, everybody’s problem

It goes without saying that academia has a problem. What is less obvious is who to blame. The system that rewards publication above everything else? The publishers who wave fake papers through? The researchers who buy their way onto them? The overworked reviewers, too few and too rushed to catch anything? Or the paper mills, for exploiting the whole thing? Probably a little of each.

I was once a phytopathologist (or at least, I was really into phytopathology), and in plant diseases there are parasites known as weakness parasites, which only successfully infect their host when it is already weakened. A textbook example is a mould called Botrytis cinerea, which thrives on plants that are stressed, wounded or senescing, and leaves healthy ones largely alone. Well, I’ve come to see paper mills as research parasites of exactly this kind (hence the cancer within cancer research). They don’t infect a healthy literature – they bloom on a weakened one. Which means their spread isn’t really the disease; it’s the clearest sign the research system is broken. And this is not getting any better as fake science grows much faster than genuine science.

On top of that, what really struck me was the impunity behind research fraud. Technically speaking, using paper mill support and producing fake science carry limited risks. I can’t speak for other countries, but in France there is no such thing as a specific “scientific fraud” offence in the law. We’ve had some cases of suspected fraud that led to disciplinary sanctions, but nothing more (see the Patrice Dunoyer and Olivier Voinnet cases). As a result, most fraudsters will never be prosecuted, universities and labs stay untroubled, journals are rarely deindexed or held accountable, and the government doesn’t go after paper mills. France may look relatively spared, but the phenomenon is rampant elsewhere – and the same legal grey area may exist in other countries. Furthermore, this isn’t only about law: publishers, universities and researchers aren’t doing much either.

So how do we get rid of parasites? Well, we don’t let them in, or we make the host no longer suitable. You don’t fix a leak by bailing out the water – you fix it by plugging the hole. The hole itself is quite big, because it encompasses academic and publisher incentives to publish, for which short-term prestige and profit may result in long-term damage. Plugging it means changing what we reward. As long as careers and profits are built on volume, someone will always sell the volume. We could try fixing the incentives, value replication over novelty, quality over count, and scrutiny over speed (I know these are nice words but it’s hard to do in practice. Honestly I can’t see it working any other way). In the meantime, we still have to deal with the infection: detecting paper mills, flagging suspicious papers, cleaning the record. As a young scientist stepping into research, I can’t help but feel this is a now-or-never moment – and that those of us just arriving have a real part to play in it. There’s a lot of work ahead, but I’m hopeful we can still safeguard the research literature.

[NOTE: this post was started on February 5th but only finished on July 27th – I had a lot of things to do]