On Jun. 4, 2025, the Center for Scientific Integrity — the non-profit organization that runs Retraction Watch — launched a project aimed at rounding up flawed and fake medical-research papers and neutralizing their impact on health guidelines. The Medical Evidence Project will be run by James Heathers, a science-integrity consultant and self-described “data thug” based in Boston, Massachusetts.

“I don’t think anything is more of an immediate threat to people than bad medical research,” Heathers says. “We’re going to find the bad medical evidence that is a threat to quality of life and tell everyone.”

The project, which has a US$900,000 grant from funder Open Philanthropy in San Francisco, California, to run for two years with a team of three to five people, aims specifically to root out flawed papers that have a serious impact on medical guidelines by skewing meta-analyses — reviews that combine the results of multiple similar studies to come to a statistically more powerful conclusion. “It is a great idea to take a very detailed look at such papers,” says Elisabeth Bik, a data sleuth in San Francisco who is not involved with the project. Earlier this year, the Center for Scientific Integrity (CSI) set up a fund for grants under Bik’s name after she donated $200,000 to the organization.

Research papers marred by innocent mistakes, misconduct or outright fakery are a serious problem in science. Oransky estimates that one in every 50 papers published by scientific journals should probably be retracted. That amounts to tens of thousands of papers among the roughly three million published each year.

Other projects have already sought to root out bad data. One trawled through 50 high-profile medical reviews and found serious concerns with about 6% of the papers that fed into them¹. The team that ran that project is working on a tool to help check papers for trustworthiness. The historical examples of papers such as these affecting health policy “are terrifying”, Heathers says.

For example, a 2009 guideline from the European Society of Cardiology recommended the use of blood-pressure-lowering medicines called beta-blockers to protect the heart during non-cardiac surgery, based heavily on a series of studies that began in 1999. However, a few years later, concerns were raised about those studies. The guideline was amended in 2013 with an ‘expression of concern’, but a 2014 investigation was unable to “confirm or dispel doubts” about the work.

The Medical Evidence Project, which launched on June 1, wants to provide a secure platform for whistle-blowers who are daunted by the idea of publishing their concerns about influential medical papers on public platforms such as PubPeer, or find that route ineffective. “I would like to be a resource for those people,” Heathers says. The project’s website will provide a mechanism to supply anonymous tips.