- \n
- The ERROR project pays reviewers to search for mistakes in the scientific literature, while rewarding authors who agree to participate.<\/li>\n
- Reviewers and authors receive bonuses depending on the extent of errors found.<\/li>\n<\/ul>\n
\n![\"\"](\"https:\/\/i0.wp.com\/thepublicationplan.com\/wp-content\/uploads\/2024\/10\/AdobeStock_968825612.jpeg?resize=1024%2C683&ssl=1\")
<\/p>\nAmid rising retraction rates<\/a>, the scientific record is increasingly scrutinised for signs of research misconduct<\/a> like fabrication<\/a> and image manipulation<\/a>. But what about detecting errors in the data underlying scientific publications?<\/p>\n
The ERROR project<\/strong><\/p>\n
Modelled on tech company \u2018bug bounty\u2019<\/a> programmes, the Estimating the Reliability & Robustness of Research (ERROR) project<\/a> offers cash rewards for reviewers identifying incorrect or misinterpreted data, code, statistical analyses, or citations in scientific papers. Following ERROR\u2019s launch<\/a> earlier this year, Julian Nowogrodzki reviewed the project so far in a recent article<\/a> in Nature<\/em><\/a>.<\/p>\n
Professor Malte Elson and colleagues are aiming to produce a blueprint for systematic error detection that will be scalable and transferable across scientific fields. Starting with highly cited psychology papers, the first review was posted<\/a> in August. ERROR plans<\/a> to cover 100 publications over 4 years, expanding into artificial intelligence, medical research, and potentially preprints.<\/p>\n
\n
\u201cThe ERROR project offers<\/em> cash rewards for reviewers identifying incorrect or misinterpreted data, code, statistical analyses, or citations in scientific papers.\u201d<\/em><\/p>\n<\/blockquote>\n
Financial incentives<\/strong><\/p>\n
The project has 250,000 Swiss francs (~\u00a3220,000) of funding from Professor Elson\u2019s institution, the University of Bern<\/a>. Reviewers can earn up to 1,000 Swiss francs each time, plus a variable bonus of up to 2,500 Swiss francs depending on the scale of errors identified. Authors receive up to 500 Swiss francs: 250 for agreeing to participate and sharing data<\/a>, plus a bonus if minimal errors are found.<\/p>\n
A challenging path<\/strong><\/p>\n
Despite the incentives, ERROR has hurdles to overcome:<\/p>\n
- \n
- Author buy-in:<\/strong> So far, authors from just 17 of 134 selected papers have agreed to participate.<\/li>\n
- Data access:<\/strong> Underlying data may have been lost or authors may cite legal reasons barring sharing.<\/li>\n
- Reviewer expertise:<\/strong> There are limited potential reviewers with sufficient technical expertise yet no conflicts of interest. Dynamics linked to seniority may also prevent some prospective reviewers taking part.<\/li>\n<\/ul>\n
The ERROR team hopes to convince research funders to allocate money for error detection \u2013 ultimately saving them from investing in flawed research. We look forward to seeing how this project helps move the needle towards a more reproducible scientific record.<\/p>\n
\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2013<\/p>\n
- Data access:<\/strong> Underlying data may have been lost or authors may cite legal reasons barring sharing.<\/li>\n
- Author buy-in:<\/strong> So far, authors from just 17 of 134 selected papers have agreed to participate.<\/li>\n
![\"\"](\"https:\/\/i0.wp.com\/thepublicationplan.com\/wp-content\/uploads\/2024\/10\/Aspirations_Micko-Calizon_1.png?resize=1024%2C125&ssl=1\")
<\/a><\/p>\n