- \n
- A \u2018ChatGPT-authored\u2019 scientific paper highlights the promise and pitfalls of using AI in research and publications.<\/li>\n<\/ul>\n
\n![\"\"](\"https:\/\/i0.wp.com\/thepublicationplan.com\/wp-content\/uploads\/2023\/10\/AdobeStock_557443709.jpeg?resize=1024%2C683&ssl=1\")
<\/p>\nUse of artificial intelligence (AI) in scientific publishing seems inevitable<\/a>. While the full capabilities of this fast-changing<\/a> technology are yet to be determined, some in medical publishing have begun to explore<\/a> ways to harness the potential of generative AI, while others urge caution<\/a> and lament a lack of structured guidance<\/a>. Recently, as reported by Gemma Conroy in Nature News<\/em><\/a>, Professor Roy Kishony and his student, Tal Ifargan, provided new fuel for the debate, by asking ChatGPT<\/a> to conduct research and write a paper<\/a> from scratch.<\/p>\n
Kishony and Ifargan used a \u2018data to paper\u2019 system, in which software acted as a \u2018go between\u2019 between humans and generative AI. This system automatically prompted ChatGPT to follow the steps of scientific research, from hypothesis generation to development of a scientific manuscript. In less than an hour, ChatGPT developed a study objective; wrote code to analyse a large, publicly available dataset; and drew conclusions based on its findings and existing literature, which it reported in a 19-page research article.<\/p>\n
The study highlighted some promising aspects of incorporating AI into research and publication pathways, namely reduced timelines and the potential to quickly generate written summaries. However, it also shone a light on a number of limitations and risks:<\/p>\n
- \n
- False narratives:<\/strong> In this case, ChatGPT claimed to \u2018address a gap in the literature\u2019, although the subject (a link between diabetes risk and diet and exercise) was already well investigated.<\/li>\n
- Decrease in research quality:<\/strong> Kishony flagged the risks of generative AI leading to \u2018p hacking\u2019 or a flood of low-quality research papers.<\/li>\n
- Incapable of self-correction: <\/strong>Stephen Heard of Scientist Sees Squirrel<\/a>\u00a0also provided commentary and analysis on the limitations<\/a> thrown up by the study, including generative AI\u2019s lack of accuracy. Expert human intervention was required throughout, to spot and correct errors.<\/li>\n
- Regurgitating existing ideas: <\/strong>Heard also emphasised that generative AI creates content based on existing source material, thus perpetuating biases and reducing innovation and creativity.<\/li>\n
- Hallucinations: <\/strong>As explained<\/a> by Jie Yee Ong in The Chainsaw<\/a>, \u2018hallucinations\u2019 are a well-known problem with generative AI. This study was no exception, with ChatGPT generating fake citations despite access to the published literature. As Ong puts it, \u201cfor now, it is best not to treat everything ChatGPT spits out as gospel\u201d.<\/li>\n<\/ul>\n
Kishony and Ifargan\u2019s carefully planned study allowed generative AI\u2019s work to be checked for accuracy by human experts. Researchers agree that these human checks and balances remain essential to ensuring the credibility of scientific research and publications in which AI plays a role.<\/p>\n
\n
Researchers agree that human checks and balances remain essential to ensuring the credibility of scientific research and publications in which AI plays a role.<\/p>\n<\/blockquote>\n
\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2014\u2013<\/p>\n
- Decrease in research quality:<\/strong> Kishony flagged the risks of generative AI leading to \u2018p hacking\u2019 or a flood of low-quality research papers.<\/li>\n
- False narratives:<\/strong> In this case, ChatGPT claimed to \u2018address a gap in the literature\u2019, although the subject (a link between diabetes risk and diet and exercise) was already well investigated.<\/li>\n
![\"\"](\"https:\/\/i0.wp.com\/thepublicationplan.com\/wp-content\/uploads\/2023\/10\/Summary_new-byline.png?resize=1024%2C125&ssl=1\")
<\/a><\/p>\n