Table 1 Hypothetical scenario of some potential ramifications following the naïve use of AI-assistance technologies in the research cycle

In this scenario, we encounter Professor Nigel Eve, a distinguished medical doctor with limited knowledge of genomics, delving into the intricacies of breast cancer research using AI-generated content. Employing a large language model AI tool, he embarks on a comprehensive literature review to explore the genetic variants associated with breast cancer

The AI model efficiently presents Professor Eve with a list of seemingly relevant genetic variations linked to breast cancer. Not seeking counsel from a genomics expert or conducting further validation, he incorporates the AI-generated findings into his research paper

However, unbeknownst to him, the AI model’s training data contains incomplete as well as potentially biased information about the genetic variants, leading to the inclusion of inaccurate and misleading details about the relationship between breast cancer and genetics in his paper

Throughout the peer review process, the reviewers, similarly relying on large language models to carry out their duties, reach the same erroneous conclusion as Professor Eve, and the paper is eventually accepted for publication

Trusting Professor Eve’s esteemed reputation as a medical doctor, readers may inadvertently accept these flawed findings, potentially steering other researchers or clinicians toward misguided avenues in their own breast cancer research

Adding to the complexity, one of the genetic variants included in Professor Eve’s research has since been disproven by the scientific community. This critical oversight, stemming from his lack of genomics expertise and failure to perform necessary due diligence, casts doubt on the credibility of his breast cancer research and may have far-reaching consequences

The ramifications of such inaccuracies could impact fellow academics and scientists, leading to misallocation of funding and valuable research time based on erroneous conclusions. Patients, too, may be affected by misguided treatment approaches inspired by this flawed research

In conclusion, Professor Eve’s expedition underscores the paramount importance of amalgamating AI tools with human domain expertise and meticulous due diligence to ensure the accuracy and integrity of research findings. Neglecting these vital steps may lead to misguided scientific pursuits, wasted resources, and, most significantly, potential harm to patients