View Article PDF
The idea of artificial intelligence (AI) was first introduced in the 1950s.[[1–4]] A subfield of AI is Natural Language Processing (NLP), which is the ability of a computer program to understand the human language.[[1,5]] However, the concept of large language models (LLM) as we know them today is a type of NLP[[6]] that is a program capable of producing human language and answers based on large data.[[7]] In recent years, many language models such as Google Neural Machine Translation (GNMT)[[8]] and the Bidirectional Encoder Representation from Transformers (BERT)[[9]] have been created. In 2018, OpenAI launched its first model, the Generative Pre-trained Transformer (GPT),[[10]] which was followed by further development resulting in GPT-2 and GPT-3.[[6,11]] In November 2022, OpenAI introduced ChatGPT, which is currently freely available online.[[12]] The latest version, GPT-4, was made publicly available for a user fee in March 2023. GPT-4 is currently the most advanced system available to the public, as it is based on more background information, with more advanced problem solving and greater accuracy.[[13]]
Since the launch of ChatGPT, it has garnered the attention of many people worldwide, including biomedical researchers, as it appears to be able to substantially assist in the reporting process of biomedical research (article writing). This raises questions such as: how should AI be understood in the context of research reporting? What can AI help researchers with? How should the implementation of ChatGPT be used in the world of research? Could this mean that a paradigm shift in the field of research reporting is on the horizon?
The capabilities of ChatGPT
ChatGPT can help researchers in various phases of writing scientific articles (Figure 1). We verified the answers given by the robot in Figure 1 and also tried other possible features in multiple sessions. It turned out that the robot can:
• identify relevant literature, information, and potential collaborators such as researchers and institutions.
• Identify relevant topics and trends in the respected research fields.
• Organise ideas and create an outline for an article.
• Conduct a literature review, such as providing relevant articles and studies.
• Write different sections of an article, such as the introduction, methods, results, and discussion.
• Produce an abstract that fits the article.
• Help with grammar, syntax, and style.
• Format the manuscript according to the journal’s guidelines.
• Give ideas as to how charts, graphs, and figures could be constructed if data is explained in text format (the answer would then also be in text format).
• Assist with the communication for research through blogs and social media by writing laymen’s descriptions and giving ideas as to what type of post could be relevant on social media.
• Write conference abstracts.
• Translate manuscripts into other languages.
• Write a covering letter.
• Produce title pages.
• Format references to specific citation styles.
These are some of the ways ChatGPT can assist researchers; however, the quality of the AI output has not yet been formally tested against corresponding human work. Nevertheless, it is obvious that the robot can provide substantial help for the researcher in the reporting phases of scientific work. This means that a paradigm shift may be on the way for how research is reported in the future, potentially making it possible to produce an astonishing number of scientific articles within a short time frame once we have learned the potential of the robotic platform and how it can assist the researcher without compromising on quality.
Even though AI can help researchers in numerous ways, it is not free from limitations. For example, ChatGPT sometimes provides incorrect answers, and it may reference an article that does not exist.[[14]] Furthermore, the possibility of bias in the responses is unknown as the end-user has no control of the input data sources for the robot. Thus, there is a theoretical risk that some information regarding a topic can be left out, which could possibly lead to misinformation being spread about a topic. An example could be that some controversial articles or data would be left out of the AI’s data. Finally, every researcher using AI should, of course, check the output information for credibility.
There has been concern that answers by the robot are so well-formulated and intelligent that it could be difficult to distinguish them from text produced by humans. However, new software can now detect AI-generated text such as GPTZero[[15]] and the AI classifier,[[16]] although they have not yet been tested systematically.
The issue of authorship
When a robot assists in the writing process of a scientific paper, it is necessary to consider whether the AI assistant should be accredited as a co-author in the byline. Another possibility would be to mention it in the methods section or give credit for the contribution in the acknowledgements section.
Since the launch of ChatGPT, publishers have been trying to create authorship policies for the new chatbot.[[14,17]] Currently, three articles and two preprints have an AI robot as a co-author.[[18–22]] Publishers and preprint servers typically agree that ChatGPT does not fulfill the authorship criteria because it cannot take responsibility for the content of the scientific paper.[[14,17]] Furthermore, the editors of Nature[[17]] and Science[[14]] will not allow AI to be listed as an author, and the publisher Taylor and Francis prefers that the AI be mentioned in the acknowledgements section as a contributor.[[17]] Due to increased use of AI, publishers will need to decide on authorship issues for this new player in the field. Nature, along with Springer Nature journals, have formulated two rules in their author guidelines[[23]]: “Large Language Models (LLMs), such as ChatGPT, do not currently satisfy our authorship criteria. Notably, attribution of authorship carries with it accountability for the work, which cannot be effectively applied to LLMs. Use of an LLM should be properly documented in the Methods section (and if a Methods section is not available, in a suitable alternative part) of the manuscript.” This is fully compliant with the authorship criteria described by the International Committee of Medical Journal Editors (ICMJE)[[24]]: “1) Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND 2) Drafting the work or revising it critically for important intellectual content; AND 3) Final approval of the version to be published; AND 4) Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.”
Thus, since the robot cannot be accountable for all aspects of the work, byline authorship is not an option for an LLM such as ChatGPT. Depending on the amount of contribution, it would be appropriate to mention ChatGPT in the methods section or as a formal contribution in the acknowledgements section.
Discussion
In its current version, the AI chatbot can, in principle, be seen as a medical writer. A medical writer is a professional author with skills in language and writing,[[25]] and they can provide grant writing, laymen descriptions, scientific articles, and more.[[ 25,26]] When outsourcing parts of the research process is already normal, why not outsource part of the writing process to a medical writer or an AI robot? We already use research assistants for data collection and statisticians for statistical analyses, so maybe it’s time to use AI for various phases of manuscript production. The cost for researchers using AI is substantially lower than paying for a medical writer, so maybe we could consider the AI robot as a low-cost medical writer.
An important issue, however, is the quality of the AI output. We don’t yet know if it is as good as a professional medical writer. With our limited experience at present, we seriously doubt that the AI quality is good enough, but since AI has learning capability and since the current versions of these models are still in their infancy, we don’t know what the future will bring.
Some researchers may be concerned that the ease of use and low cost may trigger research misconduct with fabricated results, unintentional errors, or deceptive publications. However, misconduct can occur with or without AI, and with our current knowledge of these systems, we don’t believe that it would become worse or better with AI for research reporting. Rather, AI in research reporting should be seen as a “low-cost medical writer”, although we are not fully there yet regarding quality.
In conclusion, AI could probably be used as a medical writer for at least some parts of the article production phases, but in our opinion, this does not mean that the AI should be listed as a co-author. Depending on the contribution, the AI could be thanked in the acknowledgements section or mentioned in the methods section of the paper. Documenting where the AI has assisted will heighten the transparency and credibility of the work. Furthermore, it should be noted that ChatGPT or GPT-4 are not yet at the level of a professional medical writer, and further investigation and research need to be conducted with AI to fully understand its capabilities. These interesting new developments could mean a drastic paradigm shift in the field of research reporting, where various tasks may soon be taken over by AI platforms. AI is available to every researcher, whereas a medical writer is only available with sufficient funding, so AI could potentially become more widespread than the use of a professional medical writer. These AI systems are still in their infancy, and the development is exponential. Therefore, we are facing substantial changes in research reporting where tasks other than article writing may become the main focus for researchers in biomedicine in the near future.
View Figure 1.
Summary
Abstract
Aim
ChatGPT and the newest GPT-4 are AI language models developed by OpenAI that have gained attention for their potential applications in biomedical research reporting. The models can assist researchers in various stages of writing scientific articles, including literature search, outlining, writing different sections, formatting, and translation. The use of ChatGPT or GPT-4 in research reporting has the potential to speed up the writing process, but its limitations, such as incorrect answers and biases, should also be considered. There is ongoing debate over the issue of AI authorship in scientific papers, with some publishers allowing it to be listed as a contributor in the acknowledgements section, while others do not allow it to be listed as an author. The use of ChatGPT or GPT-4 in research reporting is a recent development, and further studies and discussions are needed to determine their potential and limitations in this field.
Method
Results
Conclusion
Author Information
Acknowledgements
Correspondence
Correspondence Email
Competing Interests
1) Kaul V, Enslin S, Gross SA, History of artificial intelligence in medicine. Gastrointest Endosc. 2020 Oct;92(4): 807-812. doi: 10.1016/j.gie.2020.06.040.
2) Amisha, Malik P, Pathania M, Rathaur VK. Overview of artificial intelligence in medicine. J Family Med Prim Care. 2019 Jul;8(7):2328-2331. doi: 10.4103/jfmpc.jfmpc_440_19.
3) Ramesh AN, Kambhampati C, Monson JR, Drew PJ. Artificial intelligence in medicine. Ann R Coll Surg Engl. 2004 Sep;86(5):334-8. doi: 10.1308/147870804290.
4) Holmes JH, Sacchi L, Bellazzi R, Peek N. Artificial Intelligence in Medicine AIME 2015. Artif Intell Med. 2017 Sep;81:1-2. doi: 10.1016/j.artmed.2017.06.011.
5) Lutkevich B, Burns E. Natural language processing (NLP) [Internet]. TechTarget. 2023 [cited 2023 Jan 31]. Available from: www.techtarget.com/searchenterpriseai/definition/natural-language-processing-NLP.
6) Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Language Models are Unsupervised Multitask Learners. Cloudfront [Internet]. 2018 [cited 2023 Jan 31]. Available from: https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf.
7) Bengio Y, Ducharme R, Vincent P, Jauvin C. A Neural Probabilistic Language Model. J Mach Learn Res 2003;3:1137-1155.
8) Wu Y, Schuster M, Chen Z, Quoc V, Norouzi M, Macherey W et al. Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. arXiv [Internet]. 2016 [cited 2023 Jan 31]; 1609.08144v2. doi: 10.48550/arXiv.1609.08144.
9) Devlin J, Chang M-W, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv [Internet]. 2019 [cited 2023 Jan 31] 1810.04805v2. doi: 10.48550/arXiv.1810.04805.
10) Radford A, Narasimhan K, Salimans T, Sutskever I. Improving Language Understanding by Generative Pre-Training. Semantic Scholar [Internet]. 2018 [cited 2023 Jan 31]. Available from: https://www.semanticscholar.org/paper/Improving-Language-Understanding-by-Generative-Radford-Narasimhan/cd18800a0fe0b668a1cc19f2ec95b5003d0a5035.
11) Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P. Language Models are Few-Shot Learners. arXiv [Internet]. 2020 [cited 2023 Feb 1];14165v4. doi: 10.48550/arXiv.2005.14165.
12) Introducing ChatGPT. OpenAI [Internet]; c2015-2023 [cited 2023 Jan 30]. Available from: https://openai.com/blog/chatgpt/.
13) GPT-4 is OpenAI’s most advanced system, producing safer and more useful responses. OpenAI [Internet]; c2015-2023 [cited 2023 Mar 27]. Available from: https://openai.com/product/gpt-4/.
14) Thorp HH. ChatGPT is fun, but not an author. Science [Internet]. 2023 Jan [cited 2023 Feb 1]; 379(6630): 313. Available from: https://www.science.org/doi/10.1126/science.adg7879.
15) Ofgang E. What is GPTZero? The ChatGPT Detection Tool Explained By Its Creator. Tech & Learning [Internet]; 2023 Jan 27 [cited 2023 Jan 30]. Available from: https://www.techlearning.com/news/what-is-gptzero-the-chatgpt-detection-tool-explained.
16) Kirchner JH, Ahmad L, Aaronson S, Leike J. New AI classifier for indicating AI-written text. OpenAI [Internet] 2023 Jan 31 [cited 2023 Feb 2]. Available from: https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text/.
17) Stokel-Walker C. ChatGPT listed as author on research papers: many scientists disapprove. Nature 2023 Jan 18 [cited 2023 Feb 2];613:620-621. doi: 10.1038/d41586-023-00107-z.
18) Kung T, Cheatham M, ChatGPT, Medenilla A, Sillos C, De Leon L. Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models. Preprint at medRxiv [Internet] 2022 Dec 21 [cited 2023 Feb 2]. doi:10.1101/2022.12.19.22283643.
19) O'Connor S, ChatGPT. Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse? Nurse Educ Pract 2023;66:103537. doi:10.1016/j.nepr.2022.103537.
20) ChatGPT Generative Pre-trained Transformer, Zhavoronkov A. Rapamycin in the context of Pascal’s Wager: generative pre-trained transformer perspective. Oncoscience 2022;9:82-84. doi: 10.18632/oncoscience.571.
21) Gpt Generative Pretrained Transformer, Thunström A, Steingrimsson S. Can GPT-3 write an academic paper on itself, with minimal human input? HAL open science [Internet] 2022 [cited 2021 Feb 2]. Available from: https://hal.science/hal-03701250/document.
22) King M, chatGPT. A Conversation on Artificial Intelligence, Chatbots, and Plagiarism in Higher Education. Cell Mol Bioeng 2023;16:1-2. doi: 10.1007/s12195-022-00754-8.
23) For authors: initial submission. Nature; c2023 [cited 2023 Jan 31]. Available from: https://www.nature.com/nature/for-authors/initial-submission.
24) Defining the Role of Authors and Contributors. International Committee of Medical Journal Editors; c2023 [cited 2023 Jan 31]. Available from: https://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html.
25) Burcharth J, Pommergaard HC, Danielsen AK, Rosenberg J. Medical writers i medicinsk forskning [Medical writers in medical research]. Ugeskr Laeger 2013 Aug 19;175(34):1867-70.
26) Ultimate Guide to Becoming a Medical Writer. American Medical Writers Association; c2021 [cited 2023 Jan 31]. Available from: https://info.amwa.org/ultimate-guide-to-becoming-a-medical-writer#what_is_medical_writing.
For the PDF of this article,
contact nzmj@nzma.org.nz
View Article PDF
The idea of artificial intelligence (AI) was first introduced in the 1950s.[[1–4]] A subfield of AI is Natural Language Processing (NLP), which is the ability of a computer program to understand the human language.[[1,5]] However, the concept of large language models (LLM) as we know them today is a type of NLP[[6]] that is a program capable of producing human language and answers based on large data.[[7]] In recent years, many language models such as Google Neural Machine Translation (GNMT)[[8]] and the Bidirectional Encoder Representation from Transformers (BERT)[[9]] have been created. In 2018, OpenAI launched its first model, the Generative Pre-trained Transformer (GPT),[[10]] which was followed by further development resulting in GPT-2 and GPT-3.[[6,11]] In November 2022, OpenAI introduced ChatGPT, which is currently freely available online.[[12]] The latest version, GPT-4, was made publicly available for a user fee in March 2023. GPT-4 is currently the most advanced system available to the public, as it is based on more background information, with more advanced problem solving and greater accuracy.[[13]]
Since the launch of ChatGPT, it has garnered the attention of many people worldwide, including biomedical researchers, as it appears to be able to substantially assist in the reporting process of biomedical research (article writing). This raises questions such as: how should AI be understood in the context of research reporting? What can AI help researchers with? How should the implementation of ChatGPT be used in the world of research? Could this mean that a paradigm shift in the field of research reporting is on the horizon?
The capabilities of ChatGPT
ChatGPT can help researchers in various phases of writing scientific articles (Figure 1). We verified the answers given by the robot in Figure 1 and also tried other possible features in multiple sessions. It turned out that the robot can:
• identify relevant literature, information, and potential collaborators such as researchers and institutions.
• Identify relevant topics and trends in the respected research fields.
• Organise ideas and create an outline for an article.
• Conduct a literature review, such as providing relevant articles and studies.
• Write different sections of an article, such as the introduction, methods, results, and discussion.
• Produce an abstract that fits the article.
• Help with grammar, syntax, and style.
• Format the manuscript according to the journal’s guidelines.
• Give ideas as to how charts, graphs, and figures could be constructed if data is explained in text format (the answer would then also be in text format).
• Assist with the communication for research through blogs and social media by writing laymen’s descriptions and giving ideas as to what type of post could be relevant on social media.
• Write conference abstracts.
• Translate manuscripts into other languages.
• Write a covering letter.
• Produce title pages.
• Format references to specific citation styles.
These are some of the ways ChatGPT can assist researchers; however, the quality of the AI output has not yet been formally tested against corresponding human work. Nevertheless, it is obvious that the robot can provide substantial help for the researcher in the reporting phases of scientific work. This means that a paradigm shift may be on the way for how research is reported in the future, potentially making it possible to produce an astonishing number of scientific articles within a short time frame once we have learned the potential of the robotic platform and how it can assist the researcher without compromising on quality.
Even though AI can help researchers in numerous ways, it is not free from limitations. For example, ChatGPT sometimes provides incorrect answers, and it may reference an article that does not exist.[[14]] Furthermore, the possibility of bias in the responses is unknown as the end-user has no control of the input data sources for the robot. Thus, there is a theoretical risk that some information regarding a topic can be left out, which could possibly lead to misinformation being spread about a topic. An example could be that some controversial articles or data would be left out of the AI’s data. Finally, every researcher using AI should, of course, check the output information for credibility.
There has been concern that answers by the robot are so well-formulated and intelligent that it could be difficult to distinguish them from text produced by humans. However, new software can now detect AI-generated text such as GPTZero[[15]] and the AI classifier,[[16]] although they have not yet been tested systematically.
The issue of authorship
When a robot assists in the writing process of a scientific paper, it is necessary to consider whether the AI assistant should be accredited as a co-author in the byline. Another possibility would be to mention it in the methods section or give credit for the contribution in the acknowledgements section.
Since the launch of ChatGPT, publishers have been trying to create authorship policies for the new chatbot.[[14,17]] Currently, three articles and two preprints have an AI robot as a co-author.[[18–22]] Publishers and preprint servers typically agree that ChatGPT does not fulfill the authorship criteria because it cannot take responsibility for the content of the scientific paper.[[14,17]] Furthermore, the editors of Nature[[17]] and Science[[14]] will not allow AI to be listed as an author, and the publisher Taylor and Francis prefers that the AI be mentioned in the acknowledgements section as a contributor.[[17]] Due to increased use of AI, publishers will need to decide on authorship issues for this new player in the field. Nature, along with Springer Nature journals, have formulated two rules in their author guidelines[[23]]: “Large Language Models (LLMs), such as ChatGPT, do not currently satisfy our authorship criteria. Notably, attribution of authorship carries with it accountability for the work, which cannot be effectively applied to LLMs. Use of an LLM should be properly documented in the Methods section (and if a Methods section is not available, in a suitable alternative part) of the manuscript.” This is fully compliant with the authorship criteria described by the International Committee of Medical Journal Editors (ICMJE)[[24]]: “1) Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND 2) Drafting the work or revising it critically for important intellectual content; AND 3) Final approval of the version to be published; AND 4) Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.”
Thus, since the robot cannot be accountable for all aspects of the work, byline authorship is not an option for an LLM such as ChatGPT. Depending on the amount of contribution, it would be appropriate to mention ChatGPT in the methods section or as a formal contribution in the acknowledgements section.
Discussion
In its current version, the AI chatbot can, in principle, be seen as a medical writer. A medical writer is a professional author with skills in language and writing,[[25]] and they can provide grant writing, laymen descriptions, scientific articles, and more.[[ 25,26]] When outsourcing parts of the research process is already normal, why not outsource part of the writing process to a medical writer or an AI robot? We already use research assistants for data collection and statisticians for statistical analyses, so maybe it’s time to use AI for various phases of manuscript production. The cost for researchers using AI is substantially lower than paying for a medical writer, so maybe we could consider the AI robot as a low-cost medical writer.
An important issue, however, is the quality of the AI output. We don’t yet know if it is as good as a professional medical writer. With our limited experience at present, we seriously doubt that the AI quality is good enough, but since AI has learning capability and since the current versions of these models are still in their infancy, we don’t know what the future will bring.
Some researchers may be concerned that the ease of use and low cost may trigger research misconduct with fabricated results, unintentional errors, or deceptive publications. However, misconduct can occur with or without AI, and with our current knowledge of these systems, we don’t believe that it would become worse or better with AI for research reporting. Rather, AI in research reporting should be seen as a “low-cost medical writer”, although we are not fully there yet regarding quality.
In conclusion, AI could probably be used as a medical writer for at least some parts of the article production phases, but in our opinion, this does not mean that the AI should be listed as a co-author. Depending on the contribution, the AI could be thanked in the acknowledgements section or mentioned in the methods section of the paper. Documenting where the AI has assisted will heighten the transparency and credibility of the work. Furthermore, it should be noted that ChatGPT or GPT-4 are not yet at the level of a professional medical writer, and further investigation and research need to be conducted with AI to fully understand its capabilities. These interesting new developments could mean a drastic paradigm shift in the field of research reporting, where various tasks may soon be taken over by AI platforms. AI is available to every researcher, whereas a medical writer is only available with sufficient funding, so AI could potentially become more widespread than the use of a professional medical writer. These AI systems are still in their infancy, and the development is exponential. Therefore, we are facing substantial changes in research reporting where tasks other than article writing may become the main focus for researchers in biomedicine in the near future.
View Figure 1.
Summary
Abstract
Aim
ChatGPT and the newest GPT-4 are AI language models developed by OpenAI that have gained attention for their potential applications in biomedical research reporting. The models can assist researchers in various stages of writing scientific articles, including literature search, outlining, writing different sections, formatting, and translation. The use of ChatGPT or GPT-4 in research reporting has the potential to speed up the writing process, but its limitations, such as incorrect answers and biases, should also be considered. There is ongoing debate over the issue of AI authorship in scientific papers, with some publishers allowing it to be listed as a contributor in the acknowledgements section, while others do not allow it to be listed as an author. The use of ChatGPT or GPT-4 in research reporting is a recent development, and further studies and discussions are needed to determine their potential and limitations in this field.
Method
Results
Conclusion
Author Information
Acknowledgements
Correspondence
Correspondence Email
Competing Interests
1) Kaul V, Enslin S, Gross SA, History of artificial intelligence in medicine. Gastrointest Endosc. 2020 Oct;92(4): 807-812. doi: 10.1016/j.gie.2020.06.040.
2) Amisha, Malik P, Pathania M, Rathaur VK. Overview of artificial intelligence in medicine. J Family Med Prim Care. 2019 Jul;8(7):2328-2331. doi: 10.4103/jfmpc.jfmpc_440_19.
3) Ramesh AN, Kambhampati C, Monson JR, Drew PJ. Artificial intelligence in medicine. Ann R Coll Surg Engl. 2004 Sep;86(5):334-8. doi: 10.1308/147870804290.
4) Holmes JH, Sacchi L, Bellazzi R, Peek N. Artificial Intelligence in Medicine AIME 2015. Artif Intell Med. 2017 Sep;81:1-2. doi: 10.1016/j.artmed.2017.06.011.
5) Lutkevich B, Burns E. Natural language processing (NLP) [Internet]. TechTarget. 2023 [cited 2023 Jan 31]. Available from: www.techtarget.com/searchenterpriseai/definition/natural-language-processing-NLP.
6) Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Language Models are Unsupervised Multitask Learners. Cloudfront [Internet]. 2018 [cited 2023 Jan 31]. Available from: https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf.
7) Bengio Y, Ducharme R, Vincent P, Jauvin C. A Neural Probabilistic Language Model. J Mach Learn Res 2003;3:1137-1155.
8) Wu Y, Schuster M, Chen Z, Quoc V, Norouzi M, Macherey W et al. Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. arXiv [Internet]. 2016 [cited 2023 Jan 31]; 1609.08144v2. doi: 10.48550/arXiv.1609.08144.
9) Devlin J, Chang M-W, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv [Internet]. 2019 [cited 2023 Jan 31] 1810.04805v2. doi: 10.48550/arXiv.1810.04805.
10) Radford A, Narasimhan K, Salimans T, Sutskever I. Improving Language Understanding by Generative Pre-Training. Semantic Scholar [Internet]. 2018 [cited 2023 Jan 31]. Available from: https://www.semanticscholar.org/paper/Improving-Language-Understanding-by-Generative-Radford-Narasimhan/cd18800a0fe0b668a1cc19f2ec95b5003d0a5035.
11) Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P. Language Models are Few-Shot Learners. arXiv [Internet]. 2020 [cited 2023 Feb 1];14165v4. doi: 10.48550/arXiv.2005.14165.
12) Introducing ChatGPT. OpenAI [Internet]; c2015-2023 [cited 2023 Jan 30]. Available from: https://openai.com/blog/chatgpt/.
13) GPT-4 is OpenAI’s most advanced system, producing safer and more useful responses. OpenAI [Internet]; c2015-2023 [cited 2023 Mar 27]. Available from: https://openai.com/product/gpt-4/.
14) Thorp HH. ChatGPT is fun, but not an author. Science [Internet]. 2023 Jan [cited 2023 Feb 1]; 379(6630): 313. Available from: https://www.science.org/doi/10.1126/science.adg7879.
15) Ofgang E. What is GPTZero? The ChatGPT Detection Tool Explained By Its Creator. Tech & Learning [Internet]; 2023 Jan 27 [cited 2023 Jan 30]. Available from: https://www.techlearning.com/news/what-is-gptzero-the-chatgpt-detection-tool-explained.
16) Kirchner JH, Ahmad L, Aaronson S, Leike J. New AI classifier for indicating AI-written text. OpenAI [Internet] 2023 Jan 31 [cited 2023 Feb 2]. Available from: https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text/.
17) Stokel-Walker C. ChatGPT listed as author on research papers: many scientists disapprove. Nature 2023 Jan 18 [cited 2023 Feb 2];613:620-621. doi: 10.1038/d41586-023-00107-z.
18) Kung T, Cheatham M, ChatGPT, Medenilla A, Sillos C, De Leon L. Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models. Preprint at medRxiv [Internet] 2022 Dec 21 [cited 2023 Feb 2]. doi:10.1101/2022.12.19.22283643.
19) O'Connor S, ChatGPT. Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse? Nurse Educ Pract 2023;66:103537. doi:10.1016/j.nepr.2022.103537.
20) ChatGPT Generative Pre-trained Transformer, Zhavoronkov A. Rapamycin in the context of Pascal’s Wager: generative pre-trained transformer perspective. Oncoscience 2022;9:82-84. doi: 10.18632/oncoscience.571.
21) Gpt Generative Pretrained Transformer, Thunström A, Steingrimsson S. Can GPT-3 write an academic paper on itself, with minimal human input? HAL open science [Internet] 2022 [cited 2021 Feb 2]. Available from: https://hal.science/hal-03701250/document.
22) King M, chatGPT. A Conversation on Artificial Intelligence, Chatbots, and Plagiarism in Higher Education. Cell Mol Bioeng 2023;16:1-2. doi: 10.1007/s12195-022-00754-8.
23) For authors: initial submission. Nature; c2023 [cited 2023 Jan 31]. Available from: https://www.nature.com/nature/for-authors/initial-submission.
24) Defining the Role of Authors and Contributors. International Committee of Medical Journal Editors; c2023 [cited 2023 Jan 31]. Available from: https://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html.
25) Burcharth J, Pommergaard HC, Danielsen AK, Rosenberg J. Medical writers i medicinsk forskning [Medical writers in medical research]. Ugeskr Laeger 2013 Aug 19;175(34):1867-70.
26) Ultimate Guide to Becoming a Medical Writer. American Medical Writers Association; c2021 [cited 2023 Jan 31]. Available from: https://info.amwa.org/ultimate-guide-to-becoming-a-medical-writer#what_is_medical_writing.
For the PDF of this article,
contact nzmj@nzma.org.nz
View Article PDF
The idea of artificial intelligence (AI) was first introduced in the 1950s.[[1–4]] A subfield of AI is Natural Language Processing (NLP), which is the ability of a computer program to understand the human language.[[1,5]] However, the concept of large language models (LLM) as we know them today is a type of NLP[[6]] that is a program capable of producing human language and answers based on large data.[[7]] In recent years, many language models such as Google Neural Machine Translation (GNMT)[[8]] and the Bidirectional Encoder Representation from Transformers (BERT)[[9]] have been created. In 2018, OpenAI launched its first model, the Generative Pre-trained Transformer (GPT),[[10]] which was followed by further development resulting in GPT-2 and GPT-3.[[6,11]] In November 2022, OpenAI introduced ChatGPT, which is currently freely available online.[[12]] The latest version, GPT-4, was made publicly available for a user fee in March 2023. GPT-4 is currently the most advanced system available to the public, as it is based on more background information, with more advanced problem solving and greater accuracy.[[13]]
Since the launch of ChatGPT, it has garnered the attention of many people worldwide, including biomedical researchers, as it appears to be able to substantially assist in the reporting process of biomedical research (article writing). This raises questions such as: how should AI be understood in the context of research reporting? What can AI help researchers with? How should the implementation of ChatGPT be used in the world of research? Could this mean that a paradigm shift in the field of research reporting is on the horizon?
The capabilities of ChatGPT
ChatGPT can help researchers in various phases of writing scientific articles (Figure 1). We verified the answers given by the robot in Figure 1 and also tried other possible features in multiple sessions. It turned out that the robot can:
• identify relevant literature, information, and potential collaborators such as researchers and institutions.
• Identify relevant topics and trends in the respected research fields.
• Organise ideas and create an outline for an article.
• Conduct a literature review, such as providing relevant articles and studies.
• Write different sections of an article, such as the introduction, methods, results, and discussion.
• Produce an abstract that fits the article.
• Help with grammar, syntax, and style.
• Format the manuscript according to the journal’s guidelines.
• Give ideas as to how charts, graphs, and figures could be constructed if data is explained in text format (the answer would then also be in text format).
• Assist with the communication for research through blogs and social media by writing laymen’s descriptions and giving ideas as to what type of post could be relevant on social media.
• Write conference abstracts.
• Translate manuscripts into other languages.
• Write a covering letter.
• Produce title pages.
• Format references to specific citation styles.
These are some of the ways ChatGPT can assist researchers; however, the quality of the AI output has not yet been formally tested against corresponding human work. Nevertheless, it is obvious that the robot can provide substantial help for the researcher in the reporting phases of scientific work. This means that a paradigm shift may be on the way for how research is reported in the future, potentially making it possible to produce an astonishing number of scientific articles within a short time frame once we have learned the potential of the robotic platform and how it can assist the researcher without compromising on quality.
Even though AI can help researchers in numerous ways, it is not free from limitations. For example, ChatGPT sometimes provides incorrect answers, and it may reference an article that does not exist.[[14]] Furthermore, the possibility of bias in the responses is unknown as the end-user has no control of the input data sources for the robot. Thus, there is a theoretical risk that some information regarding a topic can be left out, which could possibly lead to misinformation being spread about a topic. An example could be that some controversial articles or data would be left out of the AI’s data. Finally, every researcher using AI should, of course, check the output information for credibility.
There has been concern that answers by the robot are so well-formulated and intelligent that it could be difficult to distinguish them from text produced by humans. However, new software can now detect AI-generated text such as GPTZero[[15]] and the AI classifier,[[16]] although they have not yet been tested systematically.
The issue of authorship
When a robot assists in the writing process of a scientific paper, it is necessary to consider whether the AI assistant should be accredited as a co-author in the byline. Another possibility would be to mention it in the methods section or give credit for the contribution in the acknowledgements section.
Since the launch of ChatGPT, publishers have been trying to create authorship policies for the new chatbot.[[14,17]] Currently, three articles and two preprints have an AI robot as a co-author.[[18–22]] Publishers and preprint servers typically agree that ChatGPT does not fulfill the authorship criteria because it cannot take responsibility for the content of the scientific paper.[[14,17]] Furthermore, the editors of Nature[[17]] and Science[[14]] will not allow AI to be listed as an author, and the publisher Taylor and Francis prefers that the AI be mentioned in the acknowledgements section as a contributor.[[17]] Due to increased use of AI, publishers will need to decide on authorship issues for this new player in the field. Nature, along with Springer Nature journals, have formulated two rules in their author guidelines[[23]]: “Large Language Models (LLMs), such as ChatGPT, do not currently satisfy our authorship criteria. Notably, attribution of authorship carries with it accountability for the work, which cannot be effectively applied to LLMs. Use of an LLM should be properly documented in the Methods section (and if a Methods section is not available, in a suitable alternative part) of the manuscript.” This is fully compliant with the authorship criteria described by the International Committee of Medical Journal Editors (ICMJE)[[24]]: “1) Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work; AND 2) Drafting the work or revising it critically for important intellectual content; AND 3) Final approval of the version to be published; AND 4) Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.”
Thus, since the robot cannot be accountable for all aspects of the work, byline authorship is not an option for an LLM such as ChatGPT. Depending on the amount of contribution, it would be appropriate to mention ChatGPT in the methods section or as a formal contribution in the acknowledgements section.
Discussion
In its current version, the AI chatbot can, in principle, be seen as a medical writer. A medical writer is a professional author with skills in language and writing,[[25]] and they can provide grant writing, laymen descriptions, scientific articles, and more.[[ 25,26]] When outsourcing parts of the research process is already normal, why not outsource part of the writing process to a medical writer or an AI robot? We already use research assistants for data collection and statisticians for statistical analyses, so maybe it’s time to use AI for various phases of manuscript production. The cost for researchers using AI is substantially lower than paying for a medical writer, so maybe we could consider the AI robot as a low-cost medical writer.
An important issue, however, is the quality of the AI output. We don’t yet know if it is as good as a professional medical writer. With our limited experience at present, we seriously doubt that the AI quality is good enough, but since AI has learning capability and since the current versions of these models are still in their infancy, we don’t know what the future will bring.
Some researchers may be concerned that the ease of use and low cost may trigger research misconduct with fabricated results, unintentional errors, or deceptive publications. However, misconduct can occur with or without AI, and with our current knowledge of these systems, we don’t believe that it would become worse or better with AI for research reporting. Rather, AI in research reporting should be seen as a “low-cost medical writer”, although we are not fully there yet regarding quality.
In conclusion, AI could probably be used as a medical writer for at least some parts of the article production phases, but in our opinion, this does not mean that the AI should be listed as a co-author. Depending on the contribution, the AI could be thanked in the acknowledgements section or mentioned in the methods section of the paper. Documenting where the AI has assisted will heighten the transparency and credibility of the work. Furthermore, it should be noted that ChatGPT or GPT-4 are not yet at the level of a professional medical writer, and further investigation and research need to be conducted with AI to fully understand its capabilities. These interesting new developments could mean a drastic paradigm shift in the field of research reporting, where various tasks may soon be taken over by AI platforms. AI is available to every researcher, whereas a medical writer is only available with sufficient funding, so AI could potentially become more widespread than the use of a professional medical writer. These AI systems are still in their infancy, and the development is exponential. Therefore, we are facing substantial changes in research reporting where tasks other than article writing may become the main focus for researchers in biomedicine in the near future.
View Figure 1.
Summary
Abstract
Aim
ChatGPT and the newest GPT-4 are AI language models developed by OpenAI that have gained attention for their potential applications in biomedical research reporting. The models can assist researchers in various stages of writing scientific articles, including literature search, outlining, writing different sections, formatting, and translation. The use of ChatGPT or GPT-4 in research reporting has the potential to speed up the writing process, but its limitations, such as incorrect answers and biases, should also be considered. There is ongoing debate over the issue of AI authorship in scientific papers, with some publishers allowing it to be listed as a contributor in the acknowledgements section, while others do not allow it to be listed as an author. The use of ChatGPT or GPT-4 in research reporting is a recent development, and further studies and discussions are needed to determine their potential and limitations in this field.
Method
Results
Conclusion
Author Information
Acknowledgements
Correspondence
Correspondence Email
Competing Interests
1) Kaul V, Enslin S, Gross SA, History of artificial intelligence in medicine. Gastrointest Endosc. 2020 Oct;92(4): 807-812. doi: 10.1016/j.gie.2020.06.040.
2) Amisha, Malik P, Pathania M, Rathaur VK. Overview of artificial intelligence in medicine. J Family Med Prim Care. 2019 Jul;8(7):2328-2331. doi: 10.4103/jfmpc.jfmpc_440_19.
3) Ramesh AN, Kambhampati C, Monson JR, Drew PJ. Artificial intelligence in medicine. Ann R Coll Surg Engl. 2004 Sep;86(5):334-8. doi: 10.1308/147870804290.
4) Holmes JH, Sacchi L, Bellazzi R, Peek N. Artificial Intelligence in Medicine AIME 2015. Artif Intell Med. 2017 Sep;81:1-2. doi: 10.1016/j.artmed.2017.06.011.
5) Lutkevich B, Burns E. Natural language processing (NLP) [Internet]. TechTarget. 2023 [cited 2023 Jan 31]. Available from: www.techtarget.com/searchenterpriseai/definition/natural-language-processing-NLP.
6) Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Language Models are Unsupervised Multitask Learners. Cloudfront [Internet]. 2018 [cited 2023 Jan 31]. Available from: https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf.
7) Bengio Y, Ducharme R, Vincent P, Jauvin C. A Neural Probabilistic Language Model. J Mach Learn Res 2003;3:1137-1155.
8) Wu Y, Schuster M, Chen Z, Quoc V, Norouzi M, Macherey W et al. Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. arXiv [Internet]. 2016 [cited 2023 Jan 31]; 1609.08144v2. doi: 10.48550/arXiv.1609.08144.
9) Devlin J, Chang M-W, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv [Internet]. 2019 [cited 2023 Jan 31] 1810.04805v2. doi: 10.48550/arXiv.1810.04805.
10) Radford A, Narasimhan K, Salimans T, Sutskever I. Improving Language Understanding by Generative Pre-Training. Semantic Scholar [Internet]. 2018 [cited 2023 Jan 31]. Available from: https://www.semanticscholar.org/paper/Improving-Language-Understanding-by-Generative-Radford-Narasimhan/cd18800a0fe0b668a1cc19f2ec95b5003d0a5035.
11) Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P. Language Models are Few-Shot Learners. arXiv [Internet]. 2020 [cited 2023 Feb 1];14165v4. doi: 10.48550/arXiv.2005.14165.
12) Introducing ChatGPT. OpenAI [Internet]; c2015-2023 [cited 2023 Jan 30]. Available from: https://openai.com/blog/chatgpt/.
13) GPT-4 is OpenAI’s most advanced system, producing safer and more useful responses. OpenAI [Internet]; c2015-2023 [cited 2023 Mar 27]. Available from: https://openai.com/product/gpt-4/.
14) Thorp HH. ChatGPT is fun, but not an author. Science [Internet]. 2023 Jan [cited 2023 Feb 1]; 379(6630): 313. Available from: https://www.science.org/doi/10.1126/science.adg7879.
15) Ofgang E. What is GPTZero? The ChatGPT Detection Tool Explained By Its Creator. Tech & Learning [Internet]; 2023 Jan 27 [cited 2023 Jan 30]. Available from: https://www.techlearning.com/news/what-is-gptzero-the-chatgpt-detection-tool-explained.
16) Kirchner JH, Ahmad L, Aaronson S, Leike J. New AI classifier for indicating AI-written text. OpenAI [Internet] 2023 Jan 31 [cited 2023 Feb 2]. Available from: https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text/.
17) Stokel-Walker C. ChatGPT listed as author on research papers: many scientists disapprove. Nature 2023 Jan 18 [cited 2023 Feb 2];613:620-621. doi: 10.1038/d41586-023-00107-z.
18) Kung T, Cheatham M, ChatGPT, Medenilla A, Sillos C, De Leon L. Performance of ChatGPT on USMLE: Potential for AI-Assisted Medical Education Using Large Language Models. Preprint at medRxiv [Internet] 2022 Dec 21 [cited 2023 Feb 2]. doi:10.1101/2022.12.19.22283643.
19) O'Connor S, ChatGPT. Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse? Nurse Educ Pract 2023;66:103537. doi:10.1016/j.nepr.2022.103537.
20) ChatGPT Generative Pre-trained Transformer, Zhavoronkov A. Rapamycin in the context of Pascal’s Wager: generative pre-trained transformer perspective. Oncoscience 2022;9:82-84. doi: 10.18632/oncoscience.571.
21) Gpt Generative Pretrained Transformer, Thunström A, Steingrimsson S. Can GPT-3 write an academic paper on itself, with minimal human input? HAL open science [Internet] 2022 [cited 2021 Feb 2]. Available from: https://hal.science/hal-03701250/document.
22) King M, chatGPT. A Conversation on Artificial Intelligence, Chatbots, and Plagiarism in Higher Education. Cell Mol Bioeng 2023;16:1-2. doi: 10.1007/s12195-022-00754-8.
23) For authors: initial submission. Nature; c2023 [cited 2023 Jan 31]. Available from: https://www.nature.com/nature/for-authors/initial-submission.
24) Defining the Role of Authors and Contributors. International Committee of Medical Journal Editors; c2023 [cited 2023 Jan 31]. Available from: https://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html.
25) Burcharth J, Pommergaard HC, Danielsen AK, Rosenberg J. Medical writers i medicinsk forskning [Medical writers in medical research]. Ugeskr Laeger 2013 Aug 19;175(34):1867-70.
26) Ultimate Guide to Becoming a Medical Writer. American Medical Writers Association; c2021 [cited 2023 Jan 31]. Available from: https://info.amwa.org/ultimate-guide-to-becoming-a-medical-writer#what_is_medical_writing.
Contact diana@nzma.org.nz
for the PDF of this article
The impact of AI and ChatGPT on research reporting
Since the launch of ChatGPT, it has garnered the attention of many people worldwide, including biomedical researchers, as it appears to be able to substantially assist in the reporting process of biomedical research (article writing).
Subscriber Content
The full contents of this pages only available to subscribers.
Login, subscribe or email nzmj@nzma.org.nz to purchase this article.
