The post-pandemic period has been a challenging time for health systems, both in New Zealand and globally.   They have buckled under strain caused by a convergence of negative factors including the COVID-19 pandemic itself, chronic under-funding and inadequate workforces for dealing with ageing populations, producing a crisis of care provision that doesn’t appear to have an immediate solution.[[1]]

This era has also seen the explosive growth of artificial intelligence (AI) powered applications across all aspects of our daily lives, which includes increasing momentum in medicine. As we approach the anniversary of the release of Open AI’s Chat-GPT3, society has realised that the introduction of these large language models (LLMs) represents one of the most significant technological transformations in history. It is hailed as a new Promethean moment, rivalling other pivotal moments such as the inventions of the printing press, nuclear energy and the internet. The potential disruptive effect of this technology and AI in general is immense—as Microsoft’s former chief strategist, Craig Mundie put it: “this is going to change everything about how we do everything”.[[2]]

Does the transformative power of AI hold out hope for the future of healthcare, or have we opened a Pandora’s box that will create more problems for the system than it solves?

“Artificial intelligence” is defined as the theory and development of computer systems with the ability to reason and perform cognitive functions such as problem solving, object and word recognition, and decision making. In addition to natural language processing applications such as Chat-GPT, it includes the overlapping subfields of machine learning, artificial neural networks and computer vision.

AI already has multiple established applications in healthcare. At the time of writing, no fewer than 80 AI-based algorithms or devices had gained FDA approval for clinical use for applications including analysing medical images and digital pathology slides, detecting drug interactions, identifying high-risk patients and coding medical notes.[[3]] AI algorithms have demonstrated equivalence with humans for performance in simple radiological, pathological and dermatological diagnostic tasks,[[4–6]] with recent clinical studies evidencing their positive impact on clinician performance with possible benefits for patient outcomes.[[7]] While such patient-centred applications have obvious benefits to individuals in terms of improving the accuracy and efficiency of reporting, other algorithms are deployed at a systems level, predicting key outcomes such as hospital readmission, mortality and sepsis.[[8]]

While these substantial gains have already been realised, the immediate future holds even more significant potential with the recent advent of generative AI such as Chat-GPT; that is, AI that can not only analyse huge volumes of text and other data, but also compose it. LLMs have the ability to rapidly synthesise high-volume, complex patient data and generate reports—for example, they could complete consultation notes or discharge summaries. Employing these algorithms may liberate clinician time to refocus on more patient-centred personal interactions, not only improving the patient experience but potentially reducing physician burnout.[[9]]

While AI has the potential to increase efficiency and optimise resource utilisation, it has pitfalls, unresolved issues and potential harmful consequences. The reliance of AI on huge datasets for model training has been a significant barrier to clinical progress. The sensitive nature of this healthcare data mandates that it be captured, stored and used in concordance with stringent ethical and privacy requirements, including protecting the rights of Indigenous peoples and recognising their data sovereignty. This is an area of rapidly evolving governance and regulation that, in the New Zealand context, has been supported by a recent update to the Privacy Act and the work of Te Mana Raraunga. Compounding this is the ongoing debate around data ownership among different stakeholders, with significant patient concern regarding the use of health data for potential commercial gain.

Where algorithms have been developed, the lack of clear reporting standards has made quality assessment difficult. This problem is exacerbated by the fact that algorithms often lack transparency, with deep learning models operating as “black boxes” that do not provide the basis of the reasoning behind their decisions, affecting the ability of patients and physicians alike to trust AI-generated recommendations and to achieve true informed consent for patients. This is further compounded by proprietary datasets’ lack of transparency precluding demographic comparisons with the populations they are to be deployed in. Recognition of software as a medical device should ensure minimum safety standards are met, but human oversight will still be needed to ensure model outputs are context appropriate and add value to clinical practice, as was well illustrated by the challenges of using an AI model to screen for diabetic retinopathy in rural Thailand.[[10]] Even if the input data is appropriate, malfunctions remain a real threat to the clinical utility of AI algorithms; for example, it is well recognised that even the latest LLMs experience “hallucinations” producing wildly inaccurate outputs. If an algorithm does produce an error, the question of liability arises and whether this lies with the algorithm, the manufacturer or the treating doctor.

These concerns, specific to medicine, are amplified in the context of wider societal apprehension emerging in response to the exponential and unregulated development of AI. Of note, these concerns have been voiced by the creators of AI themselves, as evidenced by the results of a survey showing the majority of AI developers believe there is a more than 10% probability that humans will not be able to control further advancements in AI, leading to “human extinction”.[[11]] Historian and author Yuval Noah Harari eloquently summarised the concerns of the power of LLMs and the potential for loss of human control of them. Harari postulates that humanity is defined by its ability to generate language and it represents the “operating system” of human culture, and as such “AI’s mastery of language means it can now hack the operating system of civilisation”.[[12]]

Despite the apprehension of and potential pitfalls with AI, there is no doubt it is here to stay and will have a huge impact on medicine going forward. Governance and regulatory bodies require agility to deal with the pace of change, such that it doesn’t outstrip our ability to control it. In medicine this mandates engagement of clinicians in both the development and governance of AI.

As mentioned, AI algorithms rely on high-quality data for successful development and, in preparation for this, data science principles need to be integrated into clinical workflows as standard practice. An example from the authors’ institution is the end-to-end data capture used for laparoscopic cholecystectomy (LC) including synoptic operating reports. This prospective, standardised, automatic data collection has already allowed for process analysis and the application of AI-powered algorithms for operative phase recognition and difficulty grading in LC and paves the way for automated operative report generation and computer vision-based audit of critical operative steps.[[13,14]] This integration of standardised data collection is all very well in a single institution; however, it needs to be implemented at a whole of system level. This has been recognised by the Artificial Intelligence for Health in New Zealand report,[[15]] the Digital Health Association and Health Informatics New Zealand. Examples of progress towards this goal include the establishment of the Te Whatu Ora Health Information Standards Organisation, i3 and initiatives such as Canshare, a national health informatics platform that aims to standardise the collection of data as part of clinical workflows. This has, for example, enabled progression of the authors’ SynOPsys-CRC project to implement a standardised synoptic operating report for all colorectal cancer operations across New Zealand and Australia, an initiative that will provide high-quality data suitable for the development of AI algorithms to monitor the effect of operative process on outcomes.

Using these examples of AI in surgery serve to remind us that AI is an instrument and, much like a surgical instrument, it will work well if it is used properly, but if not it can do damage. AI is in a nascent phase in medicine but has already shown utility in many areas in the field and is here for the long term, even if it has opened a Pandora’s box of regulatory, ethical and legal issues. Its potential to improve efficiency, accuracy and overall quality of care is so great the profession must confront these challenges. Overcoming them, allowing the benefits of AI to be leveraged, may at best provide a solution to the healthcare crisis and, at least, augment the performance of health professionals while leaving more time for doctor-patient interactions, making medicine more rewarding again for patients and doctors alike.