ChatGPT as a fourth arbitrator? The ethics and risks of using large language models in arbitration

Abstract

Introduction

The increasing application of large language models (LLMs) has the potential to become the primary driver of digital economic development and prompt significant changes in how many products and services are produced.¹ LLMs like ChatGPT and Llama 2 can now be used for natural language processing tasks, such as language translation, text summarization, question answering, and text completion.² Increasingly, these technologies have the potential to serve as the basis of new applications through their integration into chatbots and virtual assistants.³

The digital transformation is evident in everyday life and can potentially disrupt many professions, including law and arbitration. One of the most significant advantages of arbitration over litigation is avoiding unnecessary time delays and expenses.⁴ LLMs can enhance this advantage by automating various tasks; they can quickly draft decisions or perform document and evidence analysis and legal research.⁵ These advantages have generated strong enthusiasm for integrating LLMs as a tool to be used by arbitrators and practitioners in future arbitral proceedings.⁶

However, in the arbitration context, adopting LLMs comes with some risks. First, they lack a database of arbitration cases and may struggle to handle the substantial volume of information generated during the arbitration process, potentially leading to inaccurate outputs⁷ and ‘hallucinations’.⁸ Second, these models receive data from users, but their terms of use are hazy with regard to data security—this may pose a risk to the confidentiality of arbitration proceedings. Finally, if these models are used to draft decisions in a manner that is not fully disclosed or agreed upon by the parties, trust in arbitral proceedings may be eroded.

Given this context, this article examines the potential for LLMs to change how arbitration proceedings are conducted and strategies for mitigating the risks associated with using these models in the arbitration context. Further, this article explores the ethical issues associated with applying LLMs in arbitration. Some work has been done in this regard, namely by the Silicon Valley Arbitration and Mediation Center (SVAMC), which, in its guidelines, has identified two general principles that arbitrators must respect when using AI in arbitration, namely, non-delegation of decision-making responsibilities and respect for due process. Still, this article argues that there is a need to establish clear, practical boundaries that define the appropriate scope of application for LLMs for arbitrators and that identify behaviours that should be avoided.

Following this introduction, the second section of this article discusses the potential benefits of LLMs in legal work in general and arbitration in particular. The third section explores the operational and confidentiality risks associated with using LLMs in arbitration while also proposing strategies to enhance content reliability and diminish the risk of infringing confidentiality obligations. Finally, in the fourth section, the article proposes a set of boundaries regarding LLMs in response to the challenges posed by their potential misuse and impact on the personal mandate of arbitrators. In particular, it proposes that there are likely uses which, given the nature of arbitration and arbitral awards, are simply not allowable; uses that may require disclosure and approval by the parties; and a few occasions where the arbitral tribunal may freely use an LLM in its own discretion.

The spectacular rise of LLMs and their power to transform justice

Due to the technological advancements of LLMs, tools like ChatGPT can produce fluent descriptive responses to user queries, enabling many professionals to productively use this tool to assist in their work.⁹ Within a few months of ChatGPT’s release, law firms and legal tech companies were already announcing new ways of using generative AI tools.¹⁰ LLMs excel at retrieving data and generating summarizations of information.¹¹ For legal professionals managing a large volume of information, LLMs can generate summaries that cover key points, such as the facts and disputed issues, based on the content of multiple documents, helping legal professionals to navigate proceedings and legal transactions.¹²

Moreover, LLMs can expand legal texts by producing new content in a question-and-answer format, which is especially useful for legal document drafting. In the context of drafting legal documents, LLMs can further detect potential issues and provide suggestions, thus also operating as reviewers of legal documents.¹³ LLMs also have some potential to provide legal consulting, as they can supply automated legal advisory services through their knowledge-generation capabilities.¹⁴

LLMs may not only play a role in the general legal applications mentioned above but also have demonstrated potential value in arbitration. They can learn from arbitration precedents and other arbitration-related texts to identify and understand common terms, practices, and interpretations.¹⁵ In producing texts, LLMs can help arbitrators and arbitration lawyers produce awards and other documents faster and uncover legal perspectives they may not have considered otherwise.

The potential of LLMs to exponentially enhance the quality of machine-made translations may be a game-changer in arbitration. Especially in international arbitration, arbitrators often face the challenges of dealing with legal documents in different languages and considering varying legal norms.¹⁶ In this context, arbitrators need to deeply understand the contextual relationships between various texts, and LLMs provide ample support through their cross-legal and cross-cultural understanding capabilities.¹⁷

Given all these potential applications, it is natural that arbitration has begun to explore using LLMs, including the practical implementation of these technologies and the rules that should govern their usage.¹⁸ For example, the SVAMC has established a task force to draft guidelines on the use of AI in international arbitration, published on 30 April 2024.¹⁹ In addition, commercially minded products, such as Jus-AI, developed by Jus Mundi, have been designed to evaluate document relevance and purportedly to provide more informed decision-making.²⁰ Some arbitral institutions have also explored AI-based solutions for arbitration. Most notably, the Guangzhou Arbitration Commission, a large arbitration institution in southern China, has developed an AI arbitration secretary designed to help with many of the administrative tasks that would otherwise need to be completed by the parties, the arbitral tribunal, or the institution.²¹

The dominant sentiment appears to be that, with the continuous advancement of technology and deepening acceptance among professionals, LLMs will play an increasingly significant role in all parts of the legal industry. As noted by a commentator:

Fuelled by the progress made by digital technologies and artificial intelligence, most bar associations and other legal groups have voiced their support to the legaltech industry and agree that the path forward should be to work with them and not against them.²²

Potential risks of LLMs in arbitration and strategies to mitigate risk

While using LLMs in arbitration may bring significant benefits, it also comes with potential risks,²³ two of which concern the usage of the models in arbitral proceedings.²⁴ First, these models have been known to produce inaccurate results and sometimes create fabricated but plausible-sounding information that may not be immediately detectable. Second, the use of these models may give rise to concerns regarding the confidentiality of the information shared when using these products. Both issues necessitate adopting strategies to mitigate these risks, ensuring the safe and effective use of LLMs in the arbitral context.

Potential operational risks and strategies for mitigation

Because of the potential for using LLMs as a tool in different areas, it is easy to overestimate their true capabilities and downplay their weaknesses. While these models have an uncanny ability to mimic human written texts, they are not ‘intelligent’ in a human sense and have innate limitations to what they can achieve. As described in a paper on LLMs, the texts produced are the result of highly complex probabilistic learning systems, not any truly independent thought:

Text generated by an LM is not grounded in communicative intent, any model of the world, or any model of the reader’s state of mind. The training data never included sharing thoughts with a listener, nor does the machine have the ability to do that. This can seem counter-intuitive given the increasingly fluent qualities of automatically generated text, but contrary to how it may seem when we observe its output, an LM is a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning: a stochastic parrot.²⁵

In more practical terms, arbitrators and lawyers using LLMs in connection with arbitral proceedings must contend with the potential lack of reliability of the content generated by these models. First, the content generated by these models may suffer from logical discontinuities,²⁶ as the generated text may lose topical relevance when responding to questions. This issue is particularly pronounced in arbitration cases involving specialized domain knowledge, as the models may not accurately understand and integrate complex concepts specific to that field. Therefore, if these models are used to generate arbitration materials or summaries in the arbitration process, the generated documents may contain content unrelated to the question at hand or exhibit unclear logic.²⁷

A simple experience reported in an online source illustrates the potential problems of using these tools in arbitration (and, more broadly, in the legal context). A scholar asked ChatGPT to explain the concept of jurisdiction in investment arbitration, and ChatGPT responded by providing information on the formation of settlement agreements under domestic law, a concept irrelevant to the topic.²⁸ This experience is often reported by those who use these tools, highlighting the current limitations of these models.

Second, LLMs may produce information that is not only inaccurate but also affirms facts that, despite seeming true, are entirely made-up, a phenomenon generally referred to as ‘hallucinations’.²⁹ In this regard, a study by the Stanford RegLab and the Institute for Human-Centered AI researchers demonstrated that legal hallucinations are pervasive: hallucination rates range from 69 per cent to 88 per cent in response to specific legal queries for state-of-the-art LLMs.³⁰ These models often lack self-awareness about their errors and double down on their incorrect statements about the law when challenged on their accuracy.

Arbitrators using these models may encounter fabricated information, sometimes in ways that appear plausible but are fundamentally wrong.³¹ For example, to illustrate the seriousness of the misinformation that an LLM can produce, a legal scholar asked ChatGPT to list news reports about law professors’ involvement in criminal activity. ChatGPT quickly generated relevant content, including specific names of individuals, their affiliations, and the crimes committed, and even provided precise sources and dates for the news. However, despite searching through all the available information sources, the scholar found no supporting information for the ‘facts’ described, making it clear that the information described was fabricated.³² The concerning aspect of this experiment was that the AI-generated answers involved real names of individuals and their affiliations, as well as real news websites and newspapers. The only false aspect was the details of the crimes committed, highlighting the potential for this LLM to produce highly misleading information.

This experiment highlights the need to exercise professional and logical judgment when assessing any content generated with the help of LLMs in an arbitral setting. Human review remains necessary for any task accomplished with the help of an LLM, necessitating fact-checking and confirming the accuracy of the model’s generated content.³³ Moreover, understanding the limitations of these models is essential to allow arbitrators and lawyers to understand the tasks that can be delegated to an LLM.³⁴ An enhanced understanding of these models also makes users more prone to identify potential mistakes in the content produced. Finally, developing expertise in designing the best inputs to accomplish a particular task is another important aspect of fully taking advantage of these LLMs and diminishing the risk of undesired results.

Potential confidentiality risks and strategies for mitigation

Beyond the problems associated with the lack of reliability of the content produced, there are risks in using LLMs in arbitration that implicate data privacy and security concerns. LLMs may store data received from users in ways that are not always clear, which has raised concerns about the privacy of users’ data and the information shared with these models. In particular, there has been a lack of clarity regarding how the information input into these models is stored and to what extent it can be used when generating responses for other users.³⁵

Some individuals have been able to bypass the security restrictions of LLMs, such as ChatGPT, by jailbreaking and, from there, obtaining information regarding users of the models and other confidential information.³⁶ Scholars have issued warnings about this risk:

Numerous research papers have proven that this precise vulnerability is a troubling reality. One research paper, for instance, demonstrated an attack that successfully exposed personally-identifiable information (including an individual’s name, email address, phone number, fax number, and physical address) by querying an LLM trained on public scrapes of the Internet.³⁷

The potential for user information leakages further amplifies the confidentiality risks of using LLMs. Data leaks have become prevalent across online tools and sites,³⁸ and ChatGPT and other models are not immune. Further, according to a Search Engine Journal report, many ChatGPT accounts are being sold on the dark web,³⁹ highlighting potential concerns about using these models for sensitive information.

This is particularly significant in arbitration, where confidentiality is often a default rule,⁴⁰ with arbitrators and parties most often being bound by confidentiality by legislation, rules set by arbitration institutions,⁴¹ or codes of conduct.⁴² By inputting information related to ongoing confidential proceedings into an LLM, arbitrators, lawyers, and others involved run the risk of infringing their duties of confidentiality or, even if such a duty is not breached in a strictly legal sense, making information from proceedings potentially available to those outside the process.

Given the lack of detail regarding the scope of confidentiality in most texts that prescribe this duty, it is unclear to what extent the use of LLMs is compatible with the confidentiality of arbitration. Arbitral institutions must pay increased attention to this concern to ensure that clear standards regarding the compatibility of using these models with confidentiality obligations are developed. Such policies should clearly outline standards for handling confidential information in the context of sharing information with LLMs and similar tools, thus offering clarity on the admissibility of using these tools in the arbitration context.⁴³

Arbitrators and others using LLMs in tasks relating to arbitral proceedings should err on the side of caution and take measures to protect the confidentiality of the proceedings. Users of LLMs can anonymize the input content to ensure that it cannot be directly linked to their identity and does not include information that allows the identification of the disputing parties, proceedings, or other sensitive information. Further, to avoid the long-term storage of sensitive information, users should familiarize themselves with the options offered by these models regarding data storage. Although LLMs typically default to saving conversation records for data training, users can choose not to use this option to reduce potential privacy risks. While not guaranteed to prevent confidentiality breaches, such efforts can alleviate many of the privacy concerns raised by using these models.

LLMs in arbitration: violating the arbitrator’s personal mandate?

Besides the above-discussed general issues, a more specific issue emerges from using LLMs in arbitration: the extent to which such use is compatible with the scope of an arbitrator’s mandate. The selection of arbitration as a method for solving a dispute is underpinned by the notion that arbitration is subject to parties’ autonomy in relation to the selection of who resolves the dispute and how the dispute is resolved.⁴⁴ As a corollary of this idea, arbitrators appointed to the proceedings are bound to resolve the dispute themselves; they are generally not permitted to delegate this duty to a third party.

In more specific terms, the personal mandate of an arbitrator has a hybrid nature, shaped by both the law and the contracts of the parties involved.⁴⁵ On the one hand, arbitrators possess certain powers granted by arbitration laws that they are expected to exercise to ensure the fair conduct of arbitration proceedings.⁴⁶ On the other hand, the scope of an arbitrator’s personal mandate is directly defined by the agreements and expectations of the parties who entrust the arbitrator to handle the dispute. If arbitrators go beyond the parties’ intentions and use tools to exercise the powers originally assigned to them, they conceivably break the implied contract established with the parties, harming trust in the arbitration edifice as a whole.⁴⁷

In light of these concerns, it is necessary to consider the limitations to the tasks that arbitrators can assign to LLMs. We propose that tasks may be assigned to three categories depending on their potential effect on the outcome of the case. First, we argue that the tasks that constitute the core of the decision-making process must be handled by the arbitral tribunal and may not be accomplished through an LLM—even if parties agree to such use for that particular task. Second, tasks that do not involve core material decisions over the dispute but may shape how the arbitral tribunal perceives the facts or the law applicable to the case may be delegated to LLMs, subject to disclosure and agreement by the disputing parties. Finally, a few ancillary tasks with no effect on the proceedings, most notably copyediting, may be accomplished with the help of LLMs at the arbitral tribunal’s discretion.

Core decisions must be made by the arbitral tribunal

In arbitration, a number of tasks form the core aspects of arbitral decision-making. These tasks include analysing crucial evidence, determining the disputed facts underlying the dispute, and addressing the pivotal legal issues in the proceedings. What these have in common is their direct effect on the outcome of the case. They ultimately correspond to the judicial role of arbitrators and, therefore, should be subject to complete direct control by the arbitral tribunal.

The use of LLMs and other AI tools for these tasks is unsuitable for several reasons. Deciding the outcome of a complex case demands abilities beyond quickly stringing together words that offer a plausible textual response to an inputted request. While the ‘efficiency’ of the decision-maker is certainly a consideration when selecting how an award should be produced, high-quality decision-making demands more ‘human’ qualities such as intelligence, temperament, wisdom, and an inner commitment to justice.⁴⁸

Beyond the discussion of whether using an LLM as a substitute for an arbitral tribunal’s decision-making is an appropriate step, the more salient issue is that ‘awards’ produced in this manner may not be valid. In most legal jurisdictions, it is generally understood that arbitrators must be natural persons, even though this understanding may not be explicitly specified in the law.⁴⁹ Rules on topics such as the selection of arbitrators, independence and impartiality, and disclosure of conflicts contemplate an arbitral tribunal composed of natural persons.⁵⁰ For example, the IBA Guidelines on Conflicts of Interest in International Arbitration generally only make sense in reference to a human arbitrator.⁵¹

One could even ask whether a decision made using an LLM would still be an arbitral award under the New York Convention. Considering the available technology at the time of the adoption of the New York Convention, its provisions were drafted with the implicit understanding that arbitrators could only be human, as reflected in Article IV, which requires a duly authenticated original or certified copy of an award.⁵² This requirement has been interpreted as necessitating the personal signature of the arbitrator.⁵³ While a teleological interpretation of the New York Convention may permit extending it to awards produced in this manner,⁵⁴ until case law emerges addressing these issues, parties will be at risk of having a non-enforceable award.

Finally, determining the attribution of liability is also a concern when arbitration awards are made through LLMs. Although the arbitration rules of many institutions and some national laws stipulate that arbitrators are afforded a level of immunity for their conduct during arbitration,⁵⁵ in certain circumstances, arbitrators can be liable for illicit acts.⁵⁶ Therefore, were LLMs to be comprehensively applied for the core tasks of arbitrators and situations giving rise to liability emerged, the current system of rules would appear to be largely unprepared to address this. LLMs are not currently recognized as legal entities capable of bearing liability in most countries, which complicates the attribution of liability. For example, the European Commission believes that existing national liability laws, particularly for fault, are not suitable for regulating suits for damages caused by AI-based products and services.⁵⁷

Altogether, it is clear that the arbitration framework is still built on a series of underlying values and assumptions, including, at least for the time being, that arbitrators are, by definition, human.⁵⁸ For now, at least in many jurisdictions, an arbitral tribunal may not delegate the actual decision of a case to an LLM, even with agreement from the parties.⁵⁹

Ancillary tasks may be accomplished using an LLM subject to the parties’ agreement

Beyond these core tasks that arguably may not be delegated to LLMs, many other tasks could conceivably be achieved using an LLM, such as legal research, drafting procedural documents, and reviewing parties’ claims and evidence. The question that emerges is whether the arbitral tribunal can decide whether to do so or, in other words, whether the usage of LLMs should be considered a discretionary decision to be taken by the arbitral tribunal.

We contend that this use of LLMs should be, at least under the present conditions, only available to an arbitral tribunal to the extent that parties consent to it.⁶⁰ Delegating such tasks to LLMs could undermine the expectations of the parties when they initially chose the arbitrator.⁶¹ Many parties invest significant time and resources in identifying the individual best suited to handle their case.⁶² Therefore, the parties’ trust in the arbitration decision applies solely to the individual they appointed and not to any third-party support used in carrying out tasks. Arbitral tribunals should, therefore, use a cautious approach when deciding to use an LLM in cases to ensure that the parties’ expectations and trust remain unaffected.

Arbitration is distinguished primarily by the leading principle of party autonomy.⁶³ When both parties select an arbitrator, agree to have their dispute resolved by that arbitrator, and bind themselves to the outcome, it effectively establishes a relationship of entrustment,⁶⁴ which implies that the parties delegate the authority to resolve their dispute to the arbitrator and place themselves under the constraints of the arbitration outcome. Suppose, however, that the arbitrator chooses to entrust certain tasks to an LLM to assist in their work. Here, this effectively constitutes a ‘re-entrustment’, wherein some aspects of the matters originally entrusted to the arbitrator by the parties are re-entrusted to an LLM. The legitimacy of such re-entrustment is based on the parties’ awareness and consent to the act and content of re-entrustment. In summary, both the arbitrator’s re-entrustment and the scope of authority entrusted to the arbitration tribunal secretary must be approved by the parties.

In this sense, the role of an LLM in supporting an arbitral tribunal mirrors that of arbitration secretaries, who typically handle non-core arbitration tasks within the proceedings. As noted, for example, in the LCIA Notes for Arbitrators, tasks suitable for arbitration secretaries include various administrative duties, attending hearings, meetings, and deliberations, and some substantive tasks such as summarizing the parties’ claims, reviewing the legal basis for the parties’ claims, and drafting preliminary procedural orders and awards.⁶⁵ Many of these tasks could conceivably be delegated to an LLM.

While what tasks may be entrusted to an arbitral secretary is a controversial topic in its own right,⁶⁶ the consensus seems to be that an arbitral secretary must receive explicit consent from the parties to perform tasks.⁶⁷ By requiring the arbitral tribunal to seek the parties’ consent, arbitrators can better understand which aspects of the arbitral secretary’s role concern the parties, respond accordingly, and ensure that all parties’ procedural rights are respected.⁶⁸ This approach appears to be the most appropriate for LLMs: if tasks are entrusted to a third party, authorization from the parties should be requested, even if that third party is not human.

Overall, it appears permissible to assign specific tasks to LLMs with the agreement of the parties, provided the tribunal considers that using these models increases the efficiency of the proceedings, their usage complies with the applicable procedural rules and professional duties, and the arbitral tribunal does not lose control over the core actions of deciding the case.⁶⁹ Given the lack of case law on these matters, caution should guide an arbitral tribunal in deciding issues related to these matters, with every decision considering the case’s specific circumstances and the technical capabilities of LLMs. Further, decisions should uphold party autonomy as a key aspect of how these issues are considered and ensure that any use of these technologies is made with the agreement of the parties.⁷⁰

Tasks conceivably not requiring authorization

While the above considerations should guide an arbitral tribunal in most situations involving the usage of LLMs, in our view, a carve-out should be made for the use of LLMs for text review and copyediting work. LLMs possess notable capabilities in this regard,⁷¹ and they may be able to swiftly improve readability and ensure that a text is free of grammatical errors, effectively reducing the workload of arbitrators while enhancing the ability to convey information.⁷²

Two reasons support a carve-out for this kind of use of LLMs. First, these are tasks that do not directly or indirectly impact the outcome of the proceedings and, therefore, do not raise the same issues as using LLMs to assist in the decision-making process of the arbitral tribunal. Second, LLM technology is increasingly being incorporated into traditional spelling and grammar correction software. Therefore, a view that would limit the use of LLMs in this context would, without good reason, limit the ability of arbitral tribunals to use tools that are becoming common and are used in almost all areas.

Conclusion

With the demand from both businesses and individuals for faster and cost-effective dispute resolution methods, the legal services industry is undergoing a technological transformation. Market pressure is a driving force behind this makeover, as increasing competition between service providers has caused many firms to consider how to integrate LLMs and other AI technologies within their practice. In the arbitral context, there is space to integrate these advanced technologies to enhance efficiency, reduce costs, and provide new forms of support for arbitral professionals.

However, the introduction of emerging technologies into the arbitration field brings a series of challenges.⁷³ Mindlessly incorporating new technologies into the arbitration process, in the absence of well-defined technical rules and ethical standards, may lead to difficult legal and technical questions. Eventually, these issues will likely be overcome by developing specific guidelines, such as the aforementioned SVAMC Guidelines on the Use of Artificial Intelligence in International Arbitration, and the creation of community consensus.

More broadly, however, the issue that ultimately emerges is the relationship between humans and technology in the context of dispute resolution and to what extent technology should be allowed to drive juridical decisions. As pointed out by the French philosopher of technology, Bernard Stiegler:

Today, machines are the tool bearers, and the human is no longer a technical individual; the human becomes either the machines’ servant or its assembler: the human’s relation to technical objects proves to have profoundly changed.⁷⁴

The shift in the relationship between technology and humans will eventually necessitate reimagining how the law is applied and to what extent machines can be used to play roles that were traditionally the sole realm of human beings. In the meantime, until clear rules emerge, the solution to the legal questions posed by the use of LLMs in arbitration must be found within the traditional principles that serve as the bedrock of arbitration: party autonomy, due process, and efficient dispute resolution.

The issue of arbitral tribunals using LLMs will again bring to the forefront the importance of trust in arbitration. Because it is currently difficult to verify whether arbitrators are using LLMs in accordance with accepted practices, parties ultimately depend on arbitrators following ethical behaviours to guarantee that proceedings are conducted fairly and in accordance with the parties’ expectations.⁷⁵ Arbitrators should not betray this trust and must use LLMs in a way that respects party autonomy.

Arbitrators should continue to bear primary responsibility for decision-making, and LLMs should be seen as tools providing limited support and assistance rather than substitutes for the work that arbitrators are expected to undertake. Such balance and collaboration can help ensure the fairness and professionalism of the arbitration process, meeting the expectations of all parties involved in arbitration. Despite the potential of LLMs to enhance efficiency and handle textual content, human arbitrators’ roles and unique contributions remain indispensable in the arbitration process.

Compared to machines, a human arbitrator can establish a good reputation through their education, knowledge, and years of arbitration experience.⁷⁶ Moreover, in handling cases, they can understand and consider the rules and evidence in a case and comprehend human expectations, needs, and potential emotional factors—something that LLMs cannot replace.⁷⁷ By using LLMs cautiously and referring to general arbitration principles in the absence of explicit rules, a balance between technology and human judgment can be achieved, ensuring fairness and credibility in arbitration. This represents an important step for the arbitration field in adapting to technological advancements while maintaining its core principles.

Footnotes