Hallucinate with a straight face...
Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models. Matthew Dahl, Varun Magesh, Mirac Suzgun, Daniel E. Ho, 2 Jan 2024. “Large language models (LLMs) have the potential to transform the practice of law, but this potential is threatened by the presence of legal hallucinations — responses from these models that are not consistent with legal facts. We investigate the extent of these hallucinations using an original suite of legal queries, comparing LLMs’ responses to structured legal metadata and examining their consistency. Our work makes four key contributions: (1) We develop a typology of legal hallucinations, providing a conceptual framework for future research in this area. (2) We find that legal hallucinations are alarmingly prevalent, occurring between 69% of the time with ChatGPT 3.5 and 88% with Llama 2, when these models are asked specific, verifiable questions about random federal court cases. (3) We illustrate that LLMs often fail to correct a user’s incorrect legal assumptions in a contra-factual question setup. (4) We provide evidence that LLMs cannot always predict, or do not always know, when they are producing legal hallucinations. Taken together, these findings caution against the rapid and unsupervised integration of popular LLMs into legal tasks. Even experienced lawyers must remain wary of legal hallucinations, and the risks are highest for those who stand to benefit from LLMs the most — pro se litigants or those without access to traditional legal resources.”
See also The Economist – Generative AI could radically alter the practice of law. Even if it doesn’t replace lawyers en masse
To AI or not to AI is no longer a question.
Is AI Friend or Foe: Legal Implications of Rapid Artificial Intelligence Adoption
Conklin, Michael, Is AI Friend or Foe: Legal Implications of Rapid Artificial Intelligence Adoption (April 25, 2023). Michael Conklin, Is AI Friend or Foe: Legal Implications of Rapid Artificial Intelligence Adoption, 26 ATLANTIC L.J. ___ (forthcoming 2023). Available at SSRN: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4429539
“The term “artificial intelligence” (hereinafter “AI”) was coined in the 1950s and has been a staple in science fiction movies and literature. However, it appears that the 2020s is the decade when the real-world realities of AI are finally manifesting. In early 2023, The New York Times referenced the beginning of an “AI arms race.” ChatGPT was introduced in November 2022 and has already passed a bar exam, passed a medical licensing exam, scored in the top 10th percentile on the combined SAT, passed an entire semester’s worth of classes at a tier 1 law school, passed a University of Pennsylvania Wharton School of Business MBA Exam, and written hundreds of books sold on Amazon. Autonomous drones have already been used in military conflicts to kill soldiers. Madison Square Garden is enforcing an expansive attorney ban through the use of facial recognition software. And in 2023, Ford applied for a patent for self-repossessing automobiles. This is a brief review of Mark Deem and Peter Warren’s new book, AI on Trial. It offers diverse views regarding the rapidly evolving legal standards of AI. It is easy to read due to the conversation tone and lack of technical jargon. While the book offers an expansive coverage of AI, this review will focus on the topics of irrational fear of AI, biased AI, AI for legal determinations, immense benefits from AI compared to human labor, and the nuanced balancing act of regulating AI. The book would serve as a valuable resource for business law professors looking for topical, high-stakes, stimulating topics to act as a catalyst to ignite class discussion regarding the real-life application of law.”
No comments:
Post a Comment