So do we raise the bar or find questions AI can’t answer?
GPT-4 Passes the Bar Exam: What That Means for Artificial Intelligence Tools in the Legal Industry
Codex–The Stanford Center for Legal Informatics and the legal technology company Casetext recently announced what they called “a watershed moment.” Research collaborators had deployed GPT-4, the latest generation Large Language Model (LLM), to take—and pass—the Uniform Bar Exam (UBE). GPT-4 didn’t just squeak by. It passed the multiple-choice portion of the exam and both components of the written portion, exceeding not only all prior LLM’s scores, but also the average score of real-life bar exam takers, scoring in the 90th percentile.
Casetext’s Chief Innovation Officer and co-founder Pablo Arredondo, JD ’05, who is a Codex fellow, collaborated with Codex-affiliated faculty Daniel Katz and Michael Bommarito to study GPT-4’s performance on the UBE. In earlier work, Katz and Bommarito found that a LLM released in late 2022 was unable to pass the multiple-choice portion of the UBE. Their recently published paper, “GPT-4 Passes the Bar Exam” quickly caught the national attention. Even The Late Show with Steven Colbert had a bit of comedic fun with the notion of robo-lawyers running late-night TV ads looking for slip-and-fall clients.
… The rate of progress in this area is remarkable. Every day I see or hear about a new version or application. One of the most exciting areas is something called Agentic AI, where the LLMs (large language models) are set up so that they can “themselves” strategize about how to carry out a task, and then execute on that strategy, evaluating things along the way. For example, you could ask an Agent to arrange transportation for a conference and, without any specific prompting or engineering, it would handle getting a flight (checking multiple airlines if need be) and renting a car. You can imagine applying this to substantive legal tasks (i.e., first I will gather supporting testimony from a deposition, then look through the discovery responses to find further support, etc).
Another area of growth is “mutli-modal,” where you go beyond text and fold in things like vision. This should enable things like an AI that can comprehend/describe patent figures or compare written testimony with video evidence.
(Related)
https://www.bespacific.com/why-universities-should-return-to-oral-exams-in-the-ai-and-chatgpt-era/
Why universities should return to oral exams in the AI and ChatGPT era
The Conversation: “Imagine the following scenario. You are a student and enter a room or Zoom meeting. A panel of examiners who have read your essay or viewed your performance, are waiting inside. You answer a series of questions as they probe your knowledge and skills. You leave. The examiners then consider the preliminary pre-oral exam grade and if an adjustment up or down is required. You are called back to receive your final grade. This type of oral assessment – or viva voce as it was known in Latin – is a tried and tested form of educational assessment. No need to sit in an exam hall, no fear of plagiarism accusations or concerns with students submitting essays generated by an artificial intelligence (AI) chatbot. Integrity is 100% assured, in a fair, reliable and authentic manner that can also be easily used to assess multiple individual or group assignments. As services like ChatGPT continue to grow in terms of both its capabilities and usage – including in education and academia – is it high time for universities to revert to the time-tested oral exam?
The more explaining the better?
Large, creative AI models will transform lives and labour markets
They bring enormous promise and peril. In the first of three special articles we explain how they work
… Chatgpt embodies more knowledge than any human has ever known. It can converse cogently about mineral extraction in Papua New Guinea, or about tsmc, a Taiwanese semiconductor firm that finds itself in the geopolitical crosshairs. gpt-4, the artificial neural network which powers Chatgpt, has aced exams that serve as gateways for people to enter careers in law and medicine in America. It can generate songs, poems and essays. Other “generative ai” models can churn out digital photos, drawings and animations.
Running alongside this excitement is deep concern, inside the tech industry and beyond, that generative ai models are being developed too quickly.
A review or something new?
https://www.bespacific.com/modern-monetary-theory-an-explanation/
Modern monetary theory: an explanation
Modern monetary theory: an explanation. Professor Richard Murphy. April 2023. Funding the Future. Formerly Tx Research UK. “Modern monetary theory (hereafter, MMT) is an explanation of the way in which money works in an economy. It also explains the consequent impact that the best use of money, using this understanding, might have on behaviour in that economy. The core suggestion made by MMT is that a government is constrained by the real productive capacity of its economy and not by the availability of money, which it can always create. Secondary insights are that money is created by government spending and is destroyed by taxation. […] What I offer here are my interpretation of what I think to be core MMT understanding. I suspect there is much common ground in that. There will be much less common ground on my interpretation of what the understanding means and on the supposed (and very largely, in my opinion, unnecessary) theoretical justification for it. So be it: that is what political economy is about.”
No comments:
Post a Comment