The landscape of legal education is witnessing a groundbreaking shift as the latest generation of generative artificial intelligence demonstrates its prowess in law school examinations. A recent study has revealed that OpenAI’s newest model, dubbed o3, has achieved remarkable grades on law school finals, outperforming its predecessors and inching closer to human high achievers.
Acing Law School Finals
In an unprecedented achievement, OpenAI’s o3 model earned grades ranging from A+ to B across eight different law school finals. These exams were administered by faculty members at the University of Maryland Francis King Carey School of Law. The study, published on SSRN, highlighted the model’s ability to secure top marks in crucial subjects such as Constitutional Law, Professional Responsibility, and Property, where it achieved A+ grades.
Comparative Performance
This level of performance marks a significant advancement from earlier versions of ChatGPT, OpenAI’s previous model. While earlier iterations managed to score B’s, C’s, and even a D in law school finals during 2022 and 2023, the o3 model’s results demonstrate a marked improvement. Previous studies indicated that although ChatGPT could enhance the speed of legal writing, the quality remained mediocre. However, o3’s capabilities suggest that generative AI is rapidly catching up to high-performing human law students.
Reasoning Model Advantage
Unlike its predecessors, o3 is classified as a reasoning model. This means it doesn’t just generate immediate responses but evaluates and revises potential answers before finalizing them. This methodical approach allows for a more nuanced understanding and response to complex legal questions. The model’s ability to craft well-reasoned answers was evident in its excellent performance across various subjects.
Grades Breakdown
– A+ in Constitutional Law, Professional Responsibility, and Property
– A in Income Taxation
– A- in Criminal Procedure
– B+ in Secured Transactions and Torts
– B in Administrative Law
The study’s authors, a team of seven law professors, graded the AI’s responses on the same curve used for human students. The o3 model excelled in both multiple-choice questions and essay formats, showcasing its versatility and depth of understanding.
Limitations and Challenges
Despite its impressive performance, the o3 model faced certain limitations. Its lower grade in Administrative Law was attributed to a lack of awareness about a critical 2024 U.S. Supreme Court ruling in Loper Bright Enterprises v. Raimondo. The ruling, which overturned the Chevron doctrine, was central to the subject but occurred after o3’s knowledge cutoff date.
The study also uncovered an unexpected outcome when the model was given access to a professor’s notes. Instead of enhancing its performance, the additional information seemed to distract the AI, resulting in a poorer performance on that particular final.
Future Experimentation
The study’s authors are contemplating further research to explore the potential threat of AI-assisted cheating in academic settings. They propose instructing the AI to introduce occasional spelling and grammar errors to make its work less distinguishable from that of human students.
OpenAI’s Response
As of the report’s release, OpenAI has not commented on the findings. The study’s revelations, however, suggest a rapidly advancing AI landscape, where generative models are increasingly capable of performing at levels comparable to skilled human counterparts.
Stay Updated
For those keen on following the latest developments in AI and legal education, subscribe to updates at aitechtrend.com.
Note: This article is inspired by content from https://www.reuters.com/legal/legalindustry/artificial-intelligence-is-now-an-law-student-study-finds-2025-06-05/ . It has been rephrased for originality. Images are credited to the original source.