AI Lawyers Beat Professors in Legal Reasoning Tests, Study Finds

A Stanford-led study reports that AI lawyers using large language models (LLMs) can outperform law professors on contract law reasoning. Researchers had 16 professors from 14 U.S. law schools create 40 contract-law questions (doctrines, case law, hypotheticals, and policy issues). In 2,918 blinded comparisons, professors chose AI-generated answers over professor-written answers about 75% of the time. Google’s Gemini 2.5 Pro won 75.92% of matchups, while NotebookLM won 74.75%, with both models flagged as harmful less often than human instructors (Gemini 2.5 Pro: 3.41% harmfulness; NotebookLM: 3.64%, vs. 12.06% for professors). Additional checks suggest the performance is not purely from writing style, and researchers conclude LLMs can align with common professional criteria. The study also cautions that it does not measure whether AI responses satisfy each instructor’s personal teaching preferences, meaning results may reflect broadly acceptable quality rather than tailored pedagogy. The findings add to ongoing debate as courts and law schools adopt AI tools in legal workflows.