Get your firm's free AI report card
BLOG POST · SOLO & SMALL FIRM PRACTICE

Find the Rule: Legal Research Lawyers Can't Hand to AI

An open, empty machine cabinet labeled Ask Me with a chessboard on top, beside a magnifying glass over a statute with the rule highlighted, with the line: Nobody's inside the box.
TL;DR
  • For lawyers, AI research tools answer plain-English questions, but leading ones hallucinated 17% to 33% of the time in a Stanford test.
  • Start by naming the rule you need in your jurisdiction, then check the AI's answer against the source, not the summary.
  • Search the way judges write, with statute numbers and citations, and confirm which version of a statute applies.
SPU membership · community and strategy sessions Build better research habits with SPU membership

For lawyers, legal research now comes with a chat box. Type a question in plain English and a research tool writes back an answer with citations attached. Solo Practice University taught a course on this exact temptation more than fifteen years ago, when the big research services first pushed lawyers toward asking questions instead of building searches. Its warning holds up better than most tech predictions do: the machine can return an answer, but it can't find the rule for you.

The Mechanical Turk, again

The course is Real World Legal Research, taught for SPU between 2009 and 2014 by Richard Maseles. In 2010, West launched WestlawNext with a heavy emphasis on natural-language searching, and the instructor reached for an 18th-century hoax to describe it. The original Mechanical Turk was a chess-playing "machine" with a human hidden inside, moving the levers.

"WestlawNext is, simply, a Mechanical Turk with a somewhat more sophisticated mechanism."

The instructor was careful to say the product wasn't a fraud. The point was what it invited lawyers to believe:

"But there isn't anyone back there to answer your question."

Today's tools are far more capable than anything the course saw. Thomson Reuters launched CoCounsel Legal in August 2025 with a Deep Research feature that builds a research plan and runs it through Westlaw's own tools, including KeyCite (Thomson Reuters, LawNext). In February 2026, LexisNexis replaced Lexis+ AI with Lexis+ with Protégé (LawNext, LexisNexis).

Still, nobody is inside the box. In a preregistered Stanford study published in 2025, the AI research tools from LexisNexis and Thomson Reuters hallucinated between 17% and 33% of the time, and the authors concluded that vendors' "hallucination-free" claims were overstated (Stanford Law School, LawNext). Those tests used the 2024 versions, and the products have changed since. The lesson hasn't. ABA Formal Opinion 512 says GAI tools "lack the ability to understand the meaning of the text they generate or evaluate its context." We covered what happens when that output reaches a court in our post on hallucinated citations and malpractice risk.

"Good enough" can lose the case

The course's sharpest point was about expectations. Most of us learned to search on Google, where the engine always returns something, and the something is usually fine. The instructor noted that in most Google searches, "there's not a lot riding on the search results." Legal research is different:

"Here, a 'good enough' answer can lose the case. Worse, it can deny justice for your client."

The course framed the risk with two terms from information science. Precision is how much of what you got back is relevant. Recall is how much of what's relevant you actually got back. Bad precision wastes your time. Bad recall is the one that hurts, and the instructor called it "the lawyer's nightmare": the controlling case you never saw, or saw for the first time in your opponent's brief.

An AI research answer looks like perfect precision. Every case it cites seems on point. What it can't show you is recall, the authority it didn't retrieve. That's the gap the lawyer still has to close. Our older post asking whether the internet changed how lawyers reason worried about the same over-reliance. The tools got better. The worry still applies.

Find the rule first

The course's method starts before any search. In one lesson, the instructor took apart a textbook exercise: a hotel guest is hurt during an attempted robbery in a lobby restroom, and the question is whether the hotel is liable. The textbook had students brainstorm search terms for the people, places, claims and relief, and then string synonyms together into one long query. The instructor noticed what was missing from the list. Duty.

Once you settle on negligence, the question becomes what duty the hotel owed this guest in that part of the building, under that state's law. That's a rule, and a rule has a source. The instructor didn't soften the conclusion:

"Find the rule, dammit. That's the way law is practiced in the real world."

This is the habit that makes AI research safe to use. Before you ask the tool anything, write down the rule you need and the jurisdiction it comes from. Then ask for that rule and where it lives, and go read the source. A plain-English question invites a plain-English answer. A specific rule gives you something you can check.

Specific matters. An SPU column on a four-alarm limitations mistake told the story of a careful lawyer who found the other state's general limitations period and missed the shorter one for claims against a town. The general rule was real. It just wasn't the rule.

From question to verified answer

SkipAsk in plain EnglishAccept whatever comes back
Step 1Name the ruleThe duty, element or test, in your jurisdiction
Step 2Find its sourceStatute, regulation or controlling case
Step 3Read the sourceNot the summary
"Find the rule" is the course's method. Applying it to AI research is this post's.

Search the way judges write

The course's other core technique is just as useful for checking AI output. Lawyers are taught to search by concept, piling up synonyms in case the judge used a different word. The instructor argued that judges are much more predictable than that. They may or may not use your concept, but they will cite the statute and the case.

"We professionals search for terms that judges actually use, not concepts that they may or may not use."

So search the statute number, down to the subsection, or the reporter volume of the key case, inside your own jurisdiction's case law. The instructor's favorite example was using a section number as the whole query. That's also a fast way to test an AI answer. If the tool says a statute means something, search the section number and read how your courts have applied it. If it cites a case, pull the case and the cases that cite it.

When you're stuck, the course had a story for that too. As a law student, the instructor complained that a research assignment was like finding a needle in a haystack. The supervising lawyer's answer: "Son, you've been looking in the wrong haystacks." The question was about a statute, and the student had been browsing cases. A chat box makes it easy to keep asking the same haystack the same question.

Which version of the law?

One warning from the course gets more important with AI, not less:

"Statutes and regs have effective dates. Let me repeat that—statutes and regs have effective dates."

The instructor's point was that you need the statute as it read when your client's claim arose, not only the version in force today. An AI answer usually gives you the current text unless you ask otherwise, and it won't always flag that an amendment changed the outcome. Free tools can help on the federal side: the eCFR lets you view federal regulations as they stood on past dates back to January 2017 and compare versions (eCFR, Minneapolis Public Library). For state law, check your research service's version history or the legislature's site.

Before you rely on an AI research answer

  1. Name the rule and the jurisdictionBefore you type the question.
  2. Read the source, not the summaryStatute, regulation or case.
  3. Search the section numberSee how your courts applied it.
  4. Ask what it might have missedPrecision isn't recall.
  5. Confirm the version in effectWhen the claim arose, not today.
A starting list built from the course's method. Your practice area may need more.

The lawyer is still the researcher

None of this means giving up the tools. The course itself was never against technology. It was against handing over the thinking. Back in 2010 the instructor wrote that no algorithm adds up to a product that replaces "the skill and judgment of an intelligent lawyer," one who can find "the right law, the best law."

John O'Connor, founder of Associate and SPU's Professor of AI Fluency, puts the current version simply:

"The fact that AI drafts and researches doesn't mean lawyers go away."

ABA Formal Opinion 512 says lawyers using these tools need "a reasonable understanding of the capabilities and limitations" of them, and it warns that "uncritical reliance on content created by a GAI tool can result in inaccurate legal advice to clients or misleading representations to courts and third parties." What that requires of you depends on where you practice, so check your own jurisdiction's rules.

The same goes for anything else AI writes in your name, including the practice-area pages on your firm's website. If you'd like another set of eyes on those, SPU offers a free law firm website review. And if your research budget is the bigger worry, our companion post covers legal research on a budget, from free bar-provided tools to the one subscription not to cut. For more on AI in practice, see our technology coverage.

This post draws on a slice of the course. It also covered reading statutes and court opinions, regulations and administrative law, legislative history, secondary sources, buying research on a budget, citators, portable research tools, asking other lawyers for help, legal ethics research and legal writing.

Keep learning with SPU

SPU membership gets you the community, monthly strategy sessions and the course library, so you can use AI research tools without handing over the judgment your clients pay for.

THE WEEKLY SPU BRIEF

Get one practical lesson for your firm every week.

Free, from practicing lawyers. Every issue includes a lesson from the archive. Unsubscribe anytime.

ABOUT THE AUTHOR
The SPU Team

The SPU Team writes and edits Solo Practice University's articles. Susan Cartier Liebel founded SPU in 2009 to teach lawyers the practical side of building a practice, the part law school leaves out, with courses taught by practicing lawyers. Today SPU is a community for solo and small-firm lawyers, with an AI Fluency program for firms putting AI to work. Nothing here is legal advice: check your own jurisdiction's rules.

About Solo Practice University →