Blogs

Critical Thinking in the Age of AI Answers

By Brendan Miller posted an hour ago

  

Please enjoy this blog authored by Brendan Miller, Former AmLaw 100 Partner, Turned Legal Innovation Leader. 

NOTE: This blog is an extension of a more detailed treatment of this topic published in ILTA's Spring 2026 Peer to Peer publication: Teaching Judgment in the Age of AI – Building Legal Professionals Who Rely on AI Responsibly. What follows distills the core framework and updates it with recent evidence about where the field stands.
 
The legal industry has spent the last two years solving for AI access, deployment, and adoption. The harder challenge now is ensuring that increased AI capability is matched by increased human judgment.

I.   The Problem with Easy Answers
AI has become extraordinarily good at giving answers. With confidence. With structure. With positivity. With apparent authority.
 
What it cannot do from the outset is tell you whether you are asking the right question.  Or whether the answer it just gave you is actually correct or whether it addresses the underlying question you really have (or should have) in mind.
 
AI tools have added genuine, measurable benefits to innumerable legal and business workflows. They have lowered the cost of generating a plausible-sounding answer to nearly zero. In the legal profession, that is both remarkable and a source of compounding professional risk.

A 2026 LexisNexis survey of nearly 900 UK lawyers found that 72% identified deep legal reasoning and argumentation as the biggest skills gap among junior lawyers, and only 2% believe AI actually strengthens legal learning.

The same report found that AI may be producing lawyers who can generate answers quickly but struggle to test them properly (i.e. the “Verification Gap”).

Source: LexisNexis UK, The Mentorship Gap (2026).

The profession has made significant progress on access and availability.  It has not solved judgment. Two-plus years of scaled AI adoption have done a remarkable job of teaching legal professionals how to prompt. The next challenge, and the one that will define professional differentiation, is teaching them how to think critically about WHEN to prompt (i.e. when to use AI) and about WHAT comes back when they do prompt. This challenge extends beyond lawyers to knowledge management, innovation, legal operations, pricing, risk, and business services professionals who increasingly rely on AI-assisted workflows.
 
In legal, the value of AI for law firms, legal departments, and their clients will ultimately be determined by the quality of human judgment layered over it, not by the sophistication of the tools themselves.


II.   What “Judgment” Actually Means in the Age of AI
There is considerable discussion in legal tech circles about AI judgment. The term is used broadly, and reasonable and thoughtful practitioners and thought leaders have expressed an array of opinions on the topic. For this context, a working definition may be helpful.
 
Professional judgment in the context of AI is not simply the ability to recognize a hallucination. It is a cluster of interrelated competencies, including:

• Foundational “discernment” or the ability to thoughtfully and critically evaluate what AI produces, how it produces it, and how it behaves; see, e.g., the 4Ds of AI Fluency Framework (Anthropic Academy).
• Working knowledge of the relevant area(s) of law
• Knowing when AI is the right tool for the task, and when it is not
• Knowing what questions to ask before treating an output as usable
• Knowing which outputs require independent verification, and then actually verifying them
• Knowing how to communicate AI-assisted work to clients and courts with transparency and accuracy
• Knowing when professional accountability overrides the convenience of a plausible AI-generated result

These competencies are not theoretical; they have a regulatory foundation. ABA Formal Opinion 512 (July 2024) made clear that the duty of competence under Model Rule 1.1 now explicitly encompasses a reasonable understanding of AI capabilities and limitations. State and local bar associations have reached consistent conclusions. Courts have sanctioned attorneys not for using AI, but for failing to exercise judgment over what AI produced.
 
Mata v. Avianca, Inc. (2023) is the oft-cited example of what happens when AI is employed in legal without proper verification and judgment, but it will not be the last. The issue in that case was professional judgment abdication, not technology failure. That distinction matters enormously for how organizations design training.
 
While the primary focus of this article is on developing and exercising individual judgment, it is also absolutely true that judgment is not solely an individual competency. Organizations must develop and prioritize systemic judgment “muscle.” Data hygiene and security, governance frameworks, approved workflows, validation standards, and clearly defined compliance and accountability structures help create environments where responsible AI use becomes repeatable rather than dependent solely on individual discretion.

III.   Three Futures and Why Judgment is the Differentiator in All of Them
Thomson Reuters' recent Future of Professionals Report 2026 (global survey of more than 1,800 professionals across law, tax, audit, and compliance) offers a useful strategic framework for where legal organizations are headed. The report identifies three core paths for AI adoption in professional services:

AI to Elevate: Human expertise remains at the center. AI handles groundwork so professionals can focus on judgment, relationships, and strategic thinking that command the greatest value.
AI to Scale: Capacity is the priority. AI enables teams to handle significantly greater volume without proportional headcount growth, with professionals overseeing quality and consistency.
AI to Reimagine: Structural reinvention. AI enables entirely new service propositions or business models. Not optimization of existing ones, but rebuilding from the ground up.


The table below summarizes the three paths and their implications for how judgment develops in each:

Elevate

Scale

Reimagine

Core purpose

Human expertise at center; AI handles groundwork so professionals focus on judgment, relationships, strategic thinking

Capacity first; AI enables greater volume, improved response and consistency without proportional headcount growth

Structural reinvention; AI enables entirely new service models and business propositions

Role of judgment

Professionals focus on the calls AI cannot make; expertise and accountability command premium value

Oversight and quality control; consistency at scale; humans validate AI-assisted output

Judgment applied to AI-surfaced outputs; hybrid domain/ technology roles

Junior development

Substantive work from day one; AI compresses learning curve, but foundational judgment requires deliberate design

Breadth and QA responsibility earlier; routine tasks automated; consistency is the training framework

Hybrid roles combining domain expertise and technology capability; traditional development paths less defined

Key trade-off

May serve a narrower, premium market; junior exposure to foundational work requires intentional redesign

Throughput advantage erodes as competitors catch up; relationships may become more transactional

Highest transition risk; new models require sustained investment before payback

Source: Thomson Reuters Institute, Future of Professionals Report 2026 (2026).

There are certainly differences in approach in each of these paths, but what is striking is what the three paths share. Human judgment is not eliminated in any of them. It remains a critical aspect, but it is repositioned based on the overarching strategy for how AI is leveraged. Whether an organization is using AI to elevate expertise, scale capacity, or reimagine its model, the differentiator is the professional who can critically evaluate what AI produces and exercise sound judgment about what to do with it.

IV.   The Judgment Gap, and Why Existing Training is Not Yet Closing It

The uncomfortable reality is that most AI training programs have not been designed to build judgment. They have been designed to build familiarity and confidence in the tools.
 
The legal industry has continued to embrace and expand its use of AI, as reflected in ILTA's 2025 Technology Survey and other industry reports. But adoption momentum and judgment maturity are not the same thing, and the profession has been far more deliberate about building the former than the latter.

The legal industry has continued to embrace and expand its use of AI, as reflected in ILTA's 2025 Technology Survey and other industry reports. But adoption momentum and judgment maturity are not the same thing, and the profession has been far more deliberate about building the former than the latter.
 
The early phase of legal AI adoption prioritized access, security policies, and basic prompting techniques.  All prudent and foundational. What the field now requires is the next layer: scenario-based training that builds the instinct and skills to critically evaluate AI outputs, ethics integrated as an organizing framework rather than a compliance module, and verification protocols treated as professional disciplines rather than formalities.  See Teaching Judgment in the Age of AI – Building Legal Professionals Who Rely on AI Responsibly for further discussion.

V.   A Practical Framework: Scaling Judgment to Risk
Not all uses of AI carry the same professional stakes. This is true regardless of which of the three primary AI adoptions paths (referenced above) an organization has emphasized. Effective judgment training reflects that reality. Borrowing from eDiscovery and technology-assisted review defensibility frameworks, organizations can categorize AI use cases by risk tier and calibrate human oversight accordingly:

Risk Tier

Example Use Cases

Judgment Standard

Oversight Level

Low

Internal summaries; ideation; research surveys

Basic review; spot-check for accuracy and relevance

Light

Medium

Client-facing drafts; contract redlines; legal memoranda

Full professional review; citation verification required

Structured

High

Court filings; regulatory submissions; opinion letters

Senior attorney sign-off; explicit AI-use disclosure consideration

Mandatory audit

Within each tier, judgment training should embed a consistent set of verification disciplines. The types of checklist questions that should become reflexive for any AI-assisted legal workflow:

• Have all citations been independently verified against primary sources?
• Are the factual assumptions underlying the AI output accurate and supported?
• Does this output affect client rights, legal positions, or regulatory standing?
• Would disclosure of AI use be required or prudent in this context?
• Can I explain and defend this output -- its reasoning, its sources, its limitations -- if challenged?

These are not bureaucratic additions to legal workflow. They are the professional responsibility questions that have always governed legal work, now adapted for the AI-assisted practice environment. Treating them as a skill to be developed, rather than a formality to be waived, is the practical difference between a tool-trained attorney and a judgment-equipped one.

VI.   The Competitive Advantage is Not the Tool
The early AI race in legal was about access and use cases. Every firm of scale now has access to materially similar platforms: Harvey, Legora, CoCounsel, Copilot, etc.

What remains difficult to replicate is the quality of the human judgment layered over those platforms. Courts have sanctioned AI misuse. Bar associations have clarified overriding duties. In-house clients are already evaluating outside counsel's AI governance, and some are revisiting billing arrangements because of generative AI’s demonstrated impact on time-intensive tasks.
 
The next phase of AI maturity will be measured less by adoption rates and more by the effectiveness of the human oversight systems organizations build around these tools. The organizations that will lead are not those that stop at teaching legal professionals to prompt better. They are the ones that teach their people to decide better. That skill is not generated by an algorithm. It is cultivated—deliberately—through the kind of training frameworks the field is only beginning to build.

Brendan W. Miller, J.D. is a legal innovator: a curious, seasoned litigator and corporate attorney, technologist, strategist, and change agent. To Brendan, legal innovation is about continually being relevant for clients, by making the business and practice of law easier, better, and more valuable.


#BlogPost
#100Level
#GenerativeAI
#ArtificialIntelligence
#Just-in-Time
0 comments
217 views

Permalink