TL;DR: A large language model (LLM) is an AI trained on huge amounts of text to predict the next word and generate fluent language. That fluency is also its weakness: an LLM can state false things confidently, a failure called hallucination. For legal work the fix is grounding - tying every output to a retrieved, cited source. This guide explains how LLMs work, why they hallucinate, and how prompting, fine-tuning, and retrieval make them safer for law.
A large language model (LLM) is an AI system trained on vast amounts of text to predict and generate language. It is the technology behind the chat assistants that have moved into legal work over the past few years, and behind the tools that summarise documents, answer questions, and draft text. Understanding what an LLM is - and, just as important, what it is not - is the difference between using one safely and being embarrassed by it in front of a court. This explainer walks through the mechanics in plain terms, then turns to the question every lawyer should care about: how do you make a model that predicts plausible words produce answers you can actually rely on?
What is a large language model?
A large language model is, at heart, a statistical model of language. During training it reads enormous quantities of text - books, articles, websites, code - and learns the patterns of how words follow one another. It does not store facts in a database or look up answers; instead it encodes relationships between words as billions of numerical parameters. When you ask it something, it uses those patterns to generate a response one piece at a time. The word "large" refers to both the volume of training text and the number of parameters, which for modern models runs into the hundreds of billions.
The practical consequence is that an LLM is extraordinarily good at producing fluent, well-structured language on almost any topic, but it has no built-in notion of truth. It is optimised to produce text that is probable, not text that is correct. Most of the time probable and correct line up, which is why these tools feel so capable. The gap between the two is exactly where legal risk lives, and most of this guide is about closing it. For the wider family of tools built on this technology, see our overview of what legal AI is.
How do large language models actually work?
An LLM works by converting text into tokens, then predicting the next token over and over until it has produced a complete answer. That single mechanism - next-token prediction - explains almost everything about how these models behave, including their strengths and their failures. Once you understand it, the model stops being magic and starts being a tool with predictable edges.
Tokens and next-token prediction
A token is a chunk of text, often a word or part of a word. The model breaks your input into tokens and, for each step, calculates a probability distribution over all the tokens that could plausibly come next. It picks one - with some controllable randomness - appends it, and repeats. Writing a paragraph is just this loop running hundreds of times. Because each new token is chosen in light of everything before it, the output is usually coherent and on-topic. But the model is choosing the most plausible continuation, not consulting a source of truth, and that distinction matters enormously in law.
Training, parameters, and the context window
Training is the process of adjusting the model's parameters so its predictions match real text; it happens once, on data with a fixed cut-off date, and is enormously expensive. That has two implications for legal work. First, a model's built-in knowledge is frozen at its training cut-off, so it will not know about a statute passed or a judgment handed down afterwards unless you supply it. Second, anything the model appears to "know" about your matter must come from what you give it when you ask. That input lives in the context window - the amount of text the model can consider at once. Modern context windows are large but finite, which is why how you feed documents and sources to a model is itself a skill worth learning.
Why do large language models hallucinate?
A hallucination is fluent, confident output that is not grounded in any real source - an invented case, a misquoted statute, a plausible citation to a judgment that does not exist. Hallucination is not a bug a vendor will eventually patch away; it is a direct consequence of how LLMs work. Because the model generates the most probable next token rather than retrieving a verified fact, it will happily produce a citation that looks exactly like a real one - correct format, plausible party names, believable year - because that pattern is statistically likely, even when no such case exists.
For legal work this is the central danger. The most notorious example is the US matter Mata v. Avianca, in which lawyers filed a brief containing AI-generated cases that were entirely fabricated and were sanctioned for it. The lesson is not that LLMs are useless, but that an ungrounded model cannot be trusted to state facts. A model that sounds certain is not telling you it is right; it is telling you the words fit together well. This is also why it helps to know that an LLM is a generative system rather than a lookup tool - a distinction we draw out in machine learning vs generative AI for legal work.
General-purpose vs legal-tuned LLMs: what is the difference?
A general-purpose LLM is trained on broad text from across the internet; a legal-tuned LLM is one that has been further adapted - or wrapped in legal-specific systems - to perform better on legal tasks. The distinction matters because a general model has read some law along with everything else, but it has no special discipline about citing sources, distinguishing binding from persuasive authority, or staying within a jurisdiction. It will answer a question about a statute in the same confident voice it uses for a recipe.
Legal-tuned approaches try to close that gap, but "tuning" is not one thing. Some products fine-tune a model on legal text; others keep a general model and add retrieval, structure, and citation systems around it. As we will see, the approach that most reliably reduces hallucination for legal work is not changing the model's weights at all, but changing what the model is allowed to read when it answers. The table in the next section lays out the trade-offs so you can see why.
How do you specialize an LLM for legal work?
There are three main ways to make a general LLM more reliable for law: prompting, fine-tuning, and retrieval-augmented generation (RAG). They are not mutually exclusive - the strongest systems combine them - but they solve different problems and carry different costs, and only one of them attacks hallucination at its root.
Prompting
Prompting is the practice of writing careful instructions that steer the model toward better output: giving it a role, context, constraints, and a required format. It is the cheapest and fastest lever, requires no engineering, and often produces a large improvement in quality. Its limit is that it cannot give the model knowledge it does not have or stop it from inventing sources; it shapes behaviour, not truth. Our guide to prompt engineering for lawyers covers the techniques in depth.
Fine-tuning
Fine-tuning continues training a base model on a curated set of legal examples so it adopts the style, format, and reasoning patterns of legal work. It can make a model better at, say, drafting in a particular form or following a house style. But fine-tuning is expensive, needs high-quality data, goes stale as the law changes, and - crucially - does not stop hallucination. A fine-tuned model still generates from patterns; it has simply learned legal-sounding patterns, which can make its fabrications more convincing rather than less.
Retrieval-augmented generation (RAG)
Retrieval-augmented generation (RAG) takes a different route: instead of relying on what the model memorised, it retrieves relevant documents at the moment of the question and instructs the model to answer only from those sources. The model becomes a reader and synthesiser of supplied text rather than an oracle drawing on frozen memory. This is the approach that most directly attacks hallucination, because the answer is tied to retrieved passages you can open and check. The technique was introduced in a widely cited 2020 paper by Lewis and colleagues - see the original RAG paper - and it underpins every serious citation-first legal tool. Our explainer on RAG for legal AI goes further.
The table below compares the three approaches on the dimensions a legal buyer actually cares about.
| Approach | What it changes | Cost and effort | Effect on hallucination | Best for |
|---|---|---|---|---|
| Prompting | The instructions, not the model | Low; no engineering | Reduces, does not remove | Steering tone, format, and scope |
| Fine-tuning | The model's weights | High; needs quality data | Little to none; can mask it | Consistent style and structured output |
| Retrieval (RAG) | What the model reads at answer time | Moderate; needs a good source corpus | Strongly reduces when sources are cited | Fact-grounded, citable legal answers |
What are the limits of LLMs for legal work?
The core limit of an LLM is that it predicts plausible language, not verified truth, and no amount of fluency changes that. Several practical limits follow. A model's knowledge is frozen at its training cut-off, so it can miss recent statutes and judgments. It can misread an ambiguous question and answer a subtly different one. It can summarise a long document accurately yet drop the single qualification that decides the matter. And it has no stake in the outcome - it will not warn you that your best authority was overruled last year unless its sources show it.
None of this makes LLMs unsafe to use; it makes them unsafe to use blindly. The right mental model is a brilliant, fast, widely-read assistant with no judgment and no accountability. You get the speed; you keep the responsibility. That is why the design of the system around the model - what it reads, what it cites, what it refuses to guess - matters far more for legal work than the raw cleverness of the model itself.
Where does Judicio fit?
Judicio's answer to the hallucination problem is to ground the LLM in retrieved, cited sources rather than trusting its memory. Every finding, answer, and date is tied to an exact page and a quoted passage, and the citation labels are deterministic - generated by the system, never written by the model - so a source always carries the same reference. Web sources are archived as permanent PDFs at the moment of retrieval, so a citation cannot quietly rot, and you can export an evidence pack of everything the model relied on.
That grounding runs through the whole workspace. One upload into the File Library feeds Legal Research, Document Review, the Review Matrix, the Timeline Builder, Translation, and Drafting, and Legal Research spans 33 dedicated jurisdiction databases plus more than 100 jurisdictions through curated legal web search. For the broader research on model reliability, the Stanford Institute for Human-Centered AI publishes useful work on where these systems succeed and fail. The point is not that Judicio uses a smarter model; it is that the model is kept on a short, cited leash.
You can test it on your own matters with a 7-day free trial - 500 credits and no credit card required - and explore the full feature set or contact us for a walkthrough. Professional plans are $200 per month for 5,000 credits.
Judicio's outputs are research and drafting aids, not legal advice; you remain responsible for verifying every result and exercising professional judgment.