Building Contract Review Tooling Lawyers Will Actually Use

Contract review is an appealing target for automation. The documents are structured, the questions are repetitive, and the people doing the work are expensive. Extraction quality on standard clauses is now good enough that a demo is easy to build.

Then it goes to a legal team and does not get used. The reason is rarely accuracy. It is that the tool asks a reviewer to trust it, and offers nothing to check.

Verification cost decides adoption

A reviewer is accountable for the advice they give. If a tool says "this contract has an unusual indemnity cap" and the reviewer has to find the clause themselves to confirm it, the tool has saved nothing — they were going to read the document anyway.

Every finding needs to land next to its evidence: the exact span, in the actual document, one click away. Not a page number, not a paraphrase. The span, highlighted, in context.

This sounds like a UI detail and it is the whole product. It is the difference between a tool that saves a reviewer time and one that adds a step to their process. Teams routinely spend months on extraction quality and weeks on the reading experience, and the ratio should be closer to the reverse.

Absence is the dangerous answer

Extraction systems are far better at finding what is present than at telling you something is missing.

That asymmetry matters here more than in most domains, because in contracts the absent clause is frequently the risk. No limitation of liability. No governing law. No termination right. A tool that lists what it found, with nothing about what it looked for and did not find, gives its most confident answer in exactly the case where confidence is least warranted.

Design the output around a checklist rather than a summary. For each clause type you handle: found here, or explicitly not found, or not confident. Three states, always shown. The "not confident" bucket is the one that makes the tool safe to use, and it is the one product pressure will push you to hide.

Where the documents will break your assumptions

Real contract estates are messier than the sample set:

• Scanned paper. Executed agreements from before everything was digital, often photocopied more than once. OCR quality sets your ceiling and no model recovers what the scan lost.
• Amendments. The operative terms are spread across an original and four amendments, each modifying clauses by reference. Reading only the latest document gives a confidently wrong answer.
• Definitions that move meaning. A defined term in clause 1 changes what clause 40 means. Retrieval that pulls only the relevant-looking clause loses the definition that governs it.
• Negotiated redlines where struck-through text survives in the file, and the tool cannot tell what was agreed from what was rejected.

Amendments are the one we would flag hardest. Handling them is a document-relationship problem, not an extraction problem, and it is usually discovered after the extraction work is considered finished.

The failure mode: automating the judgement

The tempting next step is to have the system decide whether a clause is acceptable — flag it green or red against a playbook.

This is where tools lose their users. Acceptability depends on the counterparty, the deal size, the commercial relationship and what was traded for it elsewhere in the document. A tool that renders a verdict will be wrong in ways that are obvious to the reviewer, and one visibly wrong verdict costs more trust than ten correct extractions earn.

Extract and surface. Let the lawyer judge. The system's job is to make sure nothing was missed and everything is quick to check, which is a genuinely valuable job and a defensible one.

Confidentiality is an architecture constraint

Contracts are among the most sensitive documents an organisation holds, and frequently carry confidentiality obligations to the counterparty. Sending them to a third-party API may breach an agreement in the document you are processing, which is an uncomfortable place to end up.

Settle this before the architecture, not after: what the contractual terms permit, whether the provider trains on submissions, and whether a self-hosted model is required. It changes the cost model substantially and it is not a decision to discover during procurement.

When we would advise against building it

If the volume is low — tens of contracts a year — the tool will cost more than the review it replaces, and it will decay because nobody uses it enough to maintain it.

If the contracts are highly bespoke, extraction quality drops precisely where the risk concentrates, and you get a tool that works on the routine documents nobody worried about.

The strong case is high volume and reasonable standardisation: supplier agreements, NDAs, procurement paper, a portfolio being reviewed for a transaction. There, the checklist-and-evidence approach reliably takes hours out of a first pass.

What we would build first

One clause family, on one document type, with evidence spans and a three-state answer. Measure it against a set a lawyer has reviewed by hand — including documents where the clause is absent, which is the case teams forget to include and the one that reveals whether the system is safe.

Expand only when a reviewer says they would rather use it than not. That sentence, from someone accountable for the output, is the only adoption metric that has ever predicted anything.

More on how we approach this in AI engineering and custom software development.