- A model is good at telling you what a contract says. The money in a supplier agreement is usually in what it does not say, and absence is the thing a summary cannot show you.
- Purpose built legal tools still get things wrong at a rate worth planning around. A Stanford study found two of them wrong more than 17 percent of the time and a third above 34 percent.
- The study separates two failures. One is describing the law incorrectly. The other is being right while citing a source that does not support the point, which is harder to catch.
- Three letter shipping terms are the most expensive thing a model will explain confidently and generically, because the rule set has eleven of them and they move cost and risk between the parties.
- Prompt for absences rather than summaries, and the tool starts working on the part you cannot do yourself.
- Uploading a contract is a disclosure. Check the retention and training terms of whatever you paste it into before, not after.
The supplier sends a PDF. Twelve pages, defined terms in capitals, a schedule at the back, and a request to sign by Friday. A lawyer would cost more than the first order is worth, so you read it yourself, understand most of it, and sign, because the alternative is not doing the deal.
Putting a model on that document is an obvious improvement over reading it tired at eleven at night. It is also where people form the wrong expectation, because the thing it does best is the thing you least needed and the thing you needed most is the thing it does worst.
What is a model genuinely good at in a contract?
Translation and retrieval. Ask what a clause means in ordinary language and you will get a useful answer. Ask where the agreement deals with late delivery and it will find it faster than you can scroll. Ask it to list every deadline, every notice period and every number, and you get a table that would have taken you an hour.
That is real value and it is the right place to start. Most small business contract trouble is not sophisticated. It is a renewal notice period nobody diarised, a price adjustment clause nobody read, a minimum volume commitment that sounded like an aspiration. Extraction catches all three.
What follows from this is a simple division of labour. Use the tool to build the map, then do the judgement yourself, because the judgement needs things that are not in the document.
Why can it not tell you what is missing?
Because a missing clause leaves no text to read. A model working from the document sees what is there, and its instinct is to summarise and reassure. The risk in a supplier agreement is frequently a silence: no cap on price increases, no remedy if quality drops, no right to terminate for convenience, no limit on the supplier's liability being matched by one on yours.
This is fixable, and the fix is a prompt rather than a tool. Instead of asking what this contract says, ask which protections a buyer in this kind of deal would normally expect that this agreement does not contain. That question points the model at its general knowledge of contract structure rather than at the text, and it produces a checklist you can then verify yourself.
It will over produce. Some of what comes back will be irrelevant to a two thousand pound order of candle jars. That is the correct failure direction for this task, because you can dismiss an irrelevant item in five seconds and you cannot dismiss a risk you never saw.
How often are these tools actually wrong?
Often enough that the number should change how you use them. Stanford's RegLab and HAI researchers ran a preregistered evaluation of commercial legal research tools using a manually built dataset of more than 200 open ended legal queries.
The tools tested were purpose built legal products rather than general chatbots, and they did reduce errors compared with a general purpose model. They still produced incorrect information more than 17 percent of the time in two cases, and more than 34 percent in the third. These are systems sold to law firms with retrieval over authoritative databases behind them.
The paper behind that write up also draws a distinction worth carrying into your own use. One kind of failure is straightforward: the answer describes the law incorrectly. The other is described as misgrounded, meaning the answer states the law correctly but cites a source that does not actually support the claim. The write up notes the second may be more pernicious, because a citation can exist and be checkable and still not say what it was cited for.
Translate that to a contract. A model telling you clause 8.3 caps your liability is a claim with a citation. The citation exists. Whether clause 8.3 says that is a separate question, and the only way to answer it is to open clause 8.3. That habit, opening every clause the model names, removes most of the exposure, and it takes seconds.
The reliability question is not whether the model is right on average. It is whether you will notice when it is wrong. Anything it tells you that matches what you already suspected will pass unchallenged, which makes confirmation the most dangerous output shape.
Which clauses deserve a human read regardless?
Five, and they are the same five in almost every supplier agreement a small business signs.
| Clause | What it decides | Common trap | What to ask the model |
|---|---|---|---|
| Term and renewal | How long you are committed and how you leave | Automatic renewal with a notice window that closed before you looked | Every date and notice period, as a list with sources |
| Price adjustment | Whether the price you agreed is the price you pay | Increases tied to an index or to the supplier's discretion | What limits the supplier's ability to raise prices |
| Delivery terms | Who bears cost and risk while goods move | A three letter code whose meaning nobody checked | Which party pays freight, insurance and duty at each stage |
| Quality and remedy | What happens when the goods are wrong | A right to reject with an inspection window of days | The exact remedy and the exact deadline to claim it |
| Liability and indemnity | Who pays when something goes badly wrong | An uncapped indemnity from you against a capped one from them | Compare the two caps and state which is larger |
The last row is worth a moment. Asymmetry is easy for a model to spot once you ask for it directly and almost impossible to notice in a summary, because both clauses read as normal boilerplate in isolation. The question compare the liability cap you owe with the one they owe is one sentence and it has saved people a great deal of money.
Why are shipping terms the expensive one?
Because three letters carry an entire allocation of cost and risk, and a model asked what they mean will explain the standard definition rather than the one your contract actually uses.
The International Chamber of Commerce publishes the Incoterms rules, a set of eleven three letter trade terms used in contracts for the sale and purchase of goods, first published in 1936 and recognised by UNCITRAL as the global standard for interpreting the most common terms in foreign trade. Their stated purpose is to avoid costly misunderstandings by clarifying the tasks, costs and risks involved in delivering goods from seller to buyer. The current edition, Incoterms 2020, entered into force on 1 January 2020.
Two things follow for anyone importing. The version matters, because a contract referring to an older edition means an older allocation, and a model will usually answer from the most recent one it knows. And the parties can modify a term in the contract itself, at which point the definition the model recites is no longer what you agreed. A clause saying delivery on a named basis, amended as set out in Schedule 2, is a trap for a summariser and an ordinary drafting pattern for a supplier.
The practical instruction is to ask for the edition named in the contract, ask whether any schedule modifies the term, and then work out the cost split yourself from those two answers. The customs and duty side of the same question is covered in what AI can and cannot do with customs classification.
What happens to the document you upload?
It leaves your building, which is a point worth deciding before the deadline pressure rather than during it. A supplier agreement contains prices, volumes, terms you negotiated and often a confidentiality clause covering the document itself. Pasting it into a consumer chatbot is a disclosure of all of that.
The distinction that matters is not which brand you use but which tier and which terms apply to it. A business tier with contractual commitments about training and retention is a different product from the free consumer version of the same assistant, even where the model underneath is identical. Check whether inputs are used for training by default, how long they are retained, and whether you can turn both off.
There is also the question you owe the other side. If the contract contains a confidentiality obligation, processing it through a third party service may technically breach it, and a sensible obligation carve out for professional advisers will not obviously cover a software vendor. Nobody is going to sue you over this. It is worth knowing anyway, and it is the sort of question our page on how MaShop handles and isolates data exists to answer for our own side of the relationship.
What about the terms you never negotiated?
Different job, and arguably the better use of the tool. A marketplace agreement, a payment provider's terms or a fulfilment platform's conditions run to tens of thousands of words, cannot be edited, and were accepted by clicking. Nobody reads them, which means nobody knows what they agreed to until something goes wrong.
Here the absence prompt is useless, because you have no leverage to add anything. The useful question is narrower: what can the other party do to me unilaterally, and with how much notice. Ask for every provision allowing the platform to suspend an account, hold funds, change fees or alter the terms, with the notice period attached to each.
Four answers are worth having before you depend on a platform. How long can they hold your money and on what grounds. What triggers suspension and is there an appeal. How much notice do you get on a fee change. And where does a dispute go, which for many services means arbitration in a jurisdiction you have never visited.
None of this is negotiable, so the output is not a redline. It is a risk register: the things you now know can happen, which you can plan around by not keeping all your revenue on one platform. The account suspension version of that planning is set out in what a seller is actually owed when an account is suspended.
Should it draft clauses as well as read them?
For a first pass at a simple document, with the same caution that applies everywhere else. A model will produce a competent looking supply agreement, and a competent looking agreement is a specific hazard, because the flaws are invisible to the person who cannot evaluate the drafting.
Two uses are safe enough. Turning your own decisions into contract language, where you already know what you want and need it phrased, keeps the substance yours. And drafting a counter proposal to a clause you object to gives you something concrete to send, which is often what stalls a negotiation for a week.
What to avoid is generating a whole agreement for a relationship that matters and signing it because it reads well. The document is not the deal. The deal is what the two of you understood, and a clause neither party can explain is where that understanding goes to die.
When should you stop and call someone?
Three signals, and they are about size rather than about complexity.
Stop when the downside exceeds what the business can absorb. An uncapped indemnity, a personal guarantee, or a minimum commitment larger than a quarter's revenue are all in this category regardless of how straightforward the drafting looks. The cost of an hour of advice is trivially small against any of them.
Stop when the agreement governs something you cannot replace. Exclusive distribution, your only supplier for a core product, or anything touching your brand or customer list. The clause that matters there is usually what happens at the end, and end of term provisions are where inexperienced readers consistently miss the sharp edges.
Stop when the other side's answers to your questions get vaguer rather than clearer. That is not a legal signal, it is a commercial one, and no document reader will ever tell you about it.
For everything else, which is most contracts a small shop signs, the model plus your own reading is a genuine improvement on your own reading alone. The lawyer's version of this same calculation, with the tasks separated by how much verification each needs, is in where AI actually helps in legal work and where it does not.
A workflow that takes twenty minutes
Extract first. Ask for every date, deadline, notice period, number and named party, with the clause reference beside each one. Open three of them at random and check they say what the list claims. If any is wrong, treat the whole list as unverified and slow down.
Then ask for absences. Which protections would a buyer normally have here that this does not give me. Take the output as a list of questions rather than as findings.
Then ask the asymmetry questions directly. What is the largest amount I could owe under this agreement. What is the largest amount they could owe me. How do I end this and what does it cost.
Then read the five clauses in the table yourself, properly, with the model's map beside you. That is the part that cannot be delegated, and by then it is a twenty minute job rather than a two hour one. The tool did not replace the reading. It made the reading short enough that you will actually do it.
One last habit. Keep the questions you asked alongside the signed contract. In eighteen months, when something goes wrong, the record of what you checked is worth as much as the answers were, and it is the fastest way back into a document you have forgotten. The same instinct applies to any tool you bring into the business, which is the subject of the questions worth asking a vendor before you buy.