Clause Forensics #2: The same query, three different answers

Take a concrete case. The same query runs against the same vendor contract data: what SLA credits are we entitled to from this vendor in Q1 2026? Now run it three ways.

Raw documents, no structuring layer. The AI has direct read access to the contract files and nothing else. It ingests the MSA, the SLA addendum, and whatever amendments are in the folder, and it generates an answer. The trouble is it has no way to know which version is in force, which clause supersedes which, or which numbers are sensitive. So it does what these systems do: it reads everything it can see and surfaces everything it finds. It might return a dollar figure that is wrong because it read a superseded threshold. It might return the right figure alongside the pricing schedule and indemnification terms the requester was never supposed to see, because to the model it was all just text in the same file. And it cannot show its work, because there is no addressable clause to cite back to. The answer is a confident paragraph with no evidence chain and no access boundary. You cannot file a claim on it and you cannot trust what else leaked into it.

Structured, queried as a procurement user. Now the same data has been structured first, with metadata on top. The system returns the full picture: the vendor owes $84,200 in SLA credits for the quarter, based on three documented breaches of a 99.5% uptime commitment in the executed MSA and SLA addendum. It returns the claim deadline, the per-entitlement breakdown, the trigger for each one, the confidence score, and whether a human validated it. Everything a procurement lead needs to actually file the claim, with each line traced back to a specific clause.

Structured, queried as a generic AI service account. Same structured data, same query, different caller. This time the response is nothing of substance. No dollar figure, no breaches, no citations. Just a note that the service account is metadata-only by default and would need to be bound to an app context with explicit read permissions before it could see document contents at all.

Three callers against the same underlying contract, three very different outcomes. The first is dangerous. The other two are correct for who is asking, because the system understood who was asking and what they were entitled to see down to the level of the individual entitlement and the documents behind it.

Picture the same scenario under conventional access control. The contract lives in a folder. The procurement team has read access. The AI agent has a service account that was, at some point, granted access to that folder so it could "help with vendor analysis."

Now the AI agent can read the entire contract. Every clause. The pricing it was never meant to surface to the requester. The indemnification terms. The supersession history across amendments. The moment you grant document-level access, you have granted access to everything in the document, and you are trusting that whatever sits on top of it will behave.

That is the gap. File-level permissions are too coarse for the questions people and machines now ask of contract data. The unit of sensitivity in a contract is not the file. It is the clause, sometimes the sub-clause. A pricing schedule, a most-favored-nation provision, a confidential carve-out, and a routine SLA threshold can all live in the same PDF, and they do not all carry the same access profile.

Here is the part that matters. The system above could only give two different answers because the contract data had been structured first, with metadata layered on top that the raw document never carried.

Raw, a contract is a wall of text. It has no idea that section 4.3 is a pricing term or that the SLA addendum supersedes the threshold in the original MSA. It cannot tell a permission system "this clause is restricted, that one is fine to summarize." There is nothing for an access policy to attach to.

Once you structure it, that changes. Each entitlement, each clause, each obligation becomes an addressable object with its own attributes: what type it is, which document and which version it came from, what its sensitivity is, who is allowed to see it, and what it supersedes. Now access control has something to reason against. The policy can say "procurement sees the dollar figure and the evidence; a generic service account sees that an entitlement exists but not its contents unless explicitly bound." That decision happens at the clause level, not the file level, because the metadata exists at the clause level.

Without that structuring layer, you are stuck choosing between two bad options: lock the whole document and make it useless, or open the whole document and lose control of what comes out. The structuring layer is what lets you stop choosing.

Access control is just the most legible example. The same structured layer that makes clause-level permissions possible is what makes a dozen other things possible.

It is what lets a system cite its sources instead of asserting answers, because every claim traces back to a specific clause in a specific document version. It is what lets you reconcile an invoice against the contract that priced it, because both have been mapped to the same obligations. It is what lets you track a term across five amendments and know which version is currently in force. It is what lets an AI answer a question about your contracts and be checkable, because the answer is grounded in addressable objects rather than a retrieval system's best guess.

None of that is possible against raw documents. All of it becomes possible once the data has been structured and enriched on the way in.

If you are putting AI in front of your contract data, or any sensitive document corpus, the access question is not "which files can this agent open." It is "what is this caller entitled to see, at the level of the individual clause, and does the system have enough structure to make that judgment correctly."

Most systems do not, because the structure was never built. The document was treated as the atomic unit, and the metadata that would let you reason about its parts was never created. Adding that layer is unglamorous work. It happens on the way in, before anyone queries anything, and it is invisible when it works.

But it is the difference between an AI that can answer "what are we owed" with a defensible number and an evidence chain, and one that either refuses to answer or answers with everything it should have kept to itself.

Analysis powered by allcaps.ai . Clause Forensics is a weekly series on procurement.news examining one contract clause, how it breaks, and what it costs.

This analysis is for informational purposes only and must be validated against the executed agreement, amendments, invoices, and operational records.