How to Evaluate a Mortgage AI Claim
Mortgage technology buyers are currently being told a great deal about artificial intelligence, much of it by people with an incentive for the claims to be believed. This post is written by one of those people, which is a reason to apply its own tests to it.
What follows is a set of questions that separate claims which can be verified from claims which cannot, and an honest account of where the technology genuinely stands.
What works well now
Reading documents — much better than five years ago, not solved. Vision-language models read mortgage documents, including poor scans, unfamiliar layouts and formats they were not specifically built for, at a standard that was not previously available. The improvement is real and it is available to everyone, which is why it is not a differentiator. It is also not finished: handwriting, stamps, redlined figures, multi-column tables and low-quality faxes still produce errors, and the errors are not always low-confidence. Treat reading as a good and improving capability with a measurable error rate, not as a solved problem.
Classification and segmentation. Identifying what a document is and splitting a package into its parts works well.
Locating values. Finding a field on a page and reporting where it was found works well on clean, familiar documents. It degrades on the ones that matter most — handwriting, stamps, rescanned addenda, multi-column tables — and it degrades without always lowering its confidence.
Summarisation and question answering. Asking questions of a file and getting useful answers works, with the caveat below about reproducibility.
What is not solved by reading alone
Knowing whether a value is right. A model can read a figure with high confidence and the figure can still be wrong for the file — because it contradicts another document, because it is the wrong period, or because the qualifying calculation should have used a different method. Reading confidence and correctness are different properties.
Reproducibility. Ask a model the same question twice and the reasoning may differ. For most applications this is unimportant. For a decision defended in three years against the rule as it stood at the time, it matters — and the reason is precise. The model can be pinned to a version; an identifier, snapshot and settings are all recordable. What cannot be relied on is re-deriving the same conclusion from the same document. Version tracking and reproducibility are different properties, and only the second one answers an examiner.
Knowing what is missing. A model shown a file can describe what is in it. Determining that a document which should be present is absent requires knowing what the file ought to contain, which is a function of programme, property, occupancy and jurisdiction — a rules question, not a reading question.
Six questions worth asking any vendor
1. What model reads the documents, and did you build it?
Most mortgage AI is built on foundation models from a small number of providers. There is nothing wrong with that — it is the sensible engineering choice, since document reading improves industry-wide and is not where a QC platform should spend its differentiation.
What matters is whether the vendor says so. A claim of a proprietary or specially trained model is checkable: ask which model, trained on what, evaluated how. A vendor who cannot answer specifically is describing something they do not have, and the claim tells you how they will handle the next uncomfortable question.
2. Where does the disposition come from — the model or a rule?
This is the most important question on the list. If a model decides outcomes, the review inherits the model's properties: variation between runs, and reasoning that cannot be inspected the way a written rule can.
Be precise about versioning here, because the loose version of this argument is wrong. A model can be pinned to a version — a model identifier, a snapshot date, a temperature setting are all recordable. What a model identifier does not give you is exact reproducibility: the same model version can return different reasoning on the same input. Recording which model ran is not the same as being able to re-derive what it concluded.
If a stored rule decides, the outcome is reproducible from retained inputs, and the model's role is bounded to reading.
3. What happens to a finding when the underlying rule changes?
Requirements move. Ask whether a finding is tied to the version of the rule that produced it, or whether historical findings silently re-anchor to the current rule set. If it is the latter, every past review is described using a standard nobody applied at the time.
4. What is recorded when a rule does not run?
In a summary, "checked and passed" and "never checked" look identical and mean opposite things. This matters most in exactly the segments where programmes are numerous and bespoke — non-QM, [1] government, HFA — because a system with no applicable rule usually reports nothing, and nothing reads as clean.
5. What is your measured accuracy, and on what document set?
Accuracy figures are the most commonly published and least examinable claims in this category. The number is meaningless without the document set, the field set, the definition of a correct extraction, and who performed the measurement.
A vendor quoting a percentage without those should be asked for them. A vendor who has not measured should say so — and that is a more informative answer than a confident number, because it tells you what else they are prepared to assert without evidence.
Three tests are worth asking for by name, because an aggregate accuracy figure hides all of them:
- False negatives. Given files with known defects, how many did the system not flag? This is
the number that matters most and is reported least.
- False positives. How much reviewer time does the system consume on findings that turn out
to be correct values? A system that flags everything is accurate and useless.
- Confident extraction errors. How often is a value wrong and returned with high
confidence? Confidence scores that do not correlate with correctness are worse than no confidence scores, because they misdirect review effort.
6. What can our own technical team verify without taking your word for it?
The best answer is a working session on your files, where you inspect values against source pages and rules against outcomes. Anything that can only be demonstrated on the vendor's chosen documents is a demonstration of the documents.
The shape of an honest architecture
Stated plainly, because it is also what we build and you should weigh it accordingly.
A general-purpose vision-language model reads the documents. Every value it produces is retained with the document, page and position it came from, and a confidence score. A deterministic rule engine — versioned, published, containing no model — decides the outcome. Each finding records the rules that ran, the rules that were skipped, and the numbered rule version applied.
The property this arrangement gives you is narrower than it first appears, and stating it precisely matters more than stating it well.
What is fixed: the rule, its logic and its version. The same inputs, run against the same rule version and the same configuration, produce the same finding.
What is not fixed: whether the inputs were right. A deterministic rule applied to a misread value produces a wrong answer, deterministically. If extraction reads monthly income as $12,000 when the paystub says $7,000, the debt-to-income calculation is entirely reproducible and entirely wrong. The rule did not change. The conclusion did. Where programme or occupancy data is misread, the wrong rule set can be brought into scope in the first place — so extraction error can affect rule applicability, not only a value inside a rule.
Reproducibility, stated honestly, is a property of the whole record: the retained inputs, the rule version and the configuration. Re-running extraction over the same document does not necessarily reproduce the inputs, so a system that keeps only the document and the rule version cannot reproduce its own finding.
So the question to ask a vendor is not whether their rules are deterministic. It is: how do you detect an extraction error, what does a reviewer see when they disagree with a value, and who holds the final disposition? A vendor claiming to have eliminated hallucination risk is making an unfalsifiable claim.
None of this requires a proprietary model, and claiming one where none exists is a straightforwardly checkable falsehood — the kind that surfaces during technical diligence, at the worst possible moment.
The test underneath all six questions
Every question above is a version of the same one: can this be checked by someone who does not trust you?
Claims that survive that test tend to be architectural — what decides, what is recorded, what version applied. Claims that fail it tend to be quantitative and unsourced, or about capabilities that cannot be inspected from outside.
That is a workable filter, and it applies to this post as much as to anything else you are currently being shown.
Sources
- Regulation Z, 12 C.F.R. § 1026.43. The ability-to-repay requirement applies to consumer credit secured by a dwelling; § 1026.43(a) excludes extensions of credit primarily for a business, commercial or agricultural purpose.