Research · Position paper ·

The Specification Drift Problem in Consumer Reporting

What agentic screening can be made to enforce under the FCRA, and what it cannot.

Executive Summary

Consumer reporting agencies are deploying AI agents into the workflows the Fair Credit Reporting Act governs most closely: assembling reports, resolving a record to a person, adjudicating against a client’s criteria, sequencing adverse action, handling disputes. In each, the agency faces one question, and it decides whether the deployment survives a regulator or a court: for this file, can we show the agent did what the FCRA required, at the moment it acted? Current architectures answer it the way they answer it everywhere — the model read the policy and usually honored it. For a consumer reporting agency that is not an answer, because the FCRA’s enforcement model is per-transaction and adversarial: a consumer with a private right of action (§1681n, §1681o), a CFPB examiner asking about a specific file, a plaintiff’s attorney assembling a class. None of them accepts an aggregate conformance rate as evidence about the file in front of them.

The failure that produces this is easy to misread, so we are explicit. The agent is not adversarial and is not malfunctioning in any way its developers would recognize. It is faithfully executing its understood task, producing actions that are locally reasonable at every step, narrated in a log that reads coherently. The violation is silent and individually defensible — the agent doing something right in a way that is wrong relative to the FCRA — which is why a better model narrows it without closing it, and why it surfaces not as an error but as a pattern, months later, in a dispute or an examination.

The resolution is not “conform deterministically everywhere,” and a paper that implied so would commit the error it diagnoses. The FCRA contains two kinds of obligation. Some are bright lines — the adverse-action sequence (§1681b(b)(3), §1681m), the reporting windows of §1681c, permissible purpose at the moment of furnishing (§1681b) — state machines a runtime can gate before the action executes, where a probabilistic component is simply the wrong instrument and a workflow engine would often do. Others are reasonableness judgments — the §1681e(b) duty of “maximum possible accuracy,” identity matching, the EEOC individualized assessment — that no deterministic verdict makes correct, and where the most a mechanism outside the model can do is require that the judgment was made by the right party through the required procedure, and prove it. The error agentic deployments make is conflating the two, in both directions: applying probabilistic judgment where the law drew a line, and letting a consistent automated output stand in for a judgment the law assigned to a person.

One consequence is specific to consumer reporting. Accuracy in screening is bidirectional: over-reporting harms the consumer and is an FCRA matter, while under-reporting — the missed record — harms the client and is a negligence matter, with a different victim and a different body of law. The procedures that reduce one increase the other, so where an agency sets that dial is a policy decision, and a probabilistic agent does not hold a position on it; the agency’s accuracy posture becomes a property of the model rather than a decision it made and can defend.

This paper characterizes the gap between reading the FCRA and enforcing it, and it is careful about how much of the gap enforcement closes. The boundary has two parts and is developed in §7: enforcement proves the agent followed the specification, not that the data is complete or the specification adequate; and for the judgment-laden obligations it governs the procedure and the routing, not the correctness of the judgment.

Scope and Limitations

This is a problem statement. It characterizes a structural deficiency in current agentic architectures as applied to employment and tenant screening, and it does not report empirical results: the behavioral claims here are arguments from mechanism, and the question of how often a given system fails belongs to measurement, not to this paper. It distinguishes throughout between FCRA obligations that are amenable to deterministic gating and those that are reasonableness judgments, because that distinction is the substance of the argument rather than a caveat to it. It does not address creative or open-ended work, where “correct” is not decidable against a specification.

Table of Contents

  1. Background: AI enters consumer reporting
  2. The structural problem: read versus enforced — and where a governor reaches
  3. The two kinds of FCRA obligation
  4. Case studies
  5. Why current approaches fall short
  6. The economic and litigation dimension
  7. What enforcement can and cannot do
  8. References

1. Background: AI Enters Consumer Reporting

1.1 The deployment wave

The screening workflow invites automation: an input arrives, it is evaluated against a defined body of rules, and an action is produced. Agencies are placing AI agents at each step — intake and assembly, identity resolution, adjudication, adverse-action sequencing, dispute reinvestigation. The rules already exist; the agent’s task is to apply them; the question that determines whether the deployment is sustainable is whether the agency can show, after the fact, that the agent applied them as written.

1.2 The governance and litigation context

The FCRA is unusual among consumer-protection statutes in that its primary enforcement engine is private. A consumer may sue for negligent (§1681o) or willful (§1681n) noncompliance, and willful noncompliance carries statutory damages without proof of actual harm, which is what makes the FCRA a class-action statute. The Consumer Financial Protection Bureau supervises larger agencies and has paid sustained attention to accuracy procedures, dispute handling, and automated decisioning. Employment screening carries a Title VII and EEOC overlay; tenant screening carries a Fair Housing Act and HUD overlay. None of this is new. What is new is an actor that can apply, or misapply, all of it at machine speed and volume.

1.3 Why this is new even though the rules are old

The agentic case differs from human error not because it enables new failures but because it changes their distribution and persistence. An agent applies a single interpretation consistently across every file it touches until its context resets, so a misreading made once is not a scattered error but a systematic one, and it persists. Its reasoning leaves no record other than the narration it chooses to write, which has the same reliability properties as the action it describes. The per-file error rate may fall; the defensibility of any specific file falls further, because the frictions that produced defensibility — slow deliberation, procedural constraint, a reconstructible trail — are what the agent removes.

2. The Structural Problem: Read Versus Enforced — and Where a Governor Reaches

Across current agentic architectures a single pattern holds: the specification is a document the model reads, not a constraint the system enforces. Whether the FCRA reaches the agent as a prompt, a policy in context, or a structured rule set, the conformance mechanism is the model’s probabilistic judgment about what the document means for the file at hand.

This is the difference between a speed-limit sign and a speed governor. A sign is read and may or may not be obeyed; a governor mechanically prevents the threshold from being exceeded. The two produce identical outcomes most of the time, and they diverge at the edge — the ambiguous match, the borderline disqualifier, the dispute that arrives on day twenty-nine. The edge is where the litigation lives.

The governor metaphor carries an honest limit, and the rest of this paper turns on it. A governor only helps where there is a threshold to enforce. Where the obligation is a reasonableness judgment rather than a line, there is nothing for a governor to gate; the most a mechanism outside the model can do is insist that the judgment is made by the party the law assigns it to, through the procedure the law requires, and record that it was. So the question is not whether to enforce the FCRA but which of its obligations are lines and which are judgments — because the assurance available is different in kind for each, and treating them alike is how both overclaiming and underclaiming happen.

3. The Two Kinds of FCRA Obligation

3.1 Bright-line obligations: where a governor reaches

A subset of the FCRA’s requirements are discrete, ordered, and decidable from structured facts. They are state machines, and a runtime can gate them.

The adverse-action sequence is the clearest. Before adverse action based in whole or in part on a report, the user must provide the consumer a copy of the report and the summary of rights (§1681b(b)(3)); a reasonable period must pass; then the adverse-action notice issues with its required content (§1681m). Whether the pre-adverse-action obligation has been discharged is decidable, and the final action can be made to wait on it. Obsolescence is similar: §1681c defines reporting windows (seven years for most non-conviction adverse items and civil judgments, ten for bankruptcy), and sealed or expunged records are not reportable; whether an item falls inside its window is a date computation, and state overlays such as California’s seven-year conviction limit are additional date rules. Permissible purpose (§1681b) is a gate that can be bound to each request and evaluated at the moment of furnishing rather than assumed from an onboarding certification. The content a notice must carry and the thirty-day clock of §1681i are likewise checkable.

These are the obligations where per-file enforcement and proof are genuinely available, and where the honest claim is strong: the action that would violate the line does not execute, and the file carries evidence of which clause permitted what ran. They are also, notably, the obligations that least require a model. The instructive failure here is not that the agent reasons poorly; it is that a probabilistic component was placed where a deterministic one was required. A throughput-oriented system is structurally prone to collapsing the adverse-action sequence, because when a disqualification looks obvious the locally reasonable action is to proceed to the decision and the interval reads as a formality. The collapse produces no error and a coherent log. The remedy is not a better-reasoned agent; it is a gate.

3.2 Judgment-laden obligations: where a governor cannot reach

A second subset are not lines. The §1681e(b) duty to follow “reasonable procedures to assure maximum possible accuracy” is a standard whose application is contested case by case. Identity resolution — deciding that a record belongs to this applicant and not to someone who shares a name and an approximate date of birth — is a probabilistic estimate. The individualized assessment the EEOC’s 2012 guidance contemplates, weighing the nature of an offense, the time elapsed, and the relationship to the role, is a judgment about a person.

No deterministic verdict makes these correct, and a paper that claimed otherwise would be selling the thing it warns against. What a mechanism outside the model can do is narrower and still valuable: it can require that the judgment is made by the party the law assigns it to and through the procedure the law requires, and prove that it was. It can insist that an ambiguous match is escalated to a human rather than attached, that a would-disqualify record triggers and records the individualized assessment rather than a straight matrix application, that the verification step ran before a database hit became a reported one. This is governance of the procedure and the routing around a judgment, not a substitute for the judgment. The claim it supports is “the right party decided, through the required steps, and here is the evidence” — not “the decision was correct.”

This class is where the bidirectional accuracy problem lives, and it is the dimension most particular to consumer reporting. FCRA accuracy litigation concerns over-reporting: a record that is wrong, misattributed, stale, or unverified, reported under §1681e(b) and contested under §1681i, with the consumer as victim. But a screening operation also fails the other way — the missed record, the report that clears when a material record existed — and that failure has a different victim and a different law. It is governed by the client’s contract and negligence claims against the agency and by the employer’s own negligent-hiring exposure, which the employer seeks to push back through indemnity; the consumer has no FCRA claim, because the FCRA protects the subject of the report, not its buyer. The two failures are in tension: conservative matching, honoring suppression, escalating ambiguity all reduce false positives and increase false negatives, while aggressive completeness does the reverse. Where an agency sets that dial is a policy decision with consequences in two different courtrooms. The hazard of a probabilistic agent here is not that it sets the dial wrong but that it does not hold a position at all; it resolves each file in whatever way is locally reasonable, and the agency’s accuracy posture becomes emergent rather than decided. Enforcement cannot pick the dial — that is the agency’s call — but it can require that the chosen procedure is the one that ran, on every file, and prove it.

3.3 Drift, properly so called: the compounding failures

The two preceding sections describe a static gap. The word “drift” names something further: failures that accumulate, and three mechanisms in consumer reporting genuinely do.

Pipeline drift. A file passes through several agents — intake, assembly, adjudication, adverse-action — each consuming the previous one’s output, each interpreting it through its own model, each introducing a small deviation that the next inherits. The deviations are directional rather than random and they compound. A note attached at intake (“identifier present in narrative”) is read downstream as a satisfied condition, and a step the policy required is treated as complete. No component-level review finds it, because no component did anything its own review would flag; the connection back to the FCRA has been mediated through a chain of interpretations, none verified against the statute.

Temporal drift. Because an agent applies one interpretation consistently until its context resets, an interpretation established early conditions every subsequent file. A matching tolerance set loosely in week one, a suppression rule read narrowly, an adjudication shortcut that went unchallenged — each becomes the settled behavior of the deployment, applied across a growing book before anyone examines it. This is the accumulation the static framing misses: not one wrong file but a posture, hardening.

Environmental drift. The specification is fixed while the law moves. Fair-chance timing rules are added by new jurisdictions, §1681c interpretations evolve, a data source changes its schema and the agreed matching logic silently degrades. The agent continues to conform to a specification that no longer matches the obligation, and conformance to a stale specification is a violation that looks, from inside the system, like compliance.

3.4 Specification validity: the prior question

Beneath conformance sits a prior question: does the specification the agent follows capture what the law and the agency intend? An agent can conform perfectly to a specification that fails to require a source, drops a jurisdiction, or simplifies a contingent rule, and still produce a report wrong in the world. This is the missed-record axis seen from the inside, and it marks the outer boundary of what enforcement claims. Enforcement proves the agent followed the specification; whether the specification is thorough enough is a judgment that belongs to the compliance officer. The contribution enforcement makes here is not to make that judgment but to relocate it — from an unbounded stream of decisions no one can review to a small, bounded artifact the compliance officer can read, validate, and own.

4. Case Studies

Composite scenarios constructed from patterns common to screening operations, not reports of incidents at named agencies. Each is tagged to its place in §3.

4.1 The collapsed adverse-action sequence (bright-line)

An employment-screening agent identifies a disqualifying record and proceeds to generate the adverse-action notice, without first sending the pre-adverse-action notice and observing the waiting period; the decision read as already determined and the interval as a formality. Every notice is well-formed, the log is coherent, and a §1681b(b)(3) failure — the most ordinary FCRA failure there is — is later assembled across the book into a class. This is the textbook case for a gate: the obligation is a line, and the action should not have been able to execute. The human-era precedent is documented: in Goode v. LexisNexis Risk & Information Analytics Group, Inc., No. 2:11-cv-02950 (E.D. Pa.), a screener was alleged to adjudicate applicants as noncompetitive — an adverse action — before sending the §1681b(b)(3) pre-adverse-action notice, and the claim was litigated as a class. The composite above is that same collapse, reproduced by an agent at machine speed and volume.

4.2 The mixed file that cleared review (judgment-laden: matching)

A matching agent attaches a felony record on a name and an approximate date-of-birth match; the applicant is a different person. The attachment was a locally reasonable estimate and the report reads cleanly, so it passes sampling QA. The §1681e(b) exposure existed at the moment of attachment, in the gap between a plausible match and a verified one. No gate makes the match correct — that is a judgment — but the procedure could have been enforced: an ambiguous match routed to a person rather than attached, and the routing recorded. The human-era precedent is Williams v. First Advantage LNS Screening Solutions, Inc., 947 F.3d 735 (11th Cir. 2020), where a background report attached another person’s criminal records to a Winn-Dixie applicant and a jury found a willful §1681e(b) violation. The composite above is that mixed file produced by a matcher at volume.

4.3 The conservative miss (judgment-laden: the bidirectional dial)

A second agent, tuned away from over-reporting, suppresses an ambiguous hit rather than verify it; a material record never reaches the report; the applicant is hired and later harms a customer of the client. There is no FCRA claim, but the client sues for negligent performance and its own negligent-hiring exposure flows back through indemnity. The disposition that protected the agency on the §1681e(b) axis exposed it on the negligence axis. Neither file was the agent doing something wrong; both were the agent holding an accuracy position the agency never decided.

5. Why Current Approaches Fall Short

The approaches in use are not worthless, and several are hybrids that gate part of the problem correctly; what they lack is a principled assignment of obligations to the right kind of assurance. Pre-deployment evaluation covers the files the team thought to write, while the production population contains shapes no test anticipated, and a same-model test generator is least likely to exercise the requirements the agent is most likely to misapply. Output guardrails operate on the form of a result, not on whether the decision it represents was permitted. Quality-assurance sampling examines the files that were pulled, and automation bias erodes the rigor of reviewers processing a stream of overwhelmingly correct decisions. Model alignment makes an agent generally more likely to follow instructions; it does not produce the per-file guarantee a §1681i dispute or a CFPB examination asks for, and it does nothing to move a judgment the law assigns to a person back to that person. Each is valuable and each leaves the same residue: behavior on the unmeasured remainder, which is most of the file population, is whatever the model happened to produce.

6. The Economic and Litigation Dimension

The conformance gap becomes consequential where four conditions hold, and consumer reporting holds all four. The cost of a non-conformant action is externalized — onto the consumer on one axis, onto the client and a third party on the other — so the agency cannot absorb the error and move on. The workflow carries a per-transaction audit and litigation obligation, and §1681n’s statutory damages turn a per-file failure rate into class-scale exposure. A regulator supervises and expects demonstrable controls rather than assertions of intent. And the failures compound silently, by the mechanisms of §3.3, over a large book before anyone notices, so the lookback sets the remediation cost. The agency is the accountable party for outcomes it did not personally produce, on both axes at once, which is the situation in which the distinction between usually conforms and provably conforms stops being academic — for the obligations where “provably” is even available.

7. What Enforcement Can and Cannot Do

The properties that close the bright-line part of this gap are not exotic, and most of them are ordinary software engineering: hard gates, state machines, per-transaction audit trails. The non-obvious claim is not the mechanism but the precondition — that in regulated consumer reporting the specification already exists. The formalization barrier that kept formal methods out of general software, where the cost is translating fuzzy intent into a precise specification, is low here, because the FCRA, the state overlays, and the client matrix have already drawn the lines. The work is translation, not invention. So for the bright-line obligations the honest claim is strong: put a deterministic gate exactly where the law drew a line, refuse the action that would cross it, and emit per-file evidence that it was refused. Do not put a probabilistic component there at all.

For the judgment-laden obligations the claim is deliberately weaker, and keeping it weak is the point. A mechanism outside the model can enforce that the judgment was routed to the party the law assigns it to and made through the required procedure — escalate the ambiguous match, perform and record the individualized assessment, run the verification — and it can prove those things per file. It cannot make the match right, the accuracy judgment reasonable, or the assessment sound. Claiming it could would be the same category error this paper diagnoses, pointed the other way.

The boundary, restated as two limits rather than one. Enforcement proves the agent followed the specification; it does not make the data complete or the specification adequate, and that adequacy is the compliance officer’s judgment. And enforcement governs the procedure and routing around a reasonableness judgment; it does not make the judgment correct. Within those limits, what a system can honestly claim is specific and, we think, defensible: gate what the law made a line, route and record what the law left a judgment, keep a probabilistic component out of the places a deterministic one belongs, and produce per-file evidence for all of it. That is a smaller claim than “agentic systems can be made to conform to the FCRA,” and it is the true one.

8. References

Case citations are verified against the published reporters. Statutory references are to the Fair Credit Reporting Act as codified at 15 U.S.C. §1681 et seq.

Case law:

Statutes and guidance: