On June 22, 2023, U.S. District Judge P. Kevin Castel, in New York, fined two lawyers and their firm $5,000, jointly. The reason: in a case against the airline Avianca, they had filed a brief citing six court decisions as precedent. None of the six existed. All of them had been invented by an AI assistant, complete with case names, docket numbers, and excerpts that looked entirely real.
Two and a half years later, in October 2025, a similar episode unfolded on the other side of the world — with a much larger sum on the table. Deloitte Australia refunded the country's government more than 97,000 Australian dollars out of a 440,000-dollar contract, over a report reviewing an IT system used to apply penalties in the welfare program. The document contained a fabricated quote from a Federal Court ruling and references to academic studies that never existed.
The two cases happened in different sectors, in different countries, two years apart. And they landed on exactly the same conclusion: the bill for an AI mistake does not go to whoever built the model. It goes to whoever signed the document. Neither incident required a novel legal theory or a new regulation to produce consequences — existing rules on professional conduct and public contracting were enough, once someone finally checked the sources.
The pattern behind both cases
In the Avianca case, attorney Steven Schwartz went as far as asking the AI assistant itself whether the cited cases were real. The answer — also generated by the AI — confirmed they were, another complete fabrication. When the court asked for an explanation, the lawyers were slow to admit the problem, and that delay weighed heavily against them in the judge's ruling.
In the Deloitte case, the report went through an internal review before being delivered to the government in July 2025. It was only after publication that a University of Sydney researcher noticed much of the citation list did not exist, and alerted the press. The corrected version kept the original recommendations but stripped out more than a dozen fake references and footnotes.
In neither case was the AI model's maker notified, fined, or named as responsible. The fine went to the law firm. The refund came out of Deloitte's own accounts. That is the pattern any company using AI to produce a document, an analysis, or a filing needs to understand before its own incident — not after.
It is also worth noting the difference in context: one case is a civil-court filing; the other, a consulting report delivered to a public agency. Different sectors, different document types, and still the same outcome. That is what turns two isolated incidents into a pattern any company producing AI-assisted documents should take seriously — this is not exclusive to law firms or consultancies.
Why no one caught the error before it went out
The most important thing the two cases have in common is not the technology involved. It is the absence of one specific step: someone, separate from whoever wrote the document, checking the sources before it left the building.
That explains why the error slipped through in both cases. A citation invented by AI looks exactly like a real one — name, format, tone, all correct. It is only caught by someone who verifies the original source, not by someone reading for a typo or a weak argument. Without that step, the document leaves the company looking exactly as it should, and the error only surfaces once someone outside — a judge, a researcher, a journalist — goes looking for the source.
NIST, the U.S. technical standards body, describes this in what it calls the "Govern" function of its AI risk-management framework: before any technical control, an organization needs clarity on who is accountable for each AI-assisted decision, who reviews it, and who signs off on it. In both cases examined here, that question had no answer — and the price of not having one was paid later, in public. That framing matters because it moves the conversation away from banning a tool and toward building the missing role: someone whose job is explicitly to check, before release, whatever AI helped produce.
What Has to Be in Place
A governed corporate AI environment reduces this risk by design, not as a reaction to an incident.
Two-step review and approval for critical documents. Whoever writes AI-assisted content is not the same person who approves its release to a client, a court, or a public agency — configurable by document type, with segregation of duties and audited exceptions.
Knowledge with a traceable source. An answer grounded in approved internal documents can cite where each piece of information came from, instead of generating a reference no one can later verify.
Human approval before a sensitive action. Before content becomes an official deliverable, the workflow can pause and ask for a person's explicit confirmation, inside the conversation itself — not as an optional step someone forgets to take.
An audit trail. Who produced the draft, who reviewed it, who approved it, and when: all logged. When an error surfaces after publication, that trail is what lets a company reconstruct what happened in minutes, instead of denying it in public without knowing its own history.
This is how Skyller was designed: two-step review for critical content and an audit trail as the default, not a feature the team has to remember to turn on after a scare.
The cost of not having that step
The fine in New York was small; the real cost was something else. The lawyers had to write, one by one, to every judge whose name had been attached to a fabricated ruling, explaining what had happened. That is now on the record, is public, and follows both attorneys to this day.
In Deloitte's case, the refunded amount was also smaller than it first appears — less than a quarter of the original contract. The bigger cost was the international headline and the question left hanging over how many other reports from the same firm, delivered to other clients, never had anyone checking the sources.
That is the pattern that repeats: the size of the fine or the refund is almost never the real problem. The problem is not having, after the fact, a ready answer to "who approved this, and based on what." Companies that already run two-step review and an audit trail answer that question in minutes. Everyone else answers with an internal investigation that takes weeks — and often goes nowhere, because the record never existed.
There is also a third cost, harder to measure and longer-lasting: the trust of whoever reads that company's or that professional's next document. After an episode like this, every following delivery gets read with more suspicion, even one reviewed with great care — and rebuilding that trust takes far longer than avoiding the mistake would have.
A checklist for the next meeting
Before legal, finance, or communications live through their own version of these two cases, it is worth answering these questions clearly:
- Which AI-assisted document leaves the company without a second person checking the sources? If the answer is "several," that is the first gap to close.
- If a client or a court asked the company to prove where a piece of information came from, could it show the source? Without knowledge tied to a traceable source, the answer tends to be no.
- Is there a record of who approved every AI-assisted deliverable over the last six months? If not, the company is exactly where Deloitte stood before October 2025.
- Who, in practice, is accountable when a document goes out wrong? If the answer takes too long to arrive, that is a sign the structure does not exist yet — only the hope that the mistake won't happen.
