In May 2026, Onyx published a dataset for evaluating artificial intelligence assistants inside companies. It holds slightly over 500,000 documents from a fictional business, spread across nine origins — internal messages, email, sales records, meeting transcripts, tickets, wiki — and 500 questions an employee would ask.
One design decision stands out: twenty of those 500 questions are about information that does not exist in the corpus. It is not a trick or a statistical flourish. It is an evaluation category with a name of its own, and a system only scores on it by stating plainly that it found nothing.
Anyone who works in operations understands immediately why that is there. The costly mistake is not the assistant that fails to answer. It is the assistant that answers well, in confident sentences with impeccable formatting, about a policy that was revoked in 2023.
Where the time goes when nobody can find the document
A survey published in January 2026 by Slite, with slightly over a hundred professionals, estimated 3.2 hours a week spent looking for information inside one's own company. It is a small sample and the figure should be read as an order of magnitude, not as national statistics. Even so, anyone who has worked in an office recognises the scene.
The hour does not disappear in the act of opening the folder. It disappears before and after. It goes into asking the group chat whether that procedure still applies. It goes into waiting for the person who knows. It goes into redoing work because the spreadsheet used was the second-to-last version. And above all it goes into double-checking: when nobody trusts the file they found, everybody checks twice.
The cruel detail is that being an organised company does not fix this on its own. The corpus in the Onyx test was deliberately messed up, with files stored in the wrong place, near-duplicates across different systems and documents that contradict each other. Not because the fictional company was careless, but because that is what any company looks like after a few years of operating.
Why "the AI answered" is not the same as "the answer is usable"
There are three failure modes that show up every time, and none of them is solved by swapping the AI model for a newer one.
The first is the wrong version. The right text exists, but it exists in two copies, and the assistant picked the old one. From the reader's point of view the answer is perfect — well written, citing a real document, and out of date.
The second is the unresolved contradiction. Two valid documents say different things about the same subject, usually because an exception was agreed over email and never made it back into the policy. An assistant that picks one of the two without saying so has just made a decision that was not its to make.
The third is confident invention. When there is no basis, the easy path is to assemble a plausible answer from what the model learned about the world. To a hurried reader, it is indistinguishable from an answer grounded in an internal document. That is why the category of twenty questions exists in the test: without measuring refusal, a system that never says "I don't know" looks better than it is.
None of these three problems is about eloquence. All three are about which document reached the answer — and about the company having said, beforehand, which document counts.
What has to be in place
The mechanisms below are generic: they apply to any tool you are evaluating, and they are what turns "the AI answered" into "I can act on this".
A document with an owner and a current version. Someone is accountable for the text and there is a single copy that counts. Without that, no search fixes it — it will find both versions and pick one.
The source alongside the answer. The reader has to see where it came from, open the file and check it in two clicks. An answer with no origin is an opinion dressed as a fact.
Two-step review and approval for critical documents. Before a text becomes the reference the AI uses, a second person checks it. This is configurable and applies to what is critical, with separation of duties and recorded exceptions — not to every answer.
Access according to each person's role. The answer someone receives has to respect what that person could have opened on their own. A search that ignores permissions is not a better search: it is a quiet leak.
An expiry date on what expires. Certificates, licences, powers of attorney and contracts have deadlines. What has expired must stop answering on its own, without depending on someone remembering.
Refusal as expected behaviour. The tool must be able to say it found nothing, and the company must treat that as a success — because it is. That is how Skyller was designed: approved knowledge with sources, reach according to each person's role, and an explicit warning when there is not enough basis to answer.
What changes for the people doing the work
For the person handling a request, the gain is finding information with its source and checking what supports the answer before using it. This reduces manual searching and helps ground the decision.
When the text has an owner, a version and a date, checking whether it still applies becomes easier. When an answer includes its source, the person can open the policy and share the link with the customer. Review before publication reduces the risk of a document problem reaching customer service; the answer still needs checking according to the decision it supports.
When the assistant reports that it found nothing, the person can identify a gap and ask the topic owner for help. This refusal is important, but it does not prove that every other answer is correct. Accessible sources and human review help people use AI with judgment.
No tool can find a document that was never written. If a search does not locate the reimbursement policy, the answer should say that no supporting information was found in the consulted corpus. The owner can confirm whether the document exists and needs to be added, or whether the policy still needs to be written.
A route to get started
- Pick a question your team asks every week. Preferably one that currently becomes a message to a colleague. That is your test case.
- Find out how many versions of the document answer it. If there is more than one, the problem is not artificial intelligence yet. It is filing.
- Set the owner and the current version of that document. One accountable person, one copy that counts, one review date.
- Demand to see the source before you demand speed. When evaluating any tool, ask for the answer with the origin document beside it, and open the file.
- Test with a question that has no answer. Ask something you know is undocumented and watch what happens. A tool that invents an answer has just shown you what it would do with a serious subject.
- Agree on what happens after a refusal. "I didn't find it" has to become someone's task, with a deadline — otherwise the gap is still there next week.
