In January 2023, the US National Institute of Standards and Technology, NIST, published a voluntary risk management framework for AI systems — now adopted by companies in several countries, including outside the United States, as a reference for deciding what to automate and how. One of the framework's four core functions is called "Map," and its logic is straightforward: before deciding whether an AI system should move forward, a company needs to understand the context — what it's for, who could be affected, and what the cost, including non-financial cost, of a mistake would be.
The document is explicit about when this should happen: after mapping the context, the organization makes an initial go or no-go decision about that system. Not after it's built. Before.
In most companies, this step simply doesn't exist as a formal process. Someone spots a repetitive task, builds an AI automation to handle it, and only discovers — if they ever do — that the task was actually deciding something serious about a person, after the mistake has already happened.
This isn't a lack of care from whoever built the automation. It's the absence of an earlier step: nobody asked, before starting, how much weight that specific decision actually carried. Asking the right question at the right moment is cheaper than any fix made after the system is already in production and someone has already been affected by it.
Not every automation carries the same risk
The European Union arrived at a similar conclusion through a different path: regulation. The bloc sorts AI systems into four risk levels. At the top, unacceptable risk — now prohibited. Just below, high risk: systems used in decisions about employment, access to credit, education, critical infrastructure, or biometric identification, which can affect a person's safety or fundamental rights. Then comes transparency risk, for systems that just need to disclose that a person is interacting with AI. And at the base, most of today's use cases: minimal risk, like a spam filter.
The point that matters to any company, whether or not it falls under this regulation, is the logic behind the pyramid: the same label "AI automation" covers completely different situations. An assistant that summarizes internal meeting notes and a system that helps decide who gets a loan should not go through the same approval process, or the same level of oversight. Treating them as the same kind of thing is where most of the trouble starts.
Automating everything the same way is the mistake
The most common response to enthusiasm about AI inside a company is "let's automate everything" — a phrase that, without a risk filter behind it, produces two bad outcomes at once, one on each end.
On the low-risk end, applying the same heavy review and approval process to a trivial task — like summarizing an internal document — kills adoption. People stop using the tool because the process around it is slower than just doing the task by hand. On the high-risk end, the problem is the opposite and far more serious: an automation that decides something about a person's job, credit, or health runs with the same informal level of oversight as a meeting summary — which is to say, almost none.
NIST's own framework names this second risk directly: before deciding to move forward with a system, the organization needs to examine the potential costs, including non-monetary costs, of expected or actual errors from that system. A mistake in a meeting summary costs a misunderstanding. A mistake in a decision about a person costs far more — and that cost usually shows up later, when it's too late to prevent it with a better process.
The way out isn't choosing between the two extremes — not everything locked down, not everything left loose. It's recognizing, case by case, which end each automation falls on before deciding how it should operate. One simple question usually settles it: if this automation gets it wrong today, will someone outside the technology team feel the effect? If the answer is yes, the level of oversight needs to match.
What has to be in place
Classifying risk before automating only works if a few concrete mechanisms come with it — not as a promise, but as part of the design.
Approval based on risk. The higher the identified risk, the more the automation should pause and ask a person to confirm before proceeding — not on every automation, only on the ones that decide something serious.
An audit trail. Every relevant automation should log what was decided, with what information, and under what authorization, in a way that can be looked up afterward — not just while it's working well, but especially when something goes wrong.
Approved knowledge with sources. The information feeding a high-risk automation needs an owner, a current version, and periodic review — it can't come from an outdated document nobody reads anymore.
Access based on role. Only people with actual authority over that routine should be able to create, adjust, or approve an automation classified as high-risk — not just anyone with access to the tool.
Periodic review of the classification. The risk of an automation changes over time — when it starts feeding another system, reaching a different audience, or running at a larger scale. The initial classification needs to be revisited, not set once and forgotten.
This is why Skyller ships with more than 170 pre-built policy and process templates, anticipating what level of approval and record-keeping each type of routine typically requires — instead of leaving every team to figure it out on their own, the first time something goes wrong.
The payoff of classifying before deciding
The most immediate gain is speed — though not in the direction most people expect. A company that classifies risk before automating can move fast on the trivial tasks, precisely because it doesn't stack process where it isn't needed. The heavy process gets reserved for where the risk actually calls for it.
The second gain shows up later, and it's the one that avoids the real headache: when something goes wrong in a high-risk automation — and at some point something will — the difference between a company that classified risk beforehand and one that didn't is the difference between explaining what happened in minutes, with documented records and criteria, and spending weeks reconstructing, with no real certainty, what exactly the automation decided and why.
There's also an effect on internal trust. A team that knows a high-risk automation was reviewed and approved uses it with far more confidence than a team that rightly suspects nobody actually evaluated what that system does.
None of these gains depend on getting the classification perfectly right on the first try. They depend on having a classification at all — one that can be revisited — giving the company a concrete reason to treat two automations differently, instead of deciding that on impulse, a different way each time.
Five questions before automating
Before approving the next AI automation, these five questions settle most cases:
- Who is affected if this goes wrong? A customer, an employee, a supplier, or just the internal team itself — the answer already points to the risk level.
- What does it cost to fix the mistake, and who pays that cost? An internal misunderstanding is not the same as a decision that needs to be reversed after it has already affected someone outside the company.
- Where does the information used to decide come from? A document with an owner and a current version is different from a lost file nobody has reviewed in months.
- Who checks it before the decision takes effect? If the answer is "no one," that's the gap to close before anything else.
- What gets recorded afterward? Without a record, an audit or a complaint turns into piecing memory back together instead of consulting a history.
