Who it's for
An owner or manager who is being offered AI implementation and wants to tell a real effect from a technology trend.
Key idea
AI changes the economics where a task is repetitive, errors are cheap and verifiable, and volume is sufficient. Everywhere else, it creates new oversight work.
Contents
- Four signs of a suitable task
- Where the effect is usually real
- Where the effect is more often an illusion
- What an honest pilot looks like
- The question to ask a vendor
- What to do this week
Take the self-assessment
In recent years AI has gone from something exotic to a line item in every commercial proposal. That has created the reverse problem: it has become harder to tell a task where technology changes the economics from one where it adds a layer of complexity.
Rather than arguing about the technology, it's more useful to check the task against four signs.
Four signs of a suitable task
- 01
Repetition
The task is performed dozens or hundreds of times a month. A one-off task won't pay for setup, testing and maintenance, even if it can be solved brilliantly from a technical standpoint.
- 02
A cheap, noticeable error
An error has to be either inexpensive or detectable before it becomes expensive. AI makes mistakes confidently; without a way to detect them, that's more dangerous than having no automation at all.
- 03
A verifiable result
There has to be a way to say "this is correct" without expert intuition. If only an experienced person can judge the quality, you haven't reduced the work; you've moved it.
- 04
Enough data or context
Either there's a history against which quality can be checked, or the task can be solved from instructions without training. The in-between case is the most common and the most expensive.
A practical rule
If a task meets three of the four signs, a pilot is worth running. If it meets two, it's better to wait. If it meets one, it's a presentation, not a project.
Where the effect is usually real
Scenario What changes in the economics What needs to be monitored
Sorting and routing incoming requests Speed of first response, fewer losses at the handoff Classification accuracy, handling of atypical cases
Preparing drafts of documents and proposals Specialists' hours spent on routine work Mandatory human review before sending
Extracting data from unstructured sources The cost of manual entry and the number of errors Spot checks, a confidence threshold
Summarizing communications and meetings Management time spent reconstructing context Lost nuances, responsibility for decisions
Initial analytics and anomaly detection Speed of detecting a problem False alarms, trust in the signal
Support for recurring questions The cost of service as the customer base grows The moment of handoff to a person, the tone of communication
Where the effect is more often an illusion
- Strategic decisions. A model bears no responsibility and has no access to context that isn't written down.
- Tasks with a high cost of error and low verifiability: legal conclusions, medical interpretations, final financial decisions.
- Replacing a salesperson in a complex deal. Technology can prepare the ground, but trust forms between people.
- Creative tasks where the value lies in brand recognition. An average result erodes the difference, and the difference is usually the asset.
- Any task whose process isn't documented. Here AI inherits the same problem as ordinary automation: it will speed up the mess.
Related service: Process Automation
If the process isn't documented yet, that's where to start. Otherwise AI will just repeat the mistake at a higher cost.
A hidden cost item
The cost of an AI solution almost never includes quality control: who checks the sample, who responds when quality degrades, who updates the instructions when the process changes. This work doesn't disappear; it changes form. Budget for it in advance.
What an honest pilot looks like
A pilot is there to get a number, not to "try the technology." A proper pilot is set up like this:
- 01 One task. For example, "sorting incoming requests in one channel." "Let's bring AI into the company" is too broad to be a task.
- 02 The baseline is measured before the start: how much time, how many errors, how much it costs now.
- 03 The time frame is 4–6 weeks. Longer means the task was chosen too large.
- 04 The success criterion is named in advance as a number, along with the stop criterion.
- 05 A human in the loop throughout the pilot, with every intervention recorded.
- 06 At the end, the total cost of ownership is calculated, including oversight.
A pilot without a stop criterion named in advance always ends with the words "promising overall."
The question to ask a vendor
One question separates a substantive conversation from a sales pitch: "In which cases should this task not be handled by a model?" If the answer is "there are none," you're talking to a salesperson. An answer with a specific list of limitations means you're talking to an engineer.
Related service: AI Consulting
When you need to determine which tasks in your work pass the four-sign test.
What to do this week
Questions to ask yourself
Take three tasks you're being offered to automate with AI
- 01 How many times a month is each one performed?
- 02 How will you know the result is wrong, and will you know before or after it becomes expensive?
- 03 Who will check quality six months after launch?
- 04 What baseline do you have now: time, cost, number of errors?
- 05 What result after six weeks would make you stop the pilot?
Materials that continue the topic
- Why automation speeds up chaos
- The cost of doing nothing
