The useful measure is not how fast one feature responds. It is how much less the customer has to do.
Last week I was talking with a founder who wanted people to switch to his product. He’d built something useful, but potential customers were comfortable with what they already used.
He’d found me through our Idea Validator. He entered his idea and received a starting point for testing it before we spoke, so some of the work I normally walk people through had already disappeared.
That changed the question we asked about his product. Instead of listing everything it could do, we looked at what it could take out of the customer’s day.
Customers buy work that disappears. That includes the work of using the product.
Should the workflow survive?
When companies talk about AI optimisation, they often start by mapping a workflow, finding the slow steps and using AI to speed them up. That’s useful, but it assumes the workflow should survive.
Take a weekly report. If its purpose is to support a decision, perhaps the report doesn’t need to be assembled faster. The answer could arrive where the decision is made, with the right context and evidence. The preparation, transferring, handoffs, waiting and maintenance disappear with it.
The customer doesn’t need to see the model or the machinery. They notice that there’s less work between needing something and having a useful result.
Imagine a product that summarises a document in seconds. The customer may still have to find the file, upload it, explain what matters, check the answer and move it somewhere useful. The AI step is fast, but the whole job may not be any easier.
I notice the same thing with a voice tool I use. What I want is to hold a button, speak and carry on. If I have to operate the software or rewrite its output, some of the typing it saved has returned as editing.
Maintenance counts as work too. In July, I wrote about building two sites with Ploy. I could have assembled scripts to do something similar, but I’d then have to look after them. That was part of the job I wanted the service to remove.
Count the whole customer job
When you evaluate any product, including an AI product, start when the need appears and stop when the result is usable.
Count all the work in between:
- Setting up the product.
- Explaining what is needed.
- Moving information between tools.
- Waiting, checking and correcting.
- Getting help and maintaining the solution.
The measure isn’t how fast the product responds. It’s how much less the customer has to do.
Choose one task a customer already does and watch the whole thing. Pick one piece of work to remove, then deliver the simpler experience. You can do the work manually behind the scenes if needed. That’s a Mechanical Turk pretotype.
Set the pass threshold before the test. Measure time and effort alongside accuracy, rework, help needed and trust. Then watch what happens the next time the same need appears. Do customers choose the simpler route again? What still sends them back to the old process?
Some work should remain. People need to see uncertainty, correct mistakes and approve consequential actions. Removing those controls can create more work later.
Rapidly can generate a hypothesis and experiment options, but someone still has to check the assumptions, run the test and see what customers do. The software can’t turn its own suggestion into market evidence.
The aim is to remove the work that adds no value while keeping the work people need to make a good decision and trust the result.
Take one workflow being considered for AI optimisation. Ask whether AI should make each step faster, or make a large part of the workflow unnecessary. Test the smallest useful removal and count what remains.
Working on an AI product? Put the idea through the free Idea Validator and turn it into a small experiment.