Most AI projects are declared a success by the people who ran them. That is not measurement. Measurement means deciding beforehand what would count, recording where you started, and comparing honestly afterwards.
It takes little effort, and it has to be set up before the project starts. Afterwards is too late.
01Take a baseline first
Before anything is built, record the number the project is meant to change: hours per week on the task, invoices paid twice, days to answer a tender, enquiries from search. Without a before, there is no after, only an impression.
02Pick one primary measure
- Time: hours saved on a task, counted over real weeks.
- Money: purchases avoided, payments stopped, stock released.
- Quality: errors found, returns reduced, records corrected.
- Speed: days from request to answer.
- Revenue: enquiries, conversions, orders, where the link can be shown.
One measure decides the verdict. The others are context. A project with seven goals can always claim to have met one.
03Count the full cost
- The build or the subscription
- The time your own people spent on it
- Running it: hardware, hosting, licences, support
- Cleaning the data it needed
- Training, and the slow first weeks while people learn
04Compare fairly
- Use the same period before and after, and allow for seasons.
- Compare with a team or site that did not get the system, if you can.
- Wait until the novelty has gone. The second month tells you more than the first.
- Ask whether anything else changed at the same time.
- Count what people actually do, not what the system could do.
05What a spreadsheet misses
- Decisions made on better information
- Risk avoided: a payment that did not go out, a clause that was noticed
- Staff spending time on work that needs a person
- Knowledge that now stays when someone leaves
06Deciding what happens next
At the agreed date, look at the primary measure against the full cost. Extend what worked to the next process or site. Fix what nearly worked. Stop what did not, and write down why. A project that is stopped honestly is worth more than one that is quietly kept alive.
In short
- Record a baseline before the project starts.
- Choose one primary measure and let it decide the verdict.
- Count your own people's time, running costs and data clean-up.
- Compare like with like, after the novelty has worn off.
Questions
How soon should an AI project pay back?
It depends on its size. A small automation should show a result within weeks. A system that changes how a department works takes longer, and should still show movement on its measure early.
What if the benefit is hard to count?
Count the nearest thing that can be counted, and record the rest in words. If nothing can be counted at all, question whether the problem was clear.
Who should do the measuring?
Someone other than the team that built it, ideally the owner of the process it changed.
Does Quantum Beetle agree measures at the start?
Yes. The measure and the baseline are part of the scope agreed before work begins.
Sounds like your problem?
Tell us about it. We'll say honestly whether the swarm can help, and what it would take.
Read next
- AI readiness assessment: a checklist for leadership teamsAI readiness
- AI threat assessment: which parts of your business will AI erode firstAI threat assessment
- Where to start with AI in a small businessGetting started
- An AI policy for employees: what to allow, what to forbid and how to say itAI governance
- Shadow AI: when staff use AI tools with company data, and what to do about itAI risk
- An AI roadmap in three columns: now, next and laterAI planning