← All articles

AI product management

Your AI roadmap should be a list of questions

Feature roadmaps assume you know what the software will do once it is built. With AI, you find that out after you ship.

A normal roadmap is a list of features with dates. It works because traditional software is predictable. If the team builds the export button, the export button exports. The risk sits in delivery: will it be done on time, and will it be done well.

AI features break that assumption. You can build exactly what the spec says and still not know how well it works, because the output depends on a model, on your data, and on inputs nobody has seen yet. A roadmap that treats an AI assistant like an export button is usually wrong by the second sprint, and the team spends the rest of the quarter explaining why.

The problem with "ship the assistant in Q3"

A line like that hides three separate unknowns.

  • Can the model do the task at all? Nobody knows until someone tries it on real data.
  • Is it good enough for this user? That depends on a quality bar nobody has written down yet.
  • Will it stay good? Data and usage change after launch.

A date on a feature answers none of them. It just moves the surprise to the end of the quarter, when it is most expensive.

Stages that end in a decision

I now plan AI work as five stages. Each ends with a question the team must answer before moving on.

Stage The question What you need to answer it
1. Problem and baseline What does this task cost today? A number: hours, errors or money
2. Feasibility Can a model do it at all? A small experiment on real data
3. Evaluation What does "good" mean here? A test set and an agreed bar
4. Guarded pilot Does it hold up with real users? A few users, human review, full logging
5. Scale Do quality, cost and safety hold under load? Monitoring and a rollback plan

The stages are cheap at the start and expensive at the end, which is the point. If the idea fails at stage two, you have lost a week. If it fails at stage five because nobody ran stage two, you have lost a quarter.

Notice that the interface does not appear until stage four. Teams love to design the chat window first. It is the least risky part of the product.

Budget for behaviour you did not design

On the retail forecasting work I led, the model was only part of the product. Most of the effort went into what happens around it: what the planner sees when the forecast is uncertain, how they override it, and how we would know if it started to drift. Stockouts fell 35% and excess stock fell 12%. I credit the workflow around the model as much as the model.

So I reserve roadmap capacity for three things a feature roadmap usually leaves out.

  1. Evaluation. Test sets go stale as the product and its users change. Someone has to own them.
  2. Monitoring. Quality in production is a number you look at every week, like uptime.
  3. Fallbacks. What the product does when the model is wrong, slow or unavailable.

As a rough guide, I expect these to take a third of the engineering time on an AI feature in its first year. If the plan shows zero, the plan is hiding work.

What to show stakeholders

Leaders still want dates, and that is reasonable. I give them dates for decisions:

By the end of the month we will know whether this is accurate enough to pilot with ten customers.

That is a commitment the team can keep, and it tells the business something useful either way. A "no" at the end of the month is a result. It frees the team for the next idea.

I also show the stage each initiative is in. A portfolio with everything stuck at "feasibility" tells you something different from one with everything in "pilot", and neither is visible on a feature list.

Questions to ask your team this week

  • For each AI item on the roadmap, which stage is it really in?
  • Which items have a written baseline, and which are running on belief?
  • Who owns the test set after launch?
  • What does the user see when the model is unavailable?

The roadmap stops being a promise about features and becomes a plan for reducing uncertainty. In my experience that is the version that survives contact with production.

Found this useful? Share it, or get the next one through The AI Product Playbook, my LinkedIn newsletter.

The AI Product Playbook

Get new articles by email

Leave your email and I'll send you each new article on AI product management as it is published. You can also follow the newsletter on LinkedIn.