Retail AI ROI Framework: How to Design, Measure, and Scale What Works.

The world got taken over by AI.
But reportedly, 80% of all AI projects fail.

Now that more shiny new toys are available to retailers, whether off the shelf or self-built, the golden question remains: How do we get ROI from AI investments?

Insights from a panel of industry leaders underpin this guide, drawing from two recent webinars.

Our experts include:
Shirley Gao (CDIO, Pacsun), Div Singh (Director of Product Management, BJ's Wholesale Club), Roberto Croce (former SVP International, American Eagle), Natalie Berg (retail analyst and podcast host), and Pavel Ryukhov (former Head of Global Store Operations, Metro AG), alongside Scandit's own retail AI specialists.

It is intended for store technology teams and innovators looking to drive change and make the most of new AI deployments.

Here are 5 steps, drawn from the collective panel's expertise and our vision AI deployments, to help you ensure success.

• Step 1: Define KPIs before you deploy
• Step 2: Build on a data foundation you can trust
• Step 3: Design pilots that give you a real signal
• Step 4: Embed AI in daily workflows
• Step 5: Scale with change management, not just technology

These steps are sequential for a reason. The retailers who struggle to scale AI past the pilot stage are almost always missing one of the early foundations.

The most expensive mistake in retail AI is starting a pilot without agreeing on what it's trying to prove. In most instances, a goal exists, but returns can come in different forms, and accountability is essential.

Before a project begins, every stakeholder needs to answer three questions:

• What is the main KPI?
• Who owns measurement?
• What is the decision rule at the end of the pilot?

If the answer to any of those is "we'll figure it out as we go," the project will end up in the 80%.

Separate KPIs into three categories:

1. Hard returns are measurable and quantifiable: revenue lift, conversion rate, cost per contact, labor hours saved.
2. Soft returns are real but harder to quantify: strategic positioning, customer experience, brand readiness, fraud protection.
3. Long-term value: accounts for how well the investment integrates with your technology roadmap and scales across the business.

Most organizations over-index on hard returns and undervalue the others, which leads to killing projects that are actually building something important.

Also, soft KPIs are often more contested than hard ones. In luxury retail, for example, ask three different people how to measure customer experience, and you might get three different answers. So spend time discussing and agreeing on the softer returns in advance.

Finally, settle who owns the measurement. Data science and analytics, working alongside the business, should own the results. Independence is what makes the numbers credible.

GIGO. Something we are all familiar with.

This might seem like an obvious step. But it's essential and needs attention before you move on.

Your AI is only as good as the data it trains on, and data quality in most retail organizations is, unfortunately, worse than people think.

A classic example: a single SKU, described differently across a dozen fields, by a dozen different people. The algorithm sees them as different products. Its predictions and responses fall apart.

The fix is to audit your data before you build on it. Map what you have and establish a certified, governed source of truth.

But there is a more elegant approach that often gets overlooked: solving data quality at the point of entry rather than in a cleanup exercise later.

Build your interfaces so that bad data can't get in. A controlled dropdown that forces users to select from a predefined list prevents multiple variations of a data point, such as a t-shirt description, from being entered into your system.

This matters more for vision AI deployments, where systems make decisions based on the data that is captured.

In a grocery store, product catalogs serve both store and e-commerce, balance-on-hand data sits across ERPs and WMS, and price lists feed both POS and planograms. Each of these has a different owner, update frequency, and tolerance for error.

Garbage in doesn't just produce garbage out. In a vision AI system, it can amplify errors over time.

Getting a pilot running is the easy part. Scaling it is where the 80% failure rate comes into play.

The difference is down to early decisions in four areas:

• Resource allocation. AI pilots now involve a broader cast than most retail tech projects: product, IT, business, and increasingly data scientists. Decide early what you build in-house and what you outsource. AI attracts scope creep as understanding grows and use cases multiply fast. Agree on boundaries, or you will run out of resources before it ends.
• A clear path to proof of value, within 3 months. If you cannot demonstrate real value within three months, the scope is too large, or the foundations are not ready. Agree on the threshold at which you would scale, and the threshold at which you would stop.
• Real world-testing. Keep test conditions tight. Agree on the comparison group in advance and measure the metric that matters to the business. Make sure you can explain why the AI made each decision so you can articulate the reasoning behind a result.
• A Statement of Work. Write it and get every party to sign it. Agree on scope, timeline, responsibilities, and decision rules before anyone builds anything. It feels unnecessary when things are going well. It is indispensable when they are not.

Realistic hit rates for exploratory AI pilots range from 10 to 15 percent. It’s the nature of innovation. The retailers who build momentum fail fast, learn something useful from each attempt, and move on.

AI outputs need to be in the hands of the people who can act on them, fast.

For store operations, that means an alert on the device a store associate already carries.

As a practical test, how long does it take from when the AI identifies a shelf issue to when a human can do something about it? If the answer is hours or days, the workflow needs to be redesigned.

But speed of delivery is only half the equation. The other half is whether frontline workers actually want to use it.

Retail is fundamentally a people business. When Metro AG deployed AI-powered tools across its store operations, the technology worked. What pleased the team was how naturally employees took to it because it was built with them and tested with them first.

Workers didn't see it as a productivity tool sent down from head office. They saw it as something that made their daily routine easier. The clearest sign it was working? Staff wanted the rollout to move faster.

The technology is almost never the reason AI doesn't scale in retail. The reason is that not enough people in the organization believe it works, understand what to do with it, or feel ownership over the outcome.

The model that consistently works is embedding a power user in each affected functional area.

Someone who uses the tool daily, understands it well enough to help colleagues with it, and can demonstrate a small win that their team can see directly.

When an associate gets notified that a high-margin item is out-of-shelf, and a colleague watches them act decisively to fix it instantly, that moment is worth more than six months of top-down communication.

Two things tend to kill adoption before it takes hold.

• Pilot fatigue: Organizations that run pilot after pilot without achieving decent outcomes risk losing frontline team engagement.
• Misframed productivity messaging: Telling a team that AI will make them more efficient can feel like code for headcount reduction. The conversation about what changes as a result of a productivity improvement is better led by the business owner whose team it affects, not by the technology team that built the system.

The challenge with scale is an organizational one. The retailers who focus on change management as much as technology will succeed.

The five steps are not complicated. What makes them hard is that each requires decisions that most organizations often defer.

The retailers translating AI investments into real returns are not necessarily those with the most sophisticated technology. They are the ones with the organizational discipline to make hard decisions early and stick to them.

As one panelist commented, retailers are spending too much time chasing sexy AI. The real opportunity is in the boring stuff, like inventory accuracy and worker productivity.

Applied consistently, at scale, those marginal gains add up to something that really delivers ROI.

Vision AI-powered Shelf Intelligence and Execution

99.7% accurate shelf insights

About ShelfView

FAQs

Still got a question?

Contact us

Why do most retail AI pilots fail to scale?

The gap is rarely the technology. It comes from rushing into pilots without agreeing on what success looks like, from trusting algorithms nobody has verified, and from not involving frontline workers from the start.

What KPIs should we set before starting an AI pilot?

Separate KPIs into three categories: hard returns that are measurable and quantifiable, soft returns that are real but harder to quantify, and long-term value that accounts for how well the investment integrates with your technology roadmap.

How do we know if our data is ready for AI?

Audit before you build. A single SKU described differently across a dozen fields by a dozen different people looks like different products to the algorithm. Its predictions fall apart, and nobody knows why until someone goes looking.

How long should an AI pilot take to show value?

If you cannot demonstrate real value within three months, the scope is too large, or the foundations are not ready. Agree on a clear scaling threshold and a stopping threshold before anyone builds anything.

What does a realistic AI pilot success rate look like?

Realistic hit rates for exploratory AI pilots range from 10 to 15 percent. The retailers who build momentum fail fast, learn something useful from each attempt, and move on.

How do we get frontline workers to actually use AI tools?

Build with them, not for them. Workers who are involved in testing from the start see the tool as something that makes their routine easier, not a productivity initiative handed down from head office. The conversation about what changes as a result of an AI deployment is better led by the business owner whose team it affects.

What is the biggest mistake retailers make when scaling AI?

Treating it as a technology problem. The reason AI doesn't scale in retail is that not enough people in the organization believe it works, understand what to do with it, or feel ownership over the outcome. The retailers who succeed focus on change management as much as the technology itself.