From Pilot to P&L: What Retail AI Leaders Really Think

I was joined by Shirley Gao, CDIO at Pacsun, Div Singh, Director of Product Management at BJ's Wholesale Club, and Roberto Croce, former SVP International at American Eagle, as we explored one of the most pressing questions in retail today: what actually separates an AI proof of concept from a financial return you can take to a CFO?

The session was hosted by the Retail AI Council. They gathered senior practitioners who have run the pilots, defended the budgets, and lived with the results.

Here's what I came away thinking.

The ROI framework hasn't changed as much as you'd expect

The first thing that struck me was how consistent experienced retail leaders are in their evaluation of AI investments.

Despite all the changes in the technology itself, the best frameworks still separate hard returns (measurable revenue lift, cost reduction, trackable KPIs) from softer returns (operational efficiency, brand readiness, fraud protection) and longer-term value (how well an investment integrates with your technology roadmap and scales across the business).

What has changed is the discipline around applying that framework.

There's less patience now for vague pilots. Business owners want a clear north star metric before a project starts.

They want to agree upfront on who owns the measurement, what "good" looks like, and how you'll know if it isn't working. Investing time in that clarity at the beginning is the best way to get started.

Good data is still the foundation. And still underestimated

If there was one point of genuine consensus across the panel, it was this: the quality of your data determines the quality of your AI.

We've known this for years. It's still underestimated.

One panelist described building a demand forecast where a black t-shirt appeared in the system with "LG" in one SKU and "LARGE" in another. The algorithm had no way of knowing they were the same product. Years of investment, undermined by a data entry convention that nobody had standardized.

But what stayed with me was a different idea: solving data quality at the design level rather than the cleanup level. And it’s something we are focused on helping Scandit’s customers with through our intuitive interfaces and high-performance data capture.

Build your interfaces so that bad data can't get in.

For example, a dropdown that forces users to select "Large" rather than typing it free-form eliminates that class of error before it starts.

Preventing problems upstream is always cheaper than fixing them downstream.

Simulations are becoming more reliable

One of the persistent challenges in retail AI is that measurement is inherently messy. You can't run a clean, controlled experiment when a competitor opens a store next door mid-pilot, or when an unseasonable heatwave shifts demand across your test window.

For years, simulations served as a theoretical workaround, but they were often unreliable.

That's changing. Agentic AI is making simulations considerably more capable by enabling the addition of more objectives, variables, and real-world complexity.

It doesn't eliminate the need for real-world testing, but it raises the quality of the signal you get before you commit at scale.

The feedback loop is now a conversation

Perhaps the moment that stayed with me most was around how feedback loops have evolved. In early machine learning, the model gave you a number, and you acted on it. Humans were fairly passive.

Now, with agentic workflows, a store associate can tell the system it made the wrong call, explain why, and improve the algorithm. In natural language. Without a data scientist in the loop.

This matters more than it might sound. Inventory management, to take one example raised in the session, is not a pure data science problem.

Experienced buyers know things the algorithm doesn't:

• a particular vendor's reliability
• an upcoming local event
• a category quirk that's unique to their market

That knowledge used to be locked inside people's heads. Now it can flow back into the system.

At Scandit, we see this dynamic in our ShelfView deployments that are powered by vision AI.

When shelf monitoring insights are embedded into daily store workflows rather than presented in a weekly report, the quality of feedback improves significantly.

Associates notice when something doesn't look right. They flag it in the moment. The system gets smarter. The value compounds over time rather than plateauing after launch.

For vision AI, the real ROI is on the revenue side

I want to be direct about this because it reflects what we consistently observe across our retail deployments.

When vision AI is used to improve on-shelf availability or planogram compliance, the primary business case is sales, not labor reduction.

When a product is on the shelf when a customer wants it, the retailer makes money. When it isn't, they lose the sale and often the customer.

The gap between what inventory systems report and what actually sits on a physical shelf is larger than most retailers realize, and vision AI continuously closes it rather than through periodic manual audits.

Labor efficiency tends to be a side effect. The store associate who used to walk every aisle is now guided directly to the item that needs attention. They fix it faster. The time saved gets redirected to something more valuable.

It is a real productivity gain, not a reduction in headcount.

Change management is where most pilots fail

Every panelist touched on this. Repeated POCs without clear wins create fatigue. Fatigue breeds skepticism.

The adoption model that resonated most was the bottom-up approach rather than the top-down approach.

You need people embedded in each functional area who genuinely understand the tool, can show small wins to their colleagues, and build the credibility that no steering committee can manufacture.

Executive sponsorship matters for direction and budget. It doesn't create believers. Believers come from seeing something work with their own hands.

One practical point worth flagging: if you're approaching this from the technology side of the organization, be careful how you frame productivity improvements.

The conversation about what to do with the time saved is better led by the business owner whose team it affects. Technology's job is to prove the capability works. Business leadership's job is to decide what changes as a result.

Main takeaways

Retail AI is maturing, but the fundamentals haven't shifted. Measure what matters. Fix your data first. Embed AI insights into the actual work, not into a separate dashboard that gets checked when someone remembers to look at it.

The new tools, particularly around simulation quality, natural language feedback, and agentic workflows, are genuinely moving the needle on what's achievable.

But the retailers who will extract real value aren't necessarily those with the most technology.

They're the ones who've built the organizational habits to act on what the AI tells them.

That's still the hard part. And it's still what separates a pilot from a P&L impact.

Recommended reading

Blog post

Take a Look Inside the Scandit Vision AI Engine

Blog post

The Many Eyes of Retail Computer Vision - And Why Stores Need Them All

Blog post

How AI-Powered Label Scanning Reads Like a Human (But Faster)