How the Scandit Vision AI Engine Makes Shelf Technology Smarter

How do store associates know when a bestselling product is no longer on shelf?

Inventory says it’s still in stock, and there are indeed boxes of it sitting in the back room. But without walking the aisles, no-one knows that the last packet actually on-shelf was sold 10 minutes ago.

And sooner or later, a customer will come along and find a gap where the product they wanted to buy should have been.

This is the on-shelf availability (OSA) challenge, and it’s one of the biggest problems in retail.

We built ShelfView, our shelf intelligence and execution solution, to close this gap. It analyzes images captured by mobile, fixed-position, and robotic cameras, and turns pictures into prioritized, actionable insights.

ShelfView maximizes revenue potential, minimizes inventory distortion, and prevents unhappy customers from walking out of the store without the item they want.

But it isn’t easy getting computers to understand shelves in the way humans can — looking at a shelf and knowing what to fix. Here’s how we built ShelfView to do it.

What does shelf technology do?

Shelf technology, or shelf intelligence, provides insights about what is physically happening on a shelf, not what backend systems assume is happening.

It matters because manually eye-balling shelves down every aisle cannot keep pace with product flow, and cannot scale to larger stores. Shelf technology gives retailers a reliable view of what’s really on their shelves, whether items are in the right location (supporting planogram compliance), and whether prices are accurate.

It creates a tight feedback loop between shelves, associates, and systems. This minimizes inventory distortion, maximizes revenue, and prevents unhappy customers from walking out of a store empty-handed.

How ShelfView works

ShelfView is powered by the Scandit Vision AI Engine — the brain behind every Scandit scan, guiding actions and driving efficiency across an entire business. Here’s how it works:

1. Shelf capture through mobile, fixed, and robotic imaging

ShelfView captures images from any camera source — smartphones, handheld computers, fixed-position cameras, or autonomous robots — and sends them to the cloud for analysis. This hybrid data capture approach means stores of any size can use ShelfView, and scale up hardware as their needs grow.

2. Scene parsing to identify the objects that matter

First, ShelfView identifies objects in the image relevant to shelf analysis, including shelves, trays, products, and labels (and not the floor, lights, or signage). This turns a bunch of interchangeable pixels into a distinctive, structured scene that can be analyzed digitally.

Then it uses this information to determine where products sit, which labels belong to which items, and whether a gap on the shelf is supposed to be there.

3. Label extraction and product recognition to understand details

Knowing the locations of objects and labels on a shelf isn’t enough to tell store associates what they need to do, especially if objects look similar (such as a row of milk cartons from different brands).

To distinguish individual items precisely, the Scandit AI Engine then combines two analysis paths to build a precise digital picture of the shelf:

• Label extraction: It applies the same barcode scanning and Smart Label Capture capabilities as in our other products to extract barcode, price, and other relevant label information.
• Product recognition: Image recognition differentiates every object on a shelf down to the SKU level.

ShelfView combines the label-derived data with product-recognition results to determine item location and context.

When combined with scene parsing, this produces a reliable understanding of what is on the shelf, how it is priced, and whether the product is supposed to be there. ShelfView achieves 99.7% shelf insight accuracy in production environments.

The fact that you can have the availability of the products on the shelf on a daily basis is something that you can’t compare to any manual process. I am really happy that we can use it, to have better performance and better availability of products for our clients.

Piotr Lubiewa-Wielezynski, Sales Development Director, Carrefour Poland

4. Realogram creation to represent the shelf digitally

ShelfView then uses this information to create a realogram, or digital twin, of the shelf. It contains what’s on the shelf, where items sit, and information about each item.

5. Insight extraction and alert generation

Finally, ShelfView combines the realogram with historical data and desired placements such as a planogram) to extract insights based on actual shelf conditions. This process includes two key steps:

1. Historical data analysis determines what the shelf should look like and compares that to what is on the shelf now.
2. Alerts are prioritized based on what is actionable (e.g., is the out-of-shelf item available elsewhere?) and sales data (e.g, what products sell most frequently or at a decent margin).

These results drive on-screen alerts on store associates’ mobile devices that are always relevant and actionable, reducing alert fatigue and wasted effort.

Three shelf technology problems we had to solve

Building shelf technology that truly works in the fast-paced, messy, low-margin reality of actual store environments is hard. Three of the biggest challenges we had to solve were:

• Getting usable images: It’s extremely difficult to extract information from images captured at the wrong angles and distances. We solved this by building augmented reality (AR) guidance into the mobile capture flow. This helps users position devices correctly, improving capture success rates from 75% to over 97%.
• Reducing alert false positives: After the ShelfView pipeline generates a potential alert, it doublechecks it using a vision language model (VLM). This is an AI technique to interpret visual data. It reviews the shelf image and alert context to determine whether the finding is credible enough to display. This helps reduce false positives, increasing trust in the system.
• Supporting any hardware: The core AI work happens in the cloud. This physical separation between capture and analysis allows retailers to use lower-end hardware and upgrade their devices without impacting how ShelfView operates.

AI-powered shelf technology is moving from pilot to programs

Closing the gap between system belief and shelf reality has a measurable impact. After deploying ShelfView, one of our customers achieved on-shelf availability of over 95%, directly resulting in a 2.5% increase in revenue.

We’ve all visited a store with a shopping list, only to find items missing, misplaced, or priced inaccurately. ShelfView takes retailers one step closer to no frustrated shopper ever walking out of a store empty-handed.