Robots, in-store drones, AI, and machine recognition: Brick-and-mortar retailers are fighting back
Brick & mortar retail has been on its heels, but new technologies to cut the fat and increase efficiency are helping physical stores compete with ecommerce.
Pensa Systems, a company that makes in-store drones that buzz around scanning shelves to give retailers real-time inventory data and extract useful data, is one of the companies at the forefront of the retail revolution. I reached out to CEO Richard Schwartz to help explain the new wave of in-store tech.
Why is retail such a hotbed for innovation when it comes to the use of robotics?
Today, the retail industry is under pressure and in a state of disruption. Retailers are looking for more efficient ways to automate in-store activities without incurring significant cost. Historically, the supply chain (from warehouses to distribution centers to retail front/back of the store) has been quite controlled. Now the supply chain is being compressed, and there are more uses of the same on-shelf inventory for consumer shopping, such as direct to home and pickup at curb delivery to name a few. Physical stores are both a huge asset for retailers as they are close to the customer, but also very manual and labor intensive to maintain. Robotics, if not too invasive or capital intensive, is one way to shift the in-store workforce to customer facing activities and away from more tedious activities -- such as staring at the shelves to locate stockouts.
What is the value that aerial drones can bring to retail stores? Is it just the drones or other technologies that power the system, like AI?
The aerial drone is the tip of the iceberg of a bigger system involving AI and machine recognition. The drone itself is an aerial robot, with a small camera which acts as a set of 'roving eyes'. The drone is low cost, very agile, and can be deployed simply. It can literally move within a few seconds to any aisle or portion of the store. It is actually operating away from people by moving into areas which are empty and fade away to another aisle if someone enters.
At the same time, as the drone moves through the air and peers at the shelves, it is able to see the items on the shelf from many angles and perspectives -- as if there were up to 100 fixed cameras focused on every item on every shelf. This enables dramatically more accurate recognition of SKUs over fixed cameras or even ground-based rolling robots. The drone itself is neither processing the scene nor computing the image recognition or AI. That happens at the 'edge' of the network within the store location and in the cloud, so it can scale across a large number of locations with the same training applying throughout a large deployment network.
What is the AI doing actually? What else could such "autonomous perception" AI help with as other use cases?
The magic of the system is the machine learning (deep learning) that is able to recognize products on the shelf "in place" (in situ) as it moves along. The system visually recognizes products on the shelf (trained much like a person would), recognizing what it recognizes and recognizing what is new and incrementally learning. The natural variations of the products on the shelf cause the system to be smarter and more robust at the recognition. Also, an important part of the "cognition" of the AI system, is that it learns how the shelf is organized by looking at how it changes over time, how and where products are replenished etc. We infer automatically a kind of 'observed planogram' which enables the Pensa system to automatically build its own model of the shelf and how it is organized for product assortment. From that, it can automatically detect and alert for what is out of stock, running low, is mis-placed, mis-faced etc. -- without requiring detailed reference plans or API integration (which are often either unavailable or simply wrong or out of date).
This form of autonomous perception, where the AI system is learning from visually observing and concluding as a person would, can be quite important over a wide range of other use cases -- such as condition of products on the shelf, performance of end caps, floor spills, temperature or equipment problems, freezer doors left open, over-ripe produce etc. This is served by an autonomous system which can see and can be trained to draw conclusions from what it sees.
Do you think drones will eventually replace other approaches to tracking on-shelf inventory, including robots and fixed cameras? What's best for managing retail inventory?
Aerial robots are the best solution for many applications. Electro-mechanical robots are big, bulky, expensive and cannot avoid people, which can cause extra problems in terms ofliability. Most importantly, they are still not able to get an adequate view of all shelves and inventory. From the air, drones can see almost anywhere, see from many perspectives and vantage points, and can be dynamically redirected to different aisles and situations within the store. Fixed cameras and smart shelves (a la Amazon Go) can be effective for certain applications -- at potentially a high per-store capital expenditure and rigid buildout of the physical infrastructure. The aerial robot/drone can be less than 1/100 the cost of other approachesin the market, can be set up in less than 2 hours and is highly accurate and more efficient in identifying out of stocks thanother approaches, such as fixed cameras and large electromechanical ground-based robots. That said, the ultimate solution across retail categories and the entire supply chain will be a mix. In fact, from our perspective, our backend AI system can process visual inputs from multiple front-end devices -- drones, mobile phone apps etc.
Must-see offers
Will retail stores become fully automated in the future, like Amazon Go? Is that good or bad? How would robots/drones help you get there? How can tech in general help you get there?
Amazon Go is a bit of a tour de force. It uses a host of smart shelf-sensing, ceiling cameras, and other detectors to automate both the checkout experience as well as the available inventory. It will be most practical for small convenience stores, automated kiosks (such as in the airport or employee self-service). It will be least applicable in larger format stores and more varied-use inventory. Retail stores in general are turning into logistics hubs where the inventory is being touched and moved by many people for different tasks. Amazon Go will be best at automating checkout rather than inventory in supply chain.
Are there other industries that could benefit from the use of indoor aerial drones?
Yes. Training indoor autonomous drones to visually recognize what is right, what is wrong, and what is normal, has applications across the board -- across supply chain, manufacturing facilities and general office. In effect, visual perception (automated experience oversight and visual audit) is a broad-based killer application. Deep learning, image and scene recognition with autonomous camera/sensors to get around to 'see around' has broad applicability. You can also think of autonomous perception as indexing the physical world and everything within it, over time and being cognizant of normal and unexpected changes.