RFID and Retail: From Big Data to "Huge" Data?

Last week we investigated the role of IT in the supply chain. We read about ERP systems and, of course, lurking at the edge was the specter of "Big data" and the transformation that timely, large scale analytics can bring to inventory planning and route optimization.

This week we turn our attention to the technologies of the future, including the resurgence of RFID. The term "resurgence" is appropriate because, in the words of Barb Darrow at GigaOm, "RFID was one of those next-big things in IT that didn’t really pan out because the readers were expensive and the read-rate failure was high. (The reader had to be really close to the tag). But with stakes getting higher, that calculus might change as retailers revisit the idea of tagging merchandise to enable easy payment and inventory monitoring."[1] 

There is a fruitful intersection point between these two topics. In an analysis by RFID and GPS company ThingMagic, RFID might bring to retail the kind of agile real-time analysis of customer preference and demand that has traditionally been the province of internet retailers. They write, "A key to optimizing sales and margins is making near real-time decisions about merchandising, assortments and promotions. It’s easier to do this on the web because of data available from such things as what items people are clicking on, search queries, etc. Retailers can change what gets promoted on the fly with this information."[2]

They point to technology such as RFID-embedded clothing hangers that can allow retailers to track precisely which items are lifted off the rack, which ones are taken to the fitting room, and which of these trips result in sales and which result in abandoned merchandise. (See image)

RFID-enabled hangers allow retailers to track customer interaction with products (ubergizmo.com)

These technologies can be combined dynamically with mobile apps designed to assist shoppers. From the GigaOm report: "A consumer could use her smartphone to find a leaf blower or washing machine at the store, scan it with the phone for payment and schlep it to the car — all without waiting in a checkout line (or an unpleasant encounter with store security)."[1]

It is easy to imagine the intersection between these two data streams. Retailers could perform real-time, dynamic hypothesis testing on alternative promotions, the way Google constantly tweaks their advertising algorithms (essentially turning every user interaction into a massive training set). Retailers could, for instance, detect that a customer had lifted a certain jacket or blouse and immediately transmit an advertisement custom-tailored based on the user's Facebook "likes." The algorithm could automatically determine which pitch was most effective at turning interest into sale and roll the strategy out nationally, all without the intervention of an individual.

However, there are some real challenges associated with this otherwise utopian picture. Jeff Bertolucci at InformationWeek notes, "The benefits of RFID are real, but the technology also poses problem for organizations. For instance, many retailers that adopt RFID for inventory management must find ways to ingest, analyze, and archive huge volumes of new data."[3] Given the potential headaches uncovered in last week's ERP readings, it is easy to imagine the frustration that might come from a retailer trying to integrate an enormous stream of RFID data into a pre-existing solution designed to track point of sale (POS) data and facilitate inventory management.

There are even more technical problems, as well. In an article in RFID Journal, Stephen Miles cites low-level concerns that many managers might never consider. "First, there is the issue of a common registry and identifier namespace. While the Internet employs a common Domain Name Server (DNS) and unique Internet Protocol (IP) addresses, companies often have difficulty agreeing on a common registry for items designated with an Electronic Product Code (EPC). It does not help that, in the evolution of bar-code systems, portions of the retail supply chain retain a variety of legacy coding schemes"[4]

We must consider, therefore, whether the convergence of these technologies might bring the transition from big data to "huge" data. The big-box retailer of the future has the potential to communicate constantly with shoppers, monitor customer reactions to products (via Twitter and Facebook traffic) in real-time, gauge the turnover rate between initial interest and ultimate sales. This same retailer will also have to manage the integration between social media data, mobile phone promotion feedback, RFID data, POS data, legacy enterprise systems, modern ERP systems, and the computational demands arising from processing and mining such gargantuan and "high-velocity" data.(NOTE 1)

The question to consider, then, is where existing enterprises (public, private, or non profit) can move strategically at the intersection point between these factors. The most obvious link is with healthcare. Can, and should, a hospital employ RFID for all of their prescriptions, using algorithms to make sure that beyond simply prescribing the right medicine for the right patient, the processes in place actually physically move the correct prescription to the correct patient at the correct time? Could mobile phone applications perform hypothesis testing on alternative fund-raising campaigns for a symphony orchestra in real-time following a concert series, integrating known behavior patterns from Facebook and Twitter? 

What changes will come to your preferred industry from the convergence of these technologies?

NOTE 1: Velocity refers to one of the "three Vs" of big data. Typically, big data refers to data sets that have a high level of volume, variety, and velocity in some combination. High velocity data is a data stream that is constantly updated and changed. Therefore, the U.S. census could be thought of as a data set with high volume but low velocity (It is updated only once every few years, after all.)

[1] Darrow, Barb. "Big Data is a big deal - and getting bigger - for retailers." GigaOm. http://gigaom.com/2011/11/23/big-data-is-a-big-deal-and-getting-bigger-for-retailers/
[2] Lynch, Ken. "RFID, Big Data, and retail." ThingMagic. http://rfid.thingmagic.com/rfid-blog/bid/80385/RFID-Big-Data-and-Retail
[3] Bertolucci, Jeff. "RFID needs Big Data Tools." InformationWeek. http://www.informationweek.com/big-data/news/big-data-analytics/rfid-needs-big-data-tools/240007140