> Three years ago, Wal-Mart summarily announced that it would no longer share its sales data with outside companies, like Information Resources Inc. and ACNielsen, which had paid Wal-Mart for the information and then sold it to other retailers.
Anyone have more information on these types of data feeds? What PII is usually in them? They're an opaque, behind-the-scenes practice that I'd like to know more about. Wal-Mart may not have given them anything for a decade, but I'm sure tons of other retailers do.
It really depends. There are a lot of dynamics at play, and depending on the source of the transaction data it may or may not have identity data tied to it, or be in a format that would easily be identifiable when combined with other data sources. Some sources would consider identity data itself as a competitive advantage, and only sell anonymous or aggregate data to others. Other sources would have no value out of withholding the identity data, and would capitalize on the opportunity to include it in their feed for a premium.
Feel free to email me if you want to learn more (email in profile). I'm not an expert on consumer data, but used to source business-related data feeds in a past life and got a glimpse of the consumer data side of things.
Zero PII. On occasion there are loyalty programs and IDs but there is no link back to the consumer. Companies like IRI and Nielsen recruit their own panelists and build models of the consumer that they then project into census style data.
I did some assignments for what I believe was ACNielsen when I was in college. I'd get paid about €10 an hour to stand outside an Aldi store asking every customer that came out for their receipt. (Aldi also doesn't share its sales figures.) I'd also have to note every customer that didn't give their receipt. This gave ACNielsen information on the sales for that particular store. They, of course, did this at multiple locations and sold the gathered data to interested parties.
> By its own count, Wal-Mart has 460 terabytes of data stored on Teradata mainframes, made by NCR, at its Bentonville headquarters. To put that in perspective, the Internet has less than half as much data, according to experts.
~15 years later, 460TB are about 10.000 euros in cheap over-the-counter hard disks.
I find this statistic dubious at best, given that by 2004 the internet had hundreds of millions of users. Even if we're generous and assume they mean public data on the internet, that'd be less than 1MB of data per person.
How many people were passive consumers then though? Also, hough much lighter weight was the content? This is before smartphone videos were being uploaded after all, 2004 was still an era where you might have a page on your personal site for your dog and everything was static html + css.
I guess its subjective by what they mean by "the Internet has less than half as much data". I imagine what existed on file sharing sites was more (if that counts). As of 2007, I had 1 TB of specific types of content, which would have been a tiny tiny fraction of what was on sites like morpheus/kazaa at the time in 2004.
As a very rough ballpark, hard drive cost was around $1/GB, putting a single copy of everything around a half million dollars. Honestly quite a bit less then I expected for back then. Of course there are tons of other costs, but still not unreasonable for a Walmart size company.
Purchase data sharing with 3rd parties and facial recognition,how rampant are they? And do you know anything about how they track cell phone's wifi mac as a customer goes around the store?
Mining data is already too "sketch" for most people that are walking through your store, and you do not inform any of them that you are doing this to them.
Don't come to a public forum to run PR. Your username makes it obvious that you're already trying to play as if you're subverting something, but you are merely fulfilling a trope.
Don't resurrect a 14 year-old article simply to hijack any subsequent discussion for yourself.
Quit your job and join something positive for once.
Not really, many of the stores are $1M+, so a $100 purchase of groceries could just be considered an outlier. The bigger changes you'll find is when word of mouth spreads about a specific product. Ex. If Tide came out with a new version of the pods, sales will be slow (assuming no advertising) for the first few weeks, but it acts more like an exponential curve if the product catches on.
Good! The Dark pattern for the store layout is working :P
Walmart has spent a shit ton of money on their analytics systems including open source and Teradata. They have agreements with various cell companies for geo-fenced info. So they have location analytics and your "market basket" data to slice and dice. Yet the results are mediocre and Amazon is kicking their ass with strategy. They have no loyalty program (Sam's has membership) to provide compelling packaged deals to shoppers, they have their Every Day Low Price pretense but check Neighborhood Market prices that vary wildly from super-center prices especially for grocery. In reality, WMT is a quasi-monopoly that is dying under its own obesity - it needs to split up. eCommerce at $16B is a joke considering it doesnt make any profit and has take billions in overpriced investment. The management, especially in tech, barely see projects through and has lately removed several long-time employees for "fresh blood". Add turnover in SV and other places, it is chaos.
> Walmart has spent a shit ton of money on their analytics systems including open source and Teradata.
This is true! More specifically, we're using Teradata and Hadoop.
> They have agreements with various cell companies for geo-fenced info.
First that I've hear of it. If we actually did have this, it would be a major invasion of privacy. There was a pilot to be able to track expensive inventory (Like TVs) around the store to prevent theft, but it was shutdown due to privacy concerns.
> Yet the results are mediocre and Amazon is kicking their ass with strategy.
Go look at our year-over-year for e-commerce and you'll see that we're actually doing very well.
> They have no loyalty program (Sam's has membership) to provide compelling packaged deals to shoppers, they have their Every Day Low Price pretense but check Neighborhood Market prices that vary wildly from super-center prices especially for grocery.
This is where it gets interesting. Most of the variation you'll see is taxes from zone to zone. As well as logistical challenges for certain locations. Given actual places, I could get a breakdown of costs, but alas.
> In reality, WMT is a quasi-monopoly that is dying under its own obesity - it needs to split up. eCommerce at $16B is a joke considering it doesnt make any profit and has take billions in overpriced investment.
Do you watch out our financial calls?
> The management, especially in tech, barely see projects through and has lately removed several long-time employees for "fresh blood". Add turnover in SV and other places, it is chaos.
Management is terrible, I'll agree to that. But HR isn't ageist, and we have thousands of associates that are > 25 year tenure with Walmart and hundreds of those over 35 year tenure.
Turnover in SV is because we meet the bare minimum to compete in the market, not because of management, otherwise you'd see the similar numbers to our offices in Reston, Virginia.
23 comments
[ 3.0 ms ] story [ 54.7 ms ] threadAnyone have more information on these types of data feeds? What PII is usually in them? They're an opaque, behind-the-scenes practice that I'd like to know more about. Wal-Mart may not have given them anything for a decade, but I'm sure tons of other retailers do.
It really depends. There are a lot of dynamics at play, and depending on the source of the transaction data it may or may not have identity data tied to it, or be in a format that would easily be identifiable when combined with other data sources. Some sources would consider identity data itself as a competitive advantage, and only sell anonymous or aggregate data to others. Other sources would have no value out of withholding the identity data, and would capitalize on the opportunity to include it in their feed for a premium.
Feel free to email me if you want to learn more (email in profile). I'm not an expert on consumer data, but used to source business-related data feeds in a past life and got a glimpse of the consumer data side of things.
~15 years later, 460TB are about 10.000 euros in cheap over-the-counter hard disks.
or 70k € in SSDs.
http://www.mkomo.com/cost-per-gigabyte
If I had to place a number, it's well over the ExB barrier.
I'm wanting to shed some light on our practices, as we aren't as sketch as most believe.
Throw-away account for obvious reasosns.
Facial Recognition: Very, Very Little. Only in one store at the moment, and it's designed for Asset Protection and tracking a thief around the store.
As for tracking MAC addresses, we track Enterprise devices that associates use, but not customers.
Don't come to a public forum to run PR. Your username makes it obvious that you're already trying to play as if you're subverting something, but you are merely fulfilling a trope.
Don't resurrect a 14 year-old article simply to hijack any subsequent discussion for yourself.
Quit your job and join something positive for once.
I feel like everything I buy has gotten more variety since I started shopping there.
Also, how did you find out about HN?
Good! The Dark pattern for the store layout is working :P
Friend referred me.
> Walmart has spent a shit ton of money on their analytics systems including open source and Teradata.
This is true! More specifically, we're using Teradata and Hadoop.
> They have agreements with various cell companies for geo-fenced info.
First that I've hear of it. If we actually did have this, it would be a major invasion of privacy. There was a pilot to be able to track expensive inventory (Like TVs) around the store to prevent theft, but it was shutdown due to privacy concerns.
> Yet the results are mediocre and Amazon is kicking their ass with strategy.
Go look at our year-over-year for e-commerce and you'll see that we're actually doing very well.
> They have no loyalty program (Sam's has membership) to provide compelling packaged deals to shoppers, they have their Every Day Low Price pretense but check Neighborhood Market prices that vary wildly from super-center prices especially for grocery.
This is where it gets interesting. Most of the variation you'll see is taxes from zone to zone. As well as logistical challenges for certain locations. Given actual places, I could get a breakdown of costs, but alas.
> In reality, WMT is a quasi-monopoly that is dying under its own obesity - it needs to split up. eCommerce at $16B is a joke considering it doesnt make any profit and has take billions in overpriced investment.
Do you watch out our financial calls?
> The management, especially in tech, barely see projects through and has lately removed several long-time employees for "fresh blood". Add turnover in SV and other places, it is chaos.
Management is terrible, I'll agree to that. But HR isn't ageist, and we have thousands of associates that are > 25 year tenure with Walmart and hundreds of those over 35 year tenure.
Turnover in SV is because we meet the bare minimum to compete in the market, not because of management, otherwise you'd see the similar numbers to our offices in Reston, Virginia.
Thanks!