Wherever we travel in our brave new digital world, we leave behind a rich and fertile trail of breadcrumbs of data. Our mobile phones continually tell the phone company where we are, our credit cards reveal how much we spent, and where, and on what — and so on.
Welcome to the world of ‘big data’, where the chain store Target has data-mined its way into my favourite organ, the uterus — and can now accurately predict when your baby will be born.
Back in the old days, a bookstore might slowly build up a base of loyal customers by offering discounts, and mailing out a brochure every month or so. But ‘big data’ is very different. So let’s look at the three ‘V’s: volume, velocity, and variety.
First, volume. In the USA, Walmart, the giant chain store, harvests some 2.5 million gigabytes of data each hour — just from customer transactions.
Second, velocity. Today, we can analyse the data in real time. For example, you can quite accurately predict the sales figures of big shopping centres by monitoring the activity of mobile phones in the parking lot — even before customers have opened their wallets.
And thirdly, variety. The data can be GPS location data from your mobile phone, or it can be the entire contents of your address book with the phone numbers and birthdays of all your friends and their families. Big data also includes everything you have ever done on Facebook, Twitter and all the other social media — and let’s not forget the good old credit card.
Back in 2002, Target in the USA wanted to know which of its shoppers had just become pregnant — because it wanted more customers. You see, shoppers tend to shop at the same place that they’ve always gone to — and it’s hard to get them to shift. But when women get pregnant, it’s a whole new world — and the pregnant woman might be lured into changing to a new shop.
So Target hired Andrew Pole (who has master’s degrees in both statistics and economics) to see if he could data-mine his way into freshly pregnant uteruses. His starting point was the so-called ‘guest ID number’. This is a number that Target assigns to each customer, and is linked to their name or email address or credit card.
It soon became clear that women (who had voluntarily put themselves on the Target Baby Registry) were buying some 25 products around the beginning of their second trimester. These included mineral supplements such as calcium, magnesium and zinc, large quantities of unscented lotions, and so on. And if they bought a little bright blue rug, the new baby was probably a boy.
And when they started buying scent-free soap, hand sanitisers, washcloths, a handbag large enough to carry a few nappies and extra-large bags of cotton balls — well obviously, they were getting close to their delivery date. Indeed, Target found it could predict their delivery date with some 83 per cent accuracy — just from their shopping habits.
So Target began sending discount coupons for various baby items to customers, according to how they rated on Target’s own “pregnancy prediction algorithm”.
But then it got messy.
An angry father walked into a Target store just outside of Minneapolis, clutching a brochure that had just come through the mail, demanding to speak to the manager. Angrily he said: “My daughter got this in the mail! She’s still in high school, and you’re sending her coupons for baby clothes and cribs? Are you trying to encourage her to get pregnant?”
Sure enough, the brochure did carry the daughter’s name and address, and had advertisements only for maternity clothing, nursery furniture, and so on. The manager apologised.
A few days later the manager called the father to apologise again. But this time, the father was a little embarrassed. He said: “It turns out there’s been some activities in my house I haven’t been completely aware of. She’s due in August. I owe you an apology.”
Now some people like their privacy. Some Target customers were annoyed by receiving brochures packed only with baby merchandise, especially when they had not told anyone they were pregnant. So Target produced new brochures that had some non-baby merchandise sprinkled around the baby goodies, so the newly pregnant mothers didn’t realise they had been data-mined.
Target is happy. Their revenue jumped from $44 billion in 2002, to $67 billion in 2010 — helped mightily by the fact that with regard to pregnancy, Target knows, before it shows.
Now it’s early days for big data. But one step might be to have more transparency and openness about what we (the customer) are giving away — and what we get back in return.
Original post by ABC science