WMDs: HBDIIATD

Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, by Cathy O’Neil

You know that feeling when you know something, but then you realize you didn’t really “know” it? Like you know Donald Trump is an idiot, but then he says you just need to rake the forest in order to prevent catastrophic wildfires? That feeling.

Well, I knew I didn’t like it when I scouted a pair of shoes on the Hudson’s Bay website, and the next day, while I was Googling to find a nearby coffee shop I saw, not just an ad for the same shoes, but the exact same picture I’d been looking at. It occurred to me then that being stalked by a gigantic, morally-questionable search engine was a new kind of creepy. (Even creepier … Google knows where you live, which means some data weirdo at Google knows it, too.)

But this is only the beginning of the bad stuff. Just as I was thinking that extrapolations of semi-anonymized data was less horrible than the way Facebook tracks my every move, Cathy O’Neil paints a compelling picture of how this data can reinforce societal biases and prejudices if it is used by people who don’t understand the basic principles of mathematical modelling and corrective feedback loops. And  it turns  out these mathematically ignorant entities are actually extremely influential organizations that decide things like who gets lighter prison sentences, who is accepted into college, who lands a good job, who is awarded credit to purchase a home, and who can get insurance.  What happens, in poorly designed models, is that proxies are selected to act as stand-ins for actual information. For example, a popular proxy for predicting good behaviour is a good credit rating. Even in situations where a credit rating makes no proxy sense at all, such as influencing the rate you might have to pay for auto insurance, it is used because it’s easy to obtain as a data point, which makes it perfect for a lazy data modeler. What this means, for one, is that people who are more likely to be poor are targeted with higher insurance rates, impacting their financial viability even more.

Poorly constructed models that detrimentally affect the most vulnerable members of society are bad, but what’s ultimately disturbing is how much data is collected and where it’s collected from. Cell phones, website searches, online ordering, social media, and on and on and on. Here’s a thing. I was taking a quiz on Facebook the other day (yes, yes, I know what you’re thinking … but it wasn’t one of the sites that forces you to “log on” through Facebook, which I 100% refuse to do!). This particular quiz offered to guess the name of my cat, which sounded fun. So I answered a bunch of questions about my cat’s behaviour. And suddenly it was asking me where I like to go on vacation and what kinds of books I like to read. Whaaat? So they don’t even feel the need to be subtle about it anymore! (Jeff is looking forward to when the vacation ads start showing up.) And don’t even get me started on those “use your birthday month and the first letter of your last name to figure out your stripper name” games. Good god, it might just as well say “give me all your personal data  now and save us both a lot of time”.

So what’s to be done? I’m certainly not going to become a Luddite – I love my technology (when it works). I’m happy to let Google direct me “home” when I’ve been out driving somewhere new, and I’ll still answer FB questions about my pet (but maybe not about my personal habits). I seriously doubt we can ever fix the corporate collection of insane amounts of personal data, but we can start lobbying for to laws that set limits on how that data can be used. For example, there are already laws on the book that companies cannot discriminate against potential job candidates, and perhaps those laws need to be extended to the data models that pre-select the resumes for your short list. There are several more useful ideas in the book, and it’s worth reading just for that.

Rating: Borrow it. Although data nerds might like to buy their own copy.