Weapons of Math Destruction: Chapter 9: No Safe Zone Summary & Analysis

In 1896, a German statistician named Frederick Hoffman who worked for the Prudential Life Insurance Company created a WMD. According to O’Neil, he published a 330-page report claiming that the lives of Black Americans were so precarious that “the entire race was uninsurable.” Like many other WMDs, Hoffman’s analysis was statistically flawed, racist, and unfortunately widespread.

For decades to come, insurers would cling to the idea that certain groups of people simply weren’t worth insuring. Bankers and insurance companies would start delineating neighborhoods that they wouldn’t invest in—this practice was called “redlining,” and it wasn’t outlawed until 1968. Yet redlining is still pervasive in U.S. society, and it’s coded into contemporary WMDs that use flawed statistics to punish poor people and racial or ethnic minorities.

Many WMDs that perpetuate redlining are found in the insurance sector. Insurance grew out of the predictive field of actuarial science. In the late 1600s, mathematicians discovered that by comparing mortality rates of different people within a given community, they could calculate probable arcs of people’s lives. Over the next several centuries, these predictions gave rise to the insurance business.

In today’s world, more data about people’s lives is available than ever before. Rather than making insurance predictions based on large groups, insurers are getting closer to being able to provide appropriate coverage based on the individual. Insurers use faulty proxies for responsible driving (like zip code and income) to create their own ratings, or e-scores—but because a lot of the information they use is based on credit and capital, insurance continues to work against the poor in many ways. Even drunk driving convictions count less in determining a person’s premium than credit scores. By ripping off desperate, working-class people, these companies make a fortune off good drivers with bad credit scores. And because the factors that go into pricing at major insurers like Allstate aren’t clear, their algorithms constitute WMDs.

In the age of Big Data, insurers can judge us by how we drive in entirely new ways. In 2015, the U.S.’s largest trucking company (Swift Transportation) started installing cameras in long-haul trucks—one pointed at the road, the other at the driver’s face. The goal, according to Swift, was to reduce accidents—around 700 truckers die on the road in the U.S. each year. These fatal accidents are tragic, and they cost trucking companies a lot of insurance money (around $3.5 million per fatal crash). The additional surveillance also had another purpose, though: it let Swift gather a huge stream of data that could be used to optimize profits, compare individual drivers, and identify good performers. 

Now, insurance companies offer regular drivers discounts if they agree to share their driving data through a small telemetric unit (like an airplane’s black box) placed inside the vehicle. This has the potential to help drivers save money—especially younger drivers, who are often costly to insure—but it’s also a big liability for poor or disadvantaged drivers. Driving through a bad neighborhood or providing evidence of a long commute each day might raise a driver’s rate. Eventually, the insurance companies’ promises to focus more on individuals becomes moot, because individual behavior is still being compared to that of others in similar demographics. While these systems are optional now, trackers, O’Neil asserts, will likely become the norm—and people will be punished for not having them rather than rewarded for consenting to them.

Insurance companies, O’Neil predicts, will soon start sorting people into new kinds of groups or “tribes” based on behavior. A decade ago, researchers at a data company called Sense Networks started to analyze cell phone data showing where people went. They could observe dots moving on maps to find similarities between groups of these dots. As the machine began sorting dots into different colors, only the machine knew what those colors meant—even Sense’s cofounder admitted that human observers wouldn’t be able to figure out what the “dots” had in common. This opacity, O’Neil asserts, is dangerous.

In 1943, because U.S. armies and industries needed every soldier or worker they could get, the Internal Revenue Service made a significant change: they gave tax-free status to employer-based health insurance. Within a decade, 65 percent of Americans were insured through their employers. This meant that employers gained a measure of control over their employees’ bodies. Today, employers can offer rewards or impose penalties through “wellness” programs. These programs can create initiatives like “HealthPoints” in which employees accrue “points” by taking a certain number of steps in a day or going for a check-up. In other words, companies can penalize workers who don’t consent to handing over data about their personal health.

Companies like Michelin have set employees goals for things like glucose, cholesterol, and waist size—employees who don’t reach goals in at least three categories (out of several) must pay extra toward their health insurance. And in 2013, CVS announced that if employees didn’t report their levels of body fat, blood sugar, blood pressure, and cholesterol, they’d have to pay $600 a year. This drew public ire, since the company used BMI (or body mass index) as a measurement of health—but BMI scores are “crude numerical prox[ies]” that were originally based around “average” male body types. In other words, their usefulness has been all but debunked.

Even though companies assert that they’re taking  these invasive measures in the name of health, wellness programs don’t lead to lower healthcare spending—and there’s no evidence that they make workers healthier. O’Neil asserts that wellness programs aren’t yet full WMDs, since they’re often quite transparent. But they do show that employers are “overdosing” on employee data, trying to score potential workers and predict their productivity. If companies start creating their own health and productivity models, O’Neil suggests, the industry could very well become a full-fledged WMD.