Previous
Chapter 3: Arms Race
Themes
Key
LitCharts assigns a color and icon to each theme in Weapons of Math Destruction, which you can use to track the themes throughout the work.
Humanity vs. Technology
Discrimination in Algorithms
Fairness vs. Efficiency
Data, Transparency, and U.S. Democracy
Summary
Analysis
While working as a data scientist at an advertising start-up, O’Neil and her team hosted a visit from a venture capitalist who gave a speech describing the “brilliant future” of targeted online advertisements. Consumers would contribute data through their online behavior, and advertisers would target them with “valuable information” that would help them shop and live better. No longer, he joked, would web users find themselves assaulted by ads from the University of Phoenix. Though O’Neil’s coworkers laughed at the joke, she was perturbed—the internet was already preying on lower-income people with “the bait of upward mobility.”
While online advertising targeted at individual users is now common, O’Neil suggests that this doesn’t mean it isn’t predatory—and it’s especially dangerous for low-income people who are already at a disadvantage. The University of Phoenix is a for-profit college, which means that its goal is to make money rather than invest back into resources for students. So, baiting low-income people with the promise of “upward mobility” (increased class status) if they attend a college like this could put them at an even greater disadvantage, because they may not end up receiving the quality of education that they’re looking for. In this way, WMDs like targeted advertisements are deepening social inequality slowly but steadily—even as their creators joke about them.
Themes
O’Neil alleges that the internet isn’t the equalizing, democratizing force it promised to be. Instead, as Big Data and tech companies have learned more about individual users, they’ve created rankings and categorizations of people that target their vulnerabilities, feed them predatory ads, and exploit their finances. Online, for-profit universities that charge exorbitant fees advertise and direct their recruiters to people who are on government assistance, who’ve been recently incarcerated, or who are otherwise financially or socially vulnerable. “Vulnerability is worth gold” to advertisers and the makers of WMDs—and recruiters at places like ITT Technical Institute are told to “Find Out Where Their Pain Is” when they’re engaged in recruiting.
Online advertisers know that they’re deepening the gulf between the rich and the poor. But because their algorithms are aimed at maximizing exposure and profit, they don’t take fairness into account. In fact, they exploit existing social problems like economic inequality through their algorithms, trapping desperate people in dangerous feedback loops that actually make it harder for them to advance and improve their lives.
Themes
Big tech companies like Google and Facebook allow these for-profit universities to segment their target populations and advertise to them directly. The ad campaign then runs competing ads against one another to see which bring in the most users. This method is based on A/B testing. An example of A/B testing is credit card offer mailings. Credit card companies send out massive mailings to hundreds of millions of people, so that even if only a fraction of a percentage of people respond to the ad, the company is still raking in many customers. Moving these campaigns online, though, allows advertisers to track feedback much more closely and zero in on their most effective messaging.
By casting wide nets, tech giants and advertisers make sure that virtually no one is spared from their algorithms’ influence. Again, A/B testing is an example of a WMD—and it’s having devastating consequences on society by roping vulnerable people into predatory schemes driven by unregulated algorithms and data.
Themes
Now, data-crunching machines can learn as they operate. This phenomenon, called machine learning, has been studied since the 1960s, when language scientists began teaching computers to read. As the internet has grown and expanded, users have given machines “quadrillions of words” about the way humans live—data machines now have one of the biggest sample sets of raw learning material ever. As advertising programs learn more and more words, they can probe users for deeper patterns.
Computers can “learn” to a certain degree—but they’re mainly learning to target and exploit people. This illustrates how pivoting the economy toward data, algorithms, and maximizing profits at any cost is hurting rather than helping society.
Themes
Get the entire Weapons of Math Destruction LitChart as a printable PDF.
Another kind of predatory online targeting called “lead generation” uses falsified information and misleading phrases and imagery to come up with lists of prospects that can then be sold to places like for-profit universities, who then use the user information to recruit people. Fake job postings and misleading promises of routes to food stamps or Medicaid coverage target vulnerable people, gather their information, and send it to recruiters who then make follow-up calls and emails. For-profit colleges also use financial aid questionnaires on websites like the College Board to advertise themselves to those most in need of financial assistance to attend school. Then, they arrange free online resume-writing workshops in order to harvest students’ data and essentially “stalk” them with cold calls, emails, and more ads.
Here, O’Neil shows how sophisticated but predatory advertisements exist only to gather data that can then be used to power even more predatory algorithms. By specifically targeting low-income people who are eager to better their social standing, these advertisements actually deepen social inequality by influencing already struggling people to spend money on degrees that may not benefit them. So, these algorithms are essentially taking money from vulnerable people in order to maximize the company’s profits.
Themes
WMDs are damaging to peoples’ lives. But in the case of for-profit colleges and online advertising, the damage doesn’t begin until students targeted by advertising take out loans to pay their tuition and fees. Even though the Higher Education Act of 1965 states that colleges cannot get more than 90 percent of their funding from federal aid, for-profit colleges exploit this stipulation by helping poor and vulnerable students line up massive loans. By creating highly refined WMDs that target the poorest 40 percent of the U.S. population, these colleges take advantage of desperate people, take their money, and hand out essentially worthless degrees that don’t always enable them to get better jobs. In doing so, they create a damaging, destructive feedback loop.
Again, these complicated WMDs are further dividing an already stratified American society. They’re promising to help people improve their lives—but really, they’re only keeping poor people poor while making rich people (and corporations) even richer. This kind of feedback loop isn’t sustainable—if these WMDs are left to operate on their own without any regulation, they will do irreparable damage to individual lives and society more broadly.
Themes
Loan companies, too, operate WMDs in order to target and draw in customers. Some of these companies are legitimate—but many charge exorbitant interest rates, sell their customers’ data, or even hack their bank accounts. As a result, lawmakers are pushing for legislation that will govern the growing market for personal data. Yet many “effective and nefarious” WMDs, O’Neil writes, have no problem creating workarounds that will allow them to study our behavior online and off, carefully and relentlessly.
WMDs have been growing unchecked and unregulated for so long that it’s now extremely difficult for even government policy to reel them in. The algorithms are very sophisticated—and there’s always a new method of gathering and using data that works around attempts to protect society’s most vulnerable.
Themes
