Words into Touchdowns

Predicting success of NFL quarterbacks by their speech using research to identify terrorists. [Link]

That's the assertion, mission, and business plan of an Ohio-based company called Achievement Metrics. It analyzes the speech of star college players, looking for traits such as "conceptual complexity," "need for power," and "deliberativeness." It compares similar players and correlates these traits with future performance. College wide receivers whose speech shows low levels of distrust, for example, have a greater probability of becoming Pro Bowlers than their less-trusting counterparts.

That may sound batty, but it's closely related to established social science. For decades, political scientists have counted the words and analyzed the grammar of political speeches in order to understand a leader's state of mind. In recent years, this process has become both automated and refined by algorithms and data crunching. Achievement Metrics has grown out of a company called Social Science Automation that does most of its work for the U.S. government. It is attempting to bring text analysis to nontraditional areas, like the CEO's corner office and, yes, the scouting of pro athletes.

The analysts at SSA don't say a lot publicly, as much of what they do is classified. You can get a flavor of their work, though, in a 2005 paper titled "The Distinctive Language of Terrorists." The paper discusses the technique of remote assessment, which involves gathering samples of speech and determining whether someone is likely to be a terrorist. The paper demonstrates the validity of remote assessment by analyzing the speech of political leaders and known terrorists and scoring them in such categories as self-confidence, task orientation, and distrust of others. Remote assessment sorted out the bad guys from the good with impressive accuracy—although, like all probabilistic models, it was not perfect. John Kerry, for example, was classified as a terrorist. (The speech that the study sampled came from the months in the 2004 campaign when he went negative.)

The speech cues that SSA considers important are not the obvious ones—"I want to kill all Americans!"—but rather instances of words like I and we, as well as the use of qualifiers—I maybe think I want to kill some Americans, perhaps. While that's a crude presentation of the work, you get the idea: Subtle, almost undetectable patterns can be analyzed to determine how likely a political figure is to engage in terrorist activity. Roger Hall, a consulting psychologist for SSA and CEO of Achievement Metrics, claims, "If you give us unidentified speech text, we can distinguish terrorists with 90 percent accuracy." Now you understand why they don't talk publicly about their work. Is the leader of that new group in Afghanistan blowing hot air or is he trouble? That's the kind of question the intelligence community is interested in.

Back to football. About five years ago, SSA analyst Steven Hofmann and a colleague started thinking about how their work could be applied to the NFL. At the time, the league was having trouble with a rash of arrests and suspensions, and coaches and scouts wanted to know who they could count on to stay on the right side of the law. Hofmann began collecting interviews with college stars—he needs only about two pages of text to do his analysis.