Standardized shame on test results is a staple of every NFL draft season.
Prospects take the timed 50-question proficiency test known as Wonderlic each February during the NFL Scouting Combine. NFL Commissioner Roger Goodell told teams in 2012 that the results are supposed to be confidential, but like clockwork, they leak every year, prompting staff, media and fans to apply a reflection of Goldilocks level for each score reported.
Alabama quarterback Tua Tagovailoa would have scored the lowest of the qualifying draft quarters, but Albert Breer of Sports Illustrated reported on Saturday that the disclosed Tagovailoa score was incorrect. The initial report of Tagovailoa’s results prompted ESPN’s Dan Orlovsky to note in a deleted tweet on Friday since the rumor score “bothered me” more … than [Tagovailoa’s] injuries. “
Richard Sherman did not respond directly to Orlovsky, but the 49ers’ cornerback criticized the usefulness of the Wonderlic in a tweet Friday night.
Lol if ppl only knew how the guys really treat wonders. Lol, nobody cares! https://t.co/0WU52F2yQm
– Richard Sherman (@ RSherman_25) April 17, 2020
Orlovsky, to his credit, deleted the tweet questioning the score reported by Tagovailoa and said it was wrong to tweet it in the first place. But the analyst, who played seven seasons as an NFL quarterback, admitted that he was “Always sort” how Tagovailoa’s football IQ and decision making were so good when his Wonderlic score was so low.
There should be nothing to settle for Orlovsky or anyone else. Sherman is right: Wonderlic scores have little to do with field success.
Listen and subscribe to the 49ers Insider Podcast:
Study after study, the Wonderlic has virtually no predictive ability for careers in the NFL. Joseph Stromberg noted in a 2014 Vox article that several studies indicate that Wonderlic scores have no correlation with the success of the NFL. One pair of studies even found a slight negative correlation, meaning that lower scores were a better predictor of player success than high scores.
Michael Callans, a former Wonderlic executive, argued in 2012 that studies are inaccurate because they are based on leaked scores which, according to Callans, are often inaccurate. Given that Tagovailoa’s initial report and the scores of the other quarters this year were disputed in one day, this is an important point to consider when considering the usefulness of Wonderlic.
The same goes for the fact that Wonderlic has also been criticized for having an integrated racial bias, as many standardized tests have done. As writer Stephanie McCarroll observed in a Twitter thread on Friday, a 2012 study found that teams had spotted white prospects almost 15 points earlier if they had a higher than fair Wonderlic score six places for black players.
McCarroll also highlighted the Unanimous Supreme Court Decision of 1971 that the use of Wonderlic and other standardized tests that are not “clearly a reasonable measure of job performance” as a “screening force” when hired violated the Civil Rights Act of 1964, which prohibited discrimination based on race, color, religion, sex and national origin. The Wonderlic is not the only piece of the NFL evaluation puzzle, but this context raises the question of why it is a piece.
The NFL clearly recognized the inherent flaws in Wonderlic in 2013 when it instituted the Player Assessment Tool (PAT) to use alongside it. The NFL said in a 2013 memo that PAT “measures a wide range of skills”. Cyrus Mehri, a lawyer and co-founder of the Fritz Pollard Alliance, told Jarrett Bell of USA Today Sports that year that PAT “is leveling the playing field from a socio-economic perspective.”
However, the PAT scores do not seem to leak with the same frequency as the Wonderlic results.
[RELATED:[RELATED:[ENRELATION:[RELATED:Why 49ers should sign ex-Bears TE Burton, pair with Kittle]
Sherman and the other players who joined are right to extract the Wonderlic test utility. The test itself is a bygone tradition, which has spawned another bygone tradition of dedicating column inches and airtime to interpreting its results without paying too much attention to its inherent flaws.
Perhaps the 2021 project will be the year when analysts stop falling into the same trap.