Yet teams rarely beat the conventional wisdom when they reach for a low-ranked player in the first round, despite all of the incentives to select a reach only when the team is absolutely sure.
The experts have a worse record in a seemingly similar case: when several teams pass on a player talked up by analysts in the days and weeks ahead of the draft. These so-called steals can seem like a tremendous value for the lucky team that lands them. Your team might be praised on talk radio. But these picks don’t tend to turn out unusually well. On average, the steals offer no greater value to teams than one would expect for their draft slot.
Why do the experts offer value over the teams in some cases but not others? Timo Riske, a data scientist for Pro Football Focus who reached similar conclusions in an analysis of drafts since 2013, observed that “when a player is a steal, it means that several teams — maybe even all 32 teams — passed on him,” while a reach could mean that only a single team liked a player more than experts.
“Betting against the N.F.L.’s overall assessment of players is essentially saying your evaluation of a player is better than everyone else” said Ben Robinson, the founder of a website that analyzes the draft. “It’s not sustainable.”
The analysis relies on Pro Football Reference’s Approximate Value metric, an imperfect but useful measure of a player’s contribution to a team’s success. For the remainder of this article, a reach or a steal is a player selected in a slot with an expected approximate value that’s five points higher or lower than one might guess from ESPN’s ranking. This clunky definition helps account for the greater differentiation of players at the top of the draft: Selecting ESPN’s 10th-best player with the first overall pick is considered a reach by this measure, but a player would have to be ranked 170th or lower by ESPN to be considered a reach with the 100th pick.
The analysis also excludes quarterbacks, a position that poses unusual challenges for draft analysis. On one hand, many teams will pass on a highly regarded quarterback if they already have what they view as a good quarterback. On the other hand, teams rank quarterbacks with the position’s unusually high value in mind while many pundit draft boards do not.
A player projected as merely a league-average starting quarterback may be far more valuable than a player projected as, say, an above-average cornerback. But the cornerback could be ranked higher on a draft board. And a team that “reaches” on a quarterback could appear to be vindicated by the data, even if the player does not pan out, simply because of the position's high value.