NFL Draft Value Chart Controversy

The baseball season has officially started, NHL playoffs are in full swing (go caps!), and the NBA regular season is coming to an end. Despite the vast amount of live action, sports coverage today is focused on the upcoming NFL draft. This is simply fascinating and shows why the NFL is the number one sport in the US with a $30bn+ market.

With such a dominance in the sports market, it is remarkable that the league is not more quantitative in its reasoning. Take this week’s NFL draft, for example. Prevailing wisdom harks back to Jimmy Johnson’s days with the Dallas Cowboys, where he developed a draft-pick value chart that was (and still is) used to help the team evaluate the effectiveness of trading up or down in the draft. Each draft pick is assigned a value so that general managers can decide whether it is a good deal to trade the #3 pick (2200 points) for picks in rounds two (270-580 points) and three (116-265 points). Was this draft-pick value chart developed based on data? No. It was a qualitative guesstimate that, as others have discussed, has serious flaws that we explore a bit more here.

To be fair, one of the more challenging issues with quantitatively assessing the value of different picks in the draft is that it is inherently difficult to assess players in different positions. How do you compare the value of a quarterback to the value of a safety? Fortunately, pro-football-reference has developed a metric that quantifies the value of a football player over the course of their career, regardless of their position. While it may be interesting to develop a metric like this of our own, this metric will do for the present purposes.

We’ll start off the analysis with the hypothesis that, over time, teams improve their ability to draft high-quality NFL players earlier in the draft. To see if this hypothesis is correct, we can explore the career value of a player chosen at a particular point in the draft using the value metric from pro-football-reference.com, normalized by the average value of all picks in the same year (the NFL rules have changed a bit over the years, after all). If the draft market is becoming more efficient at predicting the future value of players, we should see these curves become more skewed toward early picks over time.

We plotted the average value of a pick for each decade on the same graph to see how the distribution changed over time. Remarkably, the differences between decades are minute—so much so that we plot them with transparency so you can see the similarities—suggesting that the draft market has become no more efficient at identifying high-quality talent at early points in the draft.

Let’s take a step back and just admire how close the value of NFL draft picks are over time. Its amazing that there exists such transparent empirical evidence of the value of a draft pick that NFL teams must be using a similar methodology for evaluating picks, right? Wrong.

Similar to the work done by Kevin Meers with Harvard Sports Analytics, lets contrast our historical data with the value chart developed by Jimmy Johnson. In this graph, we analyzed the value of the first 224 (= 7 rounds * 32 teams) NFL draft picks from 1960 - 2010. The black dots represent the average value of that pick over the last 50 years. While the red-line represents a regression line of best fit of y = -201.67 * ln(x) + 1161 (download tabulated version here). Finally, the blue line shows the Jimmy Johnson draft value chart based on a qualitative guesstimate.

This is bad, I mean really bad. Teams that still use this old draft value chart are getting absolutely fleeced by more savvy GMs in the league. For early picks in the first and second round, the old draft value chart overvalues those picks while later rounds are substantially undervalued (see inset). A key draft strategy against teams in the stone age is to trade down while you are in the first two rounds and trade up if you are in later rounds (Kevin Meers claims that you should never trade up!).

There are a couple of major points that our new draft value chart fails to take into account. First off, the value chart assumes that all picks add value to your team which isn’t necessarily the case. The grey dashed line on the graph represents the average value of every player drafted over the last 50 years, 270. One can argue that drafting a player for your team only helps if it raises the average value of your team. For instance, drafting a player of value 300 that replaces a player of worth 100 helps your team by a net of 200. Whereas two players of worth 150 that replace players of worth 100 and 125 only helps your team by a net of 75. Secondly, our analysis does not incorporate the average value of an undrafted free agent. An undrafted free agent may have a similar value to a 7th round draft selection and is “free” for a team. Thereby reducing the value of later picks.

In short, the debate around the Jimmy Johnson draft value chart (and others) is that teams need to do a custom analysis based on their existing talent to evaluate draft picks in later rounds. When selecting a player in the draft, they should analyze the net gain in value at that position on their roster. For instance, the Washington Redskins were willing to sell the future for the opportunity to select Robert Griffin III in order to replace the human heart attack Rex Grossman. RGIII is simply a lot more valuable to the Redskins than just about any other NFL team.