Introducing DVOA v8.0

Warning! This article is going to get deep into the weeds on how DVOA works. If you’re a longtime FTN reader and new to my DVOA stats, beware! If you’re a longtime Football Outsiders reader and want to know about the latest version of DVOA, read on.

The main goal behind DVOA v8.0 is to make DVOA more predictive for future performance. Secondary goals include making it easier to automate DVOA at some point in the future and making the rules of how DVOA scores certain plays consistent across all 42 seasons.

I made decisions about changes based on three factors:

• Improving the correlation of DVOA with future DVOA and future wins. I looked at both Weeks 1-9 compared to Weeks 10-18 as well as year-over-year correlations.
• Standardizing certain variables to make it easier to automate DVOA in the future.
• Common sense

Using correlation to improve DVOA certainly has its pitfalls. There’s a lot of noise, even in a sample of 10 years, and it’s possible that what looks like improvement isn’t actually improvement. I feel about using correlation the same way Winston Churchill felt about democracy, that it was probably the worst method to use except for all the alternatives. I mostly looked at correlations for 2013-2022 but I did look at 2018-2022 and 2013-2017 separately, and then also looked at 2003-2012, and I feel good about the changes I’ve made because they tend to match the third factor listed above: common sense.

For example, we know that red zone performance tends to regress towards a team’s overall performance. So it no longer makes sense to have bonuses for the red zone that make the red zone more important than the rest of the field. Those bonuses made sense when I originally created DVOA 20 years ago, and I was trying to balance making it both descriptive and predictive. If we want it to be more predictive, we want to remove them.

It’s just a bonus that these changes also make DVOA more descriptive, i.e. it correlates better with actual wins in the past rather than just correlating better with wins in the future.

OK, let’s look at all the changes in DVOA v8.0. Note that these changes are all for team DVOA only. A big overhaul of individual DVOA formulas is on the to do list for next offseason.

• All turnovers are now 75% as strong as they were before.
• Indoor/outdoor adjustments have been split into three categories, with smaller adjustments for retractable roof stadiums. Prior to 2018, I had marked retractable roof stadiums as either indoors or outdoors depending on the roof that week. To make it easier to automate things in the future, retractable roof stadiums are now the same for every game.
• There are no longer indoor/outdoor adjustments for runs or scrambles.
• The red zone bonus has been removed from both offense and defense.
• More plays in the fourth quarter are downweighted because of the current score (i.e. stronger adjustments for blowouts).
• For defense only, plays in the final five minutes of the third quarter are also downweighted in significant blowouts (over 21-point gap).
• Aborted plays (bad handoffs, bad snaps, incomplete lateral passes) are now their own category and do not count as either runs or passes.
• Fake punts and field goals no longer count for offense and defense DVOA. This a common-sense change, as the standard offense and defense are not on the field. In addition, the plan is to eventually create a “non-predictive” version of special teams DVOA that includes blocks, onside kicks, and fakes. I’m removing the fakes now so that we won’t double-count them after we build that.
• Prior to 2018, I manually changed backwards passes so they would be counted as passes instead of runs. This is now changed so plays that the NFL officially counts as runs are runs in DVOA as well. Eliminating this manual change probably makes pass/run DVOA splits a little less accurate but is needed to eventually automate it based on an NFL data feed.

Let’s take a look at the change in correlations for 2013-2022. These changes are tiny, to be honest. That’s just the world we live in doing football analysis. DVOA was pretty good already. There wasn’t going to be some magic change that dramatically changed the correlation to future DVOA or future wins by 0.1 or more. Normally, you wouldn’t overhaul your system based on these tiny correlations. But I feel comfortable with these changes because they match common sense, rather than just making changes for the sake of tiny improvements in correlation.

One of the strange results was that changing the indoor/outdoor variables to score retractable roofs differently from permanent domed stadiums lowered the correlations for offense but improved them for defense. The overall correlations for total DVOA stayed the same.

The average team will see its total DVOA change by about 2% in either direction with the new version of the system. 

Here’s a list of the biggest changes over the last five seasons with the new DVOA v8.0. First, let’s look at the teams with the biggest rise.

Most teams on here won’t stand out to you. It’s nice to see a team with an 11-5 record, the 2020 Browns, go from negative to positive DVOA. That seems logical. It’s nice to have the 2022 Super Bowl champions on here as well. And two of the teams that would fall just below this table are the two teams from the 2021 Super Bowl, the 2021 Bengals (+3.7%) and the 2021 Rams (+3.4%).

Now, here’s a look at the biggest fallers with the new DVOA v8.0:

Well, let nobody accuse me of designing my metrics specifically to prop up my beloved New England Patriots. It makes sense to have that 2021 Patriots team on top of the biggest changes with the new version of DVOA. In fact, I’ve gone back to 2003 so far and the 2021 Patriots have the biggest difference in DVOA v8.0 of any team in the last 20 years, not just the last five! If you were a longtime Football Outsiders reader, you may remember that our playoff odds simulation had the Patriots with absurdly high odds of winning the Super Bowl when they were 9-4 after some huge wins in the middle of the 2021 season. Those odds would definitely be lower now. It also makes sense to have the 2019 Patriots drop in Tom Brady’s last season.

Obviously, the new version isn’t perfect. You don’t want to see Super Bowl champions like the 2020 Buccaneers come out lower in the new version. But I didn’t create this to make things more accurate for just the Bucs or just all Super Bowl champions. The goal was to be more accurate for all teams.

Notice that the 2022 Giants, who were already much lower in DVOA than wins, drop further down with the new version of DVOA. In fact, the 2022 Vikings drop further down as well! That’s astonishing. What a strange team.

It takes time to gradually run DVOA v8.0 for each year going back to 1981, but as we add past years to the FTN Stats+ DVOA database, they will be added with the new DVOA v8.0 rather than the old version of DVOA. The 2022 stats on the site have been changed to the new version and the 2023 stats that come next week will be the new version as well.

At some point we’ll run some articles that go through the entire history of DVOA, updating our lists of the best and worst teams ever. I can already tell you one casualty of the new version of DVOA: the Seattle Seahawks DVOA dynasty. With the new version of DVOA, the Seahawks no longer were No. 1 for four straight seasons. They’re still on top in 2013 and 2015 but they drop to third in 2012 (behind Denver and New England) and second in 2014 (behind Denver).

Here’s a table with all of the old and new ratings for the past five years, if you want to look up the change for any particular recent team. Super Bowl teams are in bold. Eight of the 10 Super Bowl teams of the last five years see positive change with the new version of DVOA.