Week 10 DVOA: How Real are the Ravens’ Q4 Problems?

The NFL had a record six different games decided by a game-winning field goal on the last play this week. That’s going to usually wreak havoc on the connection between DVOA and win-loss records. In general, fans and commentators will wildly overrate the predictive value of a close win on a last-second field goal. These games are generally ties in the underlying statistics. They often have what I call “reverse PGWE.” The Post-Game Win Expectancy (metric explained here) will be close to 50% and sometimes the team with a higher PGWE will lose the game.

This actually was not the case in Week 10. In four of these last-minute field goal games, the team that won the game also dominated the underlying statistics. Here’s a look at single-game DVOA and PGWE for all six of these games. Remember, there’s going to be a difference between DVOA and PGWE because PGWE doesn’t incorporate opponent adjustments and accounts for things that DVOA does not, such as penalties.

The Texans-Bengals game really stands out here. It would have been a huge upset by PGWE if the Bengals had eventually won this game. The Texans outgained the Bengals by nearly a yard per play and had a higher success rate on offense, 47% to 38%.

You may notice that the two games with reverse PGWE are the two highest-profile games, in particular the games with the highest profile for people who follow my DVOA ratings. PGWE suggests that the Ravens had a slightly higher chance of a win than the Browns and that the Bills had a slightly higher chance of a win than the Broncos. Note that in each case, the winning team did have the higher DVOA. A big reason for the discrepancy between DVOA and PGWE here is penalties, as the Browns had more flags than the Ravens and the Broncos had more flags than the Bills.

I’ve written a lot about the Ravens in recent weeks, and I’m going to write some more about them today. They are having a very strange season. As you can see from the table above, they still had a positive DVOA this week despite the loss. Combine that with the fact that the Ravens had a huge lead in DVOA after Week 9, and the Ravens are still No. 1 in our ratings despite the loss. The Ravens dropped in DVOA from 49.0% to 44.8% but still have a healthy lead over the new No. 2 team, the San Francisco 49ers, who rose to 35.8% DVOA with a huge dominant win over the Jacksonville Jaguars. Opponent adjustments are a big part of this, as the 49ers would be No. 1 without them but the Ravens have played the sixth-toughest schedule in the NFL so far.

The Ravens have had a very strange season, so it looks very strange that they still rank among the best teams in DVOA history despite a 7-3 record. The Ravens are not only the highest-rated 7-3 team ever but also have a higher DVOA than any 8-2 team since 1981.

What’s strangest about the Ravens this year is that they’ve led almost wire-to-wire in every game, only to frustrate their fans with three collapses late in the fourth quarter. The Ravens still lead the NFL in point differential and all three of their losses have come by a touchdown or less. This stat from NFL Research is astonishing:

Meanwhile, Brian Burke of ESPN had a tweet showing just how extreme the win probability changes were in the three Ravens losses:

The Ravens have blown six different double-digit leads in the last two seasons. It certainly feels like this is something that is intrinsic to the Ravens’ play style and therefore something we should expect to continue for the rest of the year. That means they aren’t really Super Bowl contenders despite their high DVOA. Or does it? Are the Ravens doomed to continue to blow fourth-quarter leads and lose games they should have won?

I’m not going to try to answer that question, specifically. But I do want to answer the question: Can we expect the Ravens offense to continue to falter in the fourth quarter going forward?

If you download the full workbook of 2023 DVOA splits — which you can do as an FTN subscriber to Stats+ or Fantasy+ packages — you will see that the Ravens offense is 23rd in the fourth quarter and overtime. The offense drops with each quarter, but they’re still very high in the first three quarters. The Ravens rank third in Q1, fifth in Q2, and sixth in Q3. Then the offense disappears in the fourth quarter.

Well, the fourth quarter and overtime. The first thing to note is that the Ravens’ rank improves if you only look at the fourth quarter. In their one overtime game, against Indianapolis, the Ravens went three-and-out and then four-and-out. Take those plays out, and the Ravens rank 17th in fourth-quarter offense. That’s a little better, but it’s still a big decline from those first three quarters.

Is this an ongoing issue for the Ravens? Despite the blown losses from 2022, it doesn’t seem like it is. I went back and looked at the last three years. In previous years, the Ravens simply did not see their offensive DVOA collapse in the fourth quarter. For example, early in 2022 the Ravens lost to Buffalo on a last-second field goal. Still, in that game Lamar Jackson led a 93-yard drive in the fourth quarter that ended with an interception on fourth-and-goal from the 2. Despite the interception, that is a good offensive drive. Last year, the problem in the fourth quarter was as much the defense as the offense.

I wanted to answer the question: Just because the Ravens offense declined in the fourth quarter in Weeks 1-10, should we expect a similar decline in the fourth quarter in Weeks 11-18? The answer is pretty clearly no.

I went back and put together numbers from the last 10 years. Here are the correlations for DVOA overall and in the fourth quarter, when you compare Weeks 1-10 to Weeks 11-18.

• Correlation of total offensive DVOA: .560
• Correlation of Q4 offensive DVOA: .291

That makes it look like there is some correlation between offensive DVOA in the fourth quarter in the first and second halves of the season, right? Well, of course there is! Because good offenses overall will tend to play better in the fourth quarter, and good offenses overall tend to continue to play well in the final half of the season! What happens if we separate Q4 from the rest of DVOA by looking at the gap between overall DVOA and Q4 DVOA. Does that correlate from the first half of the season to the second half of the season? Not at all.

• Correlation of gap between total and Q4 offensive DVOA: -.004

A team’s total offense for the entire game is a much better predictor of future fourth-quarter offense than looking at just fourth-quarter offense in Weeks 1-10.

• Correlation of total offensive DVOA Weeks 1-10 to Q4 offensive DVOA Weeks 11-18: .415

Over the last 10 seasons, the team with the biggest gap between offensive DVOA and Q4 offensive DVOA for Weeks 1-10 was the 2018 San Francisco 49ers. And, in fact, the 49ers continued to struggle in the fourth quarter for the rest of that season.

But the team with the second-biggest gap was the 2016 New England Patriots. And from Week 11-17, the Patriots actually saw their DVOA in the fourth quarter go up compared to the first three quarters.

The 10 teams with the biggest Q4 gap in Weeks 1-10 saw that gap go from an average of -40.3% to an average of -9.6% in Weeks 11-18. On average, fourth-quarter offense is about 6.6% less than offense overall, for various reasons, so that gap of -9.6% is basically nothing.

Out of curiosity, I looked at defensive DVOA in the fourth quarter as well. And then I looked at the difference between the first quarter and overall DVOA for both offense and defense. This analysis is only looking at 10 years of data and was somewhat cursory, but it suggests that there is no predictive value to DVOA splits by quarter. The correlations for first-quarter DVOA are tiny, a lot smaller than you would expect given the importance placed on the idea of scripting the first 15 plays of the game.

• Correlation of gap between total and Q4 defensive DVOA: -.021
• Correlation of gap between total and Q1 offensive DVOA: .049
• Correlation of gap between total and Q1 defensive DVOA: .052

By the way, the Ravens don’t even have the biggest gap this year between total offensive DVOA and Q4 offensive DVOA. Miami has the largest gap, followed by Dallas and then Baltimore. And yes, that incorporates adjustments for the fact that teams tend to slow down their offenses when they are up by a lot in the fourth quarter.

Let’s finish up with some “best of” lists. I’ll just copy these over from Twitter to save myself some time! Make sure to follow me on both Twitter and Threads @ASchatzNFL to get these cool lists on Mondays or Tuesday mornings or whenever I feel like sending them out. (The Browns defensive DVOA is slightly different after MNF compared to the table in the tweet below.)

* * * * *

These are the FTN DVOA and DAVE ratings through 10 weeks of 2023. Our Defense-adjusted Value Over Average (DVOA) system breaks down every single play and compares a team’s performance to the league average based on situation in order to determine value over average. (Explained further here.) This week, opponent adjustments reach 100% strength and will remain that way for the rest of the year.

OFFENSE and DEFENSE DVOA are adjusted for performance indoors and consider all fumbles, kept or lost, as equal value. SPECIAL TEAMS DVOA is adjusted for type of stadium (warm, cold, dome, Denver) and week of season. As always, positive numbers represent more points so DEFENSE is better when it is NEGATIVE.

DAVE is a formula which combines our preseason forecast with weighted DVOA (giving slightly more weight to recent games) to get a more accurate projection of how a team will play the rest of the season. DAVE is currently 24% preseason forecast and 76% actual performance for teams with 10 games and 36% forecast with 64% actual performance for teams with nine games.