Whenever there's a former Big 12 quarterback in the upcoming draft class, you'll always hear extensive cries about the conference's lack of desire to pretend to play defense. If you compliment a player's choice of shoes, the first response will likely be mocking Texas Tech's secondary.
We get it.
That doesn't mean we should fully dismiss this assertion, though. After all, if we're going to evaluate these guys based on their statistics, the strength of their opponents should at least play a role in our process. You'd naturally expect an NFL-caliber quarterback to obliterate foes unworthy of sharing the same field, but what does he do against better teams?
Further making this necessary is the absence of quality Big 12 passers in the NFL. Guys like Sam Bradford and Robert Griffin III have had flashes of success, but you've also got the ghosts of Vince Young, Blaine Gabbert, and Brandon Weeden scaring off all optimism. As annoying and persistent as this stereotype of Big 12 defenses may be, there are legit reasons to wonder whether or not we should trust the numbers of quarterbacks who play there.
Thankfully, we can attempt to qualify for this.
By looking exclusively at how quarterbacks fare when facing upper-level defenses, we'll be filtering out the dud opponents who may have fluffed their stats. It's not going to fully show us the quarterback's true talent level -- supporting cast still matters -- but it can give us some extra context about the statistics themselves.
So, let's do exactly that. We're going to see how each of this year's top six quarterbacks performed against top-50 pass defenses. That ranking will be determined by Football Outsiders' S&P+ pass defense metric, which measures the all-around effectivenss of a team at stopping opposing passing games. If the guys did well when facing these teams, it means we can put a bit more faith into their statistics. If not, we may want to be more skeptical.
But before we do that, we should see how often each player was facing these types of defenses. Let's start off there before diving into their performances.
How Often Did They Face Solid Competition?
Because our previous look at each quarterback's efficiency stats focused exclusively on their final-season numbers, we'll be looking only at their opponents during the 2017 season. This means the samples for Josh Rosen and Josh Allen will be a bit smaller as both missed time due to injury, playing 11 total games each.
With that said, here's how often each player faced top-50 pass defenses, based on S&P+. The middle column shows that, and the right-hand column shows the average pass defense ranking of each team the quarterback faced that year. This will exclude any teams they faced that are not members of the Football Bowl Subdivision (FBS).
Quarterback | Games vs. Top-50 Pass Defenses | Average Pass Defense Ranking |
---|---|---|
Sam Darnold | 9 | 45.29 |
Lamar Jackson | 7 | 48.75 |
Baker Mayfield | 7 | 52.71 |
Josh Rosen | 7 | 52.82 |
Mason Rudolph | 5 | 62.85 |
Josh Allen | 3 | 78.80 |
Allen and Lamar Jackson both had one game against a non-FBS team, meaning their schedules will be a hair softer than they appear on the chart above. Let's sort through some of the takeaways here.
First, Sam Darnold had himself a tough road. Nine of his 14 games came against top-50 pass defenses, and five of them were in the top 20. His raw stats were merely mediocre, but this does give him a slight boost given how often he was facing top-tier units.
Second, the "Big 12 defense" narrative just lost a lot of weight for Baker Mayfield. Half of his games were against top-50 pass defenses, and -- like Darnold -- five of those were top-20 teams. Sure, he faced teams like Kansas, but you also have to mention matchups with Ohio State, two games against TCU, and Georgia in there, teams ranked 12th, 16th, and 6th, respectively, against the pass. We'll get to how he did when facing those teams in just a second.
If you're going to try to knock anybody for facing Big 12 defenses, it would be Mason Rudolph. He faced just two teams inside the top 20 in pass defense, and his schedule was easily the second softest of the group. Rudolph's stats were among the best in this class, but his schedule does seem to be a fair criticism.
How Did They Fare Against These Defenses?
Finding out how often players faced respectable competition is one thing; seeing how they did in those scenarios is an entirely different discussion. Both should factor into our evaluation.
If a player rarely faced elite defenses but did well in the cracks he did get at them, then we probably shouldn't hold a soft schedule as much against him. If he struggled in these instances, then we need to have a broader discussion about the validity of a player's overall stats.
As such, let's look just at how each player performed when facing top-50 pass defenses this year. AY/A stands for adjusted yards per attempt, which factors touchdowns and interceptions into a yards-per-attempt-esque number to give an overall look at that player's efficiency.
Versus Top-50 Pass Ds | Attempts | Touchdowns | Interceptions | AY/A |
---|---|---|---|---|
Baker Mayfield | 200 | 22 | 4 | 13.5 |
Mason Rudolph | 177 | 12 | 4 | 10.2 |
Josh Rosen | 315 | 16 | 5 | 8.3 |
Sam Darnold | 331 | 13 | 8 | 7.7 |
Lamar Jackson | 250 | 13 | 8 | 7.0 |
Josh Allen | 90 | 1 | 4 | 3.2 |
What's the more shocking number on that chart: Mayfield's destruction of universes or Allen's complete collapse? Let's start off with the good.
To give some context on Mayfield's AY/A, the single-season record in college football history is 12.91. That was set by -- shocker -- Mayfield in 2017 (breaking his own previous record of 12.30). His AY/A against top-50 defenses was somehow even better than that number. He threw at least two touchdowns against every top-50 pass defense he faced while throwing just four total interceptions. There are no words to describe how impressive this is.
Because of this, if you're going to attempt to invalidate Mayfield's stats, you'll have to do so without the crutch of the "Big 12 defense" narrative. Yes, he faced some bad defenses. But he also faced a good number of great ones, and he absolutely gutted them. It's fine to have concerns about Mayfield, but all of the data says he is a stud in the making.
Now to the bad. Allen's AY/A was more than 10 points lower than Mayfield's. He threw just one touchdown to four interceptions. If we take those numbers for what they are, this dude has no business going in the first round.
Obviously, though, it's not that simple. Having a team like Wyoming go up against a top-50 defense is a whole heck of a lot different than asking Oklahoma to do so. Supporting cast does matter, and Allen's going to be at a much bigger disadvantage than others in this discussion. We can't use those numbers alone to craft an argument saying that Allen is an awful quarterback.
However, even when we looked at Allen's numbers in the Mountain West Conference, they left something to be desired. His AY/A against those teams was 6.9 in 2017. That was a step down from his 2016 mark of 9.4, but these are the games where Allen should be on more of a level playing field. He still didn't pop statistically. Again, this isn't to say that he can't be a successful NFL quarterback, but he truly does need to be an outlier to get there.
Rudolph didn't come out smelling too pretty in our look at each player's schedule. But he redeems himself quite a bit here. In those five games against top-50 pass defenses, Rudolph performed well with a 10.2 AY/A, just a step below his full-season mark of 10.7. We do need to be cautious because of the defenses he faced, but it's clear we can't use that one argument to fully disqualify all that he did this year.
Because Rosen and Darnold are so often near the top of individual draft rankings, they're going to be compared to each other often. And although Darnold had more games against quality pass defenses, it was Rosen who pushed out the more impressive performances.
The big separator for the two in this instance is interceptions. Darnold had 8 in 331 attempts while Rosen had just 5 in 315. Interception-happy college quarterbacks can be successful in the NFL -- just look at Jameis Winston and Deshaun Watson -- but it's certainly a mark against Darnold here.
This is the third area in which Rosen holds a slight edge over Darnold. The first is that Rosen had more games in college with at least 10 pass attempts, which has been a relevant category for evaluating quarterbacks in the past. The second was that Rosen performed better while trailing than Darnold, meaning he was able to succeed even when the defense knew he was going to pass. Rosen also trailed often thanks to UCLA's 120th-ranked defense, according to S&P+.
These three pillars aren't enough to say -- with absolute conviction -- that Rosen is better than Darnold. There are plenty of other factors that will weigh into that decision. But Rosen has a lot going for him from a statistical perspective. If he starts to fall -- as ESPN's Mel Kiper recently predicted he would -- Rosen could wind up being a value pick.
Finally, based on the chart above, it would seem as if Jackson struggled when Louisville faced tough opponents. That's not entirely true. In reality, he had one awful game in the TaxSlayer Bowl against Mississippi State, and it tanked his numbers in this split. If you exclude that game, his AY/A jumps to 7.9. While there is some concern here, these numbers should not be used to write Jackson off as a potential stud quarterback prospect.
Conclusions
As always, it's worth re-stating that NFL scouts are good at what they do, and if they deem a quarterback worthy of being a top-end selection, we do need to factor that into our evaluation. But the stats do tell a pretty interesting tale.
The biggest takeaways here are around our Big 12 quarterbacks in Mayfield and Rudolph. Mayfield faced plenty of tough defenses, and he ripped out their souls. This further validates his already-godly statistics and makes him look even better than he did before.
With Rudolph, it's a bit more complicated. His schedule was soft, but he measured up well when he did get chances against good units. There's still reason to believe he could wind up being a steal at the end of the first round based on what the data is saying.
We can't use these numbers to make definitive statements about any quarterback. But we also can't just flat out ignore them. Stats are always better with context, and looking at how quarterbacks do against tough competition provides us with exactly that. Although we'll never have a perfect way to evaluate quarterbacks based on their college statistics, this practice gives us a better idea of which ones are legit and which deserve a bit extra scrutiny.