Questionable calls are made at every point of the NFL season, whether it’s the first week of preseason or a tie game in overtime for the Super Bowl. Most of us can remember where we were when the Calvin Johnson Rule came crashing onto the scene in 2011, and the replacement refs’ “Fail Mary” play was just as infamous.

On the whole, though, the referees get most of these calls right, and they get so little credit for it. So, the next time you see someone in vertical black-and-white stripes, extend a hand; thank them.

Still, when celebrations receive a whistle blow nearly every game, when breathing on a receiver or blocking a rusher so well that they fall over nets a rain of yellow flags, we have to wonder how much penalties are affecting the play on the NFL field these days. Many people believe that penalties have ruined some of the biggest moments in the league in recent years, and that -- like in days of old -- refs should “let them play” through it.

But is that true? Are penalties really changing the game on the field?

Seeing Yellow

Now, when people try to claim that football is being ruined by new rules aimed at aiding player safety, that’s just not true.

The additional emphasis placed on calling helmet-to-helmet hits, horse-collar tackles, launching, and leading with the crown in recent years led to a 54.46 percent decrease in concussions from 2012 to 2014, per the NFL’s Health and Safety Office. In addition, concussions from helmet-to-helmet hits alone were down in that span -- from 91 in 2012 to just 52 in 2014 (a 75 percent decrease). That’s what rules are there for: to help protect players and keep the game fair and exciting.

With this much positive impact shown, it might seem then like the NFL is throwing flags all willy-nilly these days. Outside of Jerome Boger’s crew, which has led the league in penalties called each of the last two seasons, penalties in the regular season are surprisingly way, way down from last year.

The table below shows the number of penalties called from 2012 to 2016 (per NFLPenalties.com), in total and per-game, and shows the difference between penalties called and those accepted by teams.

There is a very clear spike in the 2014 season in penalties called, due to the policies put in place to emphasize player safety and calls to err on the side of caution.

Despite the new plateau of roughly 16 penalties per game, these penalties peaked in 2015, and we’ve now started to regress somewhat in the volume of penalties called. As officials get a better grasp on how to identify calls for the new rules and changed definitions, it appears that the calls are becoming less and less frequent once more.

It doesn’t seem like an additional two or three penalties per game would make anything painful or unwatchable. In addition, the number of penalties accepted and actually affecting the outcomes on the field is less than one per game -- 0.83 -- when you compared 2016 to 2012.

As a matter of fact, it seems like the emphasis on calling violent penalties is actually working very well and helping to drive down the overall number of penalties committed.

We realize that 2015 was a year where a lot of penalties were called, but the unnecessary roughness calls are barely measurably higher at this point in time than they were five years ago (0.06 per-game difference), roughing the passer is at a five-year low, and total personal fouls have dropped from 209 called in 2015 to 175 this year (0.14 per-game difference).

The volume of penalties is dropping again; it’s important that we recognize that.

Zebra Crossing

But how impactful have penalties been in games this year?

In the Kansas City Chiefs’ Divisional Round loss to the Pittsburgh Steelers, a questionable holding call negated the Chiefs’ successful two-point conversion at the end of the game. Instead of tying things up at 18-all, it moved them back 10 yards and forced them to retry it (they failed to convert).

When we look at the game impact of that, we can see just how upset viewers might be about a borderline call swinging the game the way it did. We can see some of the impact on the scoreboard, but how much did this penalty really affect the game itself?

Net Expected Points (NEP) is an analytic that describes the contribution a play (or player) makes to their team’s chances of scoring. By adding down-and-distance value to standard box score information, we can see just how much each play and each team as a whole influence the outcome of games. For more info on NEP, check out our glossary.

Penalties carry NEP values, too, and they also help to swing the outcomes of games in terms of Win Probability Added (WPA). The table below shows the average NEP and WPA that an accepted penalty has cost in both 2015 and 2016 (the two years we currently have detailed play-by-play data). How much value has been left on the field by penalties over the last two years?

Shockingly, penalties have swung games more critically this year than last year, according to both NEP value and win probability.

Part of this is due to penalties being called on later downs at higher rates. In 2015, third- and fourth-downs saw penalties just 7.12 percent of the time, while this year that skyrocketed to 13.06 percent. Losing yardage on critical downs can shift a play’s value from middling to near-none, just as getting a critical defensive pass interference call on a fourth-down incompletion can give new life to a drive.

In the fourth-quarter of Week 1 of 2016, for instance, in a game between the Carolina Panthers and Denver Broncos, quarterback Cam Newton attempted a pass to Kelvin Benjamin that sailed incomplete on 4th-and-21. Cornerback Chris Harris was flagged for illegal use of hands, however, which gave the Panthers five yards and a new set of downs. That play cost the Broncos 2.37 NEP and 32.02 WPA.

Even in terms of the volume high-impact NEP and WPA, 2016 has seen more game-changing calls than 2015. The table below shows the amount of penalties costing more than 3.00 NEP and more than 10.00 WPA.

Maybe this is good for the game, however. No, teams are not being allowed to just “play through it” so much anymore, but the no-calls that haven’t been made in big spots in the past aren’t slipping through the cracks as much anymore. We as fans hate to see the game slowed down, but the officials are getting their calls right and making sure that in the most important situations, they are stepping up and protecting both the game and the NFL's players.

It’s never easy to wear the zebra stripes, but not all heroes wear capes.