Major League Baseball umpires heard their job approval ratings plummet in Washington, D.C., during this World Series, culminating with chants of disdain from Nationals fans after missed calls on Sunday in Game 5. And that was before the controversial runner interference call on Washington shortstop Trea Turner in Game 6. Every decision, every call, every mistake is amplified in postseason baseball — and when the work behind the plate could affect the outcome of the game, everyone notices.
To be human is to be imperfect. Deciphering borderline pitches traveling at 100 mph and breaking balls that move more than ever is not easy. And there will always be errors made behind the plate unless humans are replaced with an automated zone (which MLB began experimenting with this past summer in the independent Atlantic League).
But one study tells us that MLB might do a better job of getting balls and strikes called correctly simply by employing different umpires in the postseason. Mark Williams, a professor in Boston University’s Questrom School of Business, looked at called pitches from 2008 to 2018 and compared the more than 4 million pitches against ball-location data provided by MLB tracking cameras.1 He calculated ball and strike accuracy performance for each umpire, producing a bad-call rate per umpire per season, and has launched an app that evaluates and updates umpire performance.
“Baseball has a problem behind home plate, too many ball-strike calling errors,” Williams told FiveThirtyEight.
MLB has disputed Williams’s findings. League spokesman Michael Teevan noted that the missed-call rates MLB uses internally differ from those of the Boston University data and that MLB’s methodology “takes into account the margin of error of the tracking system.” (Williams says that, via Statcast and PITCHf/x, he is using the same underlying pitch-tracking and zone data as MLB, and maintains that umpires miss far more calls than the league is willing to admit.) Teevan also said factors other than just ball-strike calls are important in determining which umpires are used in crucial postseason games.
In the postseason, there were 252 pitches called strikes outside the zone and 195 called balls that were in the strike zone. That’s 447 missed calls out of 5,459 called balls and strikes, a missed-call rate of 8.2 percent. That’s better than the regular-season miss rate of 9.1 percent,2 but those are still hundreds of errant calls influencing game outcomes.
Only three umpires who received an assignment behind the plate during LCS and World Series play ranked in the top 10 this season in missed-call rate, according to Williams’s data, though Nos. 11 and 12 did work home plate in the World Series. Three umpires who called LCS games ranked in the bottom half of MLB’s 76 full-time umpires, as did three umpires assigned to the World Series. And the postseason has featured four of the worst 15 game-calling umpires behind home plate.
Many of the game’s best ball-strike umpires are invited to the playoffs and placed behind the plate, but not all of them. Why not? Teevan said assignments are “merit based” but that the evaluation criteria goes beyond ball-strike accuracy. “A variety of factors [are taken] into account, including experience, skill sets, communication and situation-handling,” he told FiveThirtyEight.
That experience might be part of the issue. Williams found that less-experienced umpires often performed better than veterans in ball-strike performance. Moreover, Williams found that there was typically little change in an umpire’s year-to-year missed call rates, suggesting that improving umpiring skills is difficult. The average service time of all MLB umpires this year was 16 years. The umpires in the LCS who called games from behind the plate had slightly less experience, averaging 14.6 years in the league. But the average crept up again among World Series umpires, to 16.4 years.
Ball-strike calls are incredibly important. Offensive performance in the majors is tied to the count, and just one missed pitch can have a significant effect. During the regular season, hitters had a .351 batting average on a count of two balls and one strike, versus a .161 batting average on a count of one ball and two strikes — a difference of nearly 200 points in batting average. Moreover, umpires are responsible for calling more and more balls and strikes as fewer balls are being put in play because of the record strikeout levels of recent years. This season marked a record for pitches thrown in a season,3 and a record for the number of called strikes in a season.
As long as humans — and not robots — are behind the plate, a certain number of calls will be missed. But on baseball’s biggest stage, it’s more important to get them right.