Evaluating Mike Shildt’s Bullpen Management

The first full season at the helm of the Cardinals has been an interesting one for Mike Shildt. His team jumped out to one of the best records in baseball in April, stumbled through a miserable May, and surged into first place after the All-Star break. Along the way, he has staunchly defended his players, who seem to enjoy playing for him. He made an impassioned and oddly timed plea for better coverage of his team’s much-improved baserunning and defense, and aggressively used some... interesting... players out of position and at the top of the lineup. Whatever you think of all of those moves, he has the Cardinals in first place during the first week of September for the first time since 2015. Today, let’s focus on one of the strongest segments of the team. One of the most significant reasons they’re in first place has been the bullpen. How well has Shildt deployed his relievers this season?

First, let’s establish that the Cardinals have one of the best bullpens in baseball. This should be fairly easy. Here are their league-wide bullpen ranks in virtually any key category you could want through Tuesday’s games:

• DRA (Deserved Run Average): 1st
• FIP: 1st
• xFIP: 10th
• WPA: 3rd
• WPA/LI: 1st
• K%: 2nd
• K-BB%: 3rd
• wOBA: 1st
• xwOBA: 2nd

They’re first, second, or third in improving the team’s chances to win (WPA and WPA/LI), controlling the strike zone (K% and K-BB%), fielding independent pitching, deserved run average, weighted on-base average, and expected weighted on-base average. That’ll do, pig. The only category where they aren’t in the top three- xFIP- still has them top 10 in baseball. They’ve done all of this without the benefit of their closer and most talented reliever, Jordan Hicks, who went down with a season-ending injury in late June.

Lazy analysis would look at those rankings and say “Mike Shildt has done an amazing job with the bullpen!” Nobody wants that. Instead, let’s use some of those components and riff a bit on a FiveThirtyEight study from a few years back.

The Study

We’ll compile all relievers with at least 20 innings this season. Each of those pitchers will be ranked among their teammates in DRA. I’m using DRA because it strips out the most context to determine a pitcher’s quality.

Now we’ll determine the importance of the situations in which a reliever is being used. In other words, we need their leverage index. FanGraphs has a variety of metrics for that, but we’ll use gmLI– a pitcher’s average leverage index when he enters the game. We’ll rank each reliever among their teammates in gmLI.

With both of those data points, we’ll determine the gap between each reliever’s team-wide rank in DRA and their rank in leverage. If you want an example of how this looks, Josh Hader leads Brewers’ relievers in DRA, and his gmLI is also first. His gap is zero. Miguel Castro’s DRA is second in the Orioles bullpen, but his gmLI is sixth. His gap is four. Jesse Chavez has the sixth best DRA on the Rangers, but his gmLI is third. His gap is three.

Now we know how well each manager lines up his best pitchers with the highest leverage situations. The final step is to determine the average gap for each team’s set of relievers. The smaller the gap, the better the manager is at using the appropriate relievers. Here’s how each team ranks:

This is a great start but it doesn’t tell the whole story. For instance, the Rays have one of the best bullpens in the game but somehow rank dead last here. It’s hard not to think there’s opener-related highjinx causing that discrepancy. There are also some terrible bullpens ranked highly. The Phillies are 24th in bullpen FIP, the Rockies are 28th, the Orioles are 30th, the Blue Jays are 22nd, and the Mets are 20th. Managers for those teams have wisely lined up their best and worst relievers in appropriate leverage situations, but it’s also much easier to do that when you have a large variance of quality in your bullpen. Brandon Hyde isn’t a genius for using Michael Gyvens with the game on the line instead of Jimmy Yacabonis or Branden Kline. There’s a wide variance in effectiveness between the best and worst relievers for several of those teams. It’s not the same as deciphering the razor-thin difference in effectivenesss between John Brebbia and John Gant.

We can measure the variance using standard deviation in DRA for the individual members of the various bullpens, and then put it in a scatterplot with the DRA/gmLI gap.

I’ve highlighted some outliers and added black lines to illustrate the average. The lower the standard deviation, the harder the choices are for a manager. The higher it is, the more obvious their choices have been. The Blue Jays and Orioles don’t have a lot of tough decisions to make, but they’re doing a good job of identifying the right pitchers. The Braves are making some of the worst choices based on leverage, but it’s also much harder for them to discern the right reliever than it is for almost every other team. Dave Roberts has been a little worse than average in making his choices but he has the highest degree of difficulty. The Rangers have poorly aligned their best pitchers with appropriate leverage despite having easier choices.

Shildt has some of the hardest choices to make in the game, with only the Dodgers showing less variance in quality among bullpen options. Even with that impediment, he ranks tenth in lining up quality with leverage. Of the teams ahead of him, only Minnesota and Cleveland (the dot just to the right of the Twins) have comparably small variances of quality in the bullpen, and it’s still not as tough as Shildt’s job.

Using the teams ahead of him in lining up DRA with leverage, actual effectiveness is quite a tie-breaker. Yet again, Cleveland and Minnesota are the only teams that can compete using FIP, WPA, WPA/LI, and K-BB%, and neither surpasses the Cardinals in those categories. They’re merely close enough to notice.

Once you add up the three components- how accurate are his choices, how difficult are his decisions, and how effective have his choices been- Mike Shildt has arguably been the best bullpen manager in baseball this season.