Mike Matheny and Bullpen Efficiency

As you might have heard, the Cardinals and field manager Mike Matheny just agreed to a three year extension, giving Matheny a guaranteed salary in St. Louis for the next four years. Understandably, there are now portions of the fanbase that are upset.

I personally have wished for awhile now that the Cardinals would hire Gabe Kapler, a very sabermetric-savvy ex-player currently serving as Director of Player Development for the Dodgers. If the alternative is someone else who thinks and manages like a typical manager, it’s hard for me to get too worked up.

In light of Matheny’s extension, let’s look at how well he’s done at managing the bullpen. That’s a broad subject, so let’s break it down. I wanted to see how well Matheny was at getting his best relievers into the highest leverage situations, and how well he did at getting his worst relievers into the lowest leverage situations.

I actually already did this last month. What I did was this: I grabbed each Cardinals’ reliever’s xFIP and gmLI (average leverage when entering a game), and ranked them. I then took the difference between the two, to get a look at how well Matheny did at using his bullpen options efficiently.

I came to the conclusion that besides the egregious use of Kevin Siegrist (which was due to him over-performing his xFIP the whole year in terms of ERA), Matheny had actually did pretty well. In general, the community response agreed with that. However, what we were lacking at the time was context: how had other managers performed following this method?

Remember here that we’re talking about just one part of one aspect of managing a baseball team. This isn’t ranking managers for their overall performance or all bullpen usage. This is simply how well a manager gets his best relievers into the highest leverage situations and worst relievers into the lowest leverage situations. There’s other considerations when considering bullpen use, such as how often a manager gets the platoon advantage.

So what I did was this: For each of the 30 teams, I ranked every pitcher who threw more than ten innings in relief by xFIP and gmLI. I took the absolute value of the difference between xFIP rank and gmLI rank for every reliever. I then multiplied that difference by the percentage of innings each pitcher threw. That scales the magnitude of the mistake as well as controls for teams whose bullpen threw significantly more or less innings, or had several more or less pitchers than average.

Then I totaled up that score by team, which I decided to call Bullpen Efficiency or BE. Since we’re calculating the size of the difference between a pitcher’s performance and the leverage he was brought in to, the lower score the better. Here’s how Matheny fared compared to the rest of the league:

The Cardinals placed 12th using this method, and 6th among teams that have to consider when the pitcher’s spot in the order comes up. Matheny places much better than two managers who get a lot of respect as strong managers of the bullpen: Bruce Bochy and Terry Francona. Both managers place in the back half of MLB teams.

You might be skeptical of this stat if it finds both those managers to be below average, and Matheny as above average. I understand that sentiment, it surprised me too, and at first I had to spend a lot of time double and triple checking my code to make sure it was doing what I wanted it to. But it checks out. So let’s look at how the Giants’ bullpen usage looks:

The Giants’ best reliever by xFIP - Derek Law - entered games with the ninth-lowest leverage. Javier Lopez posted strong numbers before 2016, but was the team’s second-worst reliever by xFIP. However, that didn’t stop Bochy from inserting him into the second-highest leverage situations. Santiago Castilla also posted weaker numbers than the last two years.

So it seems like Bochy was too slow to go to newer and better options in 2017. Perhaps you could spin that as a positive, in that he trusts his guys. I wouldn’t buy that though. I’m not going to say Bochy is over-rated or anything, as I said this is only measuring a specific part of bullpen management. Let’s look at Francona’s use with the Indians:

The Indians have a trio of relievers that were misused: Dan Otero was under-utilized, Jeff Manship and Zach McAllister were over-utilized. During the regular season at least, Francona’s highly publicized use of Andrew Miller didn’t get Miller into the highest leverage situations. However, since he still held the second-highest leverage, the penalty was very small. Also, Miller did manage to throw 29 innings in the final two months of the year for the Tribe, which is an advantage this method doesn't capture.

Now let’s look at Matheny, who outperformed both of these two:

Matheny misused Siegrist worse than Francona or Bochy misused anyone, but he still did better overall. Matt Bowman and Tyler Lyons were both under-utilized as a result of Siegrist’s over-use, but after that, it was pretty smooth sailing.

Finally, just to see the team that scored best, here’s the Red Sox:

First off, wow, I didn’t realize Craig Kimbrel had a season well below his own excellent standards. His xFIP in 2016 was a full run higher than any of the other six years of his career. It’s hard to blame manager John Farrell for giving Kimbrel the highest leverage on the team, like this method does. However, even when doing so, the Red Sox still rated first. The biggest difference the team had was 8, and that was for Fernando Abad, who threw less than 13 innings on the year.

Let’s get back to the Cardinals though. Still, the biggest misuse of a player we’ve seen thus far is Kevin Siegrist. Let’s look at the top 10 relievers in terms of putting a reliever in too much leverage:

Indeed, Siegrist was the second most over-utilized reliever in the majors this year. In that context it’s surprising that Matheny did so well. Erasmo Ramirez, like Siegrist, out-performed his xFIP by a run. You can’t say the same for Matt Albers though, who had a 6.31 ERA.

On the flip side of things, let’s look at the ten most under-utilized relievers:

Otero and Law make an appearance here, as part of what brought the Indians and Giants’ scores down. Bowman and Lyons’ under-use was not big enough to make the top 10.

Again, let’s keep in mind that this is measuring one narrow aspect of a manager’s job. However, the results look pretty good. Matheny makes mistakes with bullpen management, that’s for sure. Still, it’s hard not to be pleasantly surprised that Matheny did a better job than average at putting his relievers in the situations they should have been in. I'm not saying this means Matheny is a good manager now, but he passes this test at least.