April Turnover: The In-Season Fluidity of Cardinal Bullpens

There’s a dopey Missouri expression about the weather. “If you don’t like the weather in Missouri, stick around a few minutes.” It wasn’t until recently that I discovered that everyone everywhere uses this expression about their own locale, unless you live in the upper Midwest (the weather expression there: “We have two seasons- winter and road construction” or “shovel and swat” depending upon local mosquito size). Even more recently I discovered that the “stick around” phrase most likely originated with Mark Twain referencing New England. Maybe it’s not an expression about Missouri, but at least it’s Missouri-adjacent. I digress. The point is that people in a lot of places like to joke that their undesirable weather can change on a dime until you find something more to your liking. As you may be able to deduce, I’d like to discuss the 2018 Cardinals’ bullpen. Stick around for a few minutes and maybe we’ll find a bullpen more to our liking.

You see, the Cardinals’ bullpen through April has been volatile. There’s dichotomy when looking at the big picture of the ‘pen thus far, and there’s just as much dichotomy wrapped up in most individual performers. Jordan Hicks’ stuff is electric (that’s great!), but his peripherals and rate stats are dreadful (that’s awful!). Dominic Leone has splendid K and BB rates (that’s great!), but he’s been stung by a terrifying HR rate (that’s awful!). The Norris/Holland/Gregerson off-season acquisitions have netted the Cardinals a solid closer option in Norris (that’s great!), but they keep trying to shoehorn the worst of them so far into the closer role (that’s awful!). First they’re up, then they’re down. Even looking at the big picture will give your brain whiplash. They’re 12th in bullpen ERA through April (not bad!) but 21st in FIP (not good!) and really awful in many other measures (that’s REALLY not good!).

The good news is that bullpens tend to have a lot of in-season turnover, both in St. Louis and league-wide. One way or another, they’re going to try to stabilize this situation, whether it’s through minor league callups, the waiver wire, or trades. Injuries are also going to have their say before the smoke clears, and a large chunk of this bullpen will look drastically different moving forward. To determine how much of a bullpen looks different after April, let’s take a look at the percent of bullpen turnover from April through the end of the season. When I say percent of “bullpen turnover,” I’m referring to the percentage of innings from May until the end of the year that are thrown by pitchers who didn’t pitch for the MLB club in April. These are specifically innings thrown in relief. In the graph below, you’ll see the maximum and minimum turnover percentage by individual teams by year, the league average turnover, and the Cardinals’ own turnover percentage highlighted in red. I started with 2009 for two reasons. First, it was John Mozeliak’s first real season at the helm in St. Louis, as 2008 was something of an evaluation year. Second, I didn’t want to go too far back because the dynamics of the game have changed drastically this century. Going too far back might net some unintended weird results.

Other than a few outliers, the Cardinals are pretty much right in line with the average amount of attrition or turnover. 2013 was an outlier on the high end. Lots of changes were needed that season. You may recall 2013 as the season in which the Cardinals gave lots of April innings to Mitchell Boggs and Mark Rzepczynski just before they burst into a pile of ash, along with letting a raw, still-learning Joe Kelly fumble his way around a strike zone. By the end of the season, Carlos Martinez, Tyler Lyons, Seth Maness, and Kevin Siegrist arrived on the scene to help restore some semblance of order. 2009 was an outlier in the opposite direction (a very low turnover rate), but it’s partially skewed. The Cardinals gave a lot of auditions to relievers in April that year. Boggs, Blaine Boyer, and P.J. Walters were just three of the 12 relievers who appeared in April 2009. As such, if any of those 12 pitchers appeared later in the year, it didn’t count as “turnover” here, even if they were hardly key contributors early in the season. The result was a lower turnover rate than other years. In fact, it was the lowest turnover rate in baseball that season.

In an average season since 2009, 28.8% of the Cardinals’ innings from May until the end of the year are thrown by pitchers who weren’t used at any point in April out of the bullpen. League-wide, it’s 30.2%. There’s about a one-in-three chance that whichever reliever is giving you nightmares right now won’t continue to pitch as the season progresses. That’s the good news. It’s a war of attrition and a large chunk of April bullpen innings are unlikely to remain on the roster. Do these changes actually improve the bullpen?

I’ve zeroed in on three metrics to gauge the effectiveness of these changes. I’m going to use FIP, WPA (Win Probability Added), and WPA/LI (WPA divided by leverage index, which creates a context-neutral version of WPA). I’ve collected data since 2009 for those three metrics both through April, and through the full season, for all MLB bullpens. From there, I’ve given each team a percentile rank amongst the sample. Finally, I’ve looked at the change in percentile rank in those three categories. A positive change in percentile rank means the bullpen’s standing improved from the end of April until the end of the season. A negative change means the bullpen got worse, at least in terms of that metric. This time, I have info since 2008. Pardon the inconsistency. All we need here is a simple slope chart for each of the three metrics. Seasons with positive changes are marked in red, negative marked in black/charcoal (accountants will hate me for that).

This is a mixed bag. Five of the ten seasons saw WPA improvement, and five of the ten seasons saw WPA/LI improvement. It’s been a coin flip whether or not they would improve in those categories. FIP, on the other hand, has seen regression in seven of the ten seasons. If the WPA and WPA/LI categories have generally seen improvement and FIP has regressed, the implication is that there’s a luck element (or enhanced defense, important caveat) to the improvement in the two WPA categories. That condition applies to 2011 and 2014. Then there’s 2013 again as a huge outlier. The way the franchise handled the bullpen that season was a master stroke. More recently, there have been some positive trends in 2016 and 2017- giant WPA/LI gains both years, and a giant FIP gain in 2017- but they’re undermined a bit by WPA regression in both years and FIP regression in 2016.

In the aggregate in this time frame (2008-2017), the Cardinals have the 14th best WPA gain from April until the end of the season, league-wide. They’re 13th in WPA/LI, but a disappointing 24th in FIP gain. In other words, the Cardinals have been slightly above average in enhancing their WPA and WPA/LI, but well below average in addressing bullpen FIP after April. This is not to say that the moves they’re making in-season aren’t working, or have been mediocre. After all, some of their gains in these three categories could be suppressed by pitchers who were on the roster in April and remained throughout the year. Whatever the case, the net result has been a decline in FIP, and very mild gains in WPA and WPA/LI.

Let’s try to decipher whether the gains and declines in our three categories have been because of post-April additions, or because of the April pitchers pitching deeper into the season. I’ve calculated the percentage of total bullpen innings that each reliever has accrued, both in April and from May until the end of the season. We’re going to identify the pitchers who gained the most in usage by subtracting April usage percentage from the ROS (rest of season) usage percentage. My cutoff is going to be a 5% increase in usage. That cutoff is approximately 20 innings pitched (and more) from May until the end of the year. If it seems low to you, keep in mind that it helps us include every trade deadline acquisition in the sample.

There are also pitchers whose usage increased dramatically in August until the end of the season- mid-season call-ups, lower leverage trade deadline targets, and simply relievers who managed to prove themselves by July when they hadn’t done so prior. I’m going to duplicate my process from above using through July as the first set of data and from the end of July until the end of the year as the second set. Any pitcher gaining 5% or more in usage in that group will also be included along with the post-April gainers. Now we have any pitcher who increased his usage either 5% or more from May 1 until the end of the season, or from August 1 until the end of the season. I’ll call this group “Additions”, and our remaining group are “Holdovers.” Technically, the “Additions” group includes a few pitchers who pitched in April, but they saw their usage increase significantly.

For some reason, I couldn’t find LI specifically. I found lots of variants (pLI, exLI, inLI) but not just plain ol’ Leverage Index. That being the case, you won’t see WPA/LI here. That still leaves us with FIP and WPA to compare between our two groups. Here’s how that looks:

The Addition group bests the holdovers in FIP in five of ten seasons, and pretty soundly so from 2011 to 2013. The FIP from bullpen additions from 2009 to 2017, on the aggregate, is 3.64, compared to 3.88 from holdovers. Admittedly, the bullpen additions have only thrown about a third of the innings as the holdovers. Maybe with a larger sample, FIP would tilt back towards the holdovers. But as it stands, the relievers acquired after April by the front office have outperformed the holdover relievers by about a quarter of a run of FIP per 9 IP.

It gets really interesting when we look at WPA. As you may know, WPA is a counting stat. Having more innings (or more plate appearances) makes a player more likely to rack up a higher WPA, at least in terms of the sample sizes we’re discussing here. In theory, that should give the holdovers a sizable advantage. From May 1 until the end of the season, they’ve posted three times the innings of the new additions. But that’s not what has happened. Even with fewer innings, the additions have bested the holdovers in WPA in seven of ten seasons. The collective WPA for additions over this timeframe is 10.81, compared to 9.73 for the holdovers. They’ve outperformed the holdovers. Using either of these measures- FIP or WPA- the additions outperform the holdovers.

There’s one last piece of this puzzle. How do these FIP gains for additions (or increased usage relievers) compare to the rest of the league? After all, it seems intuitive that pitchers receiving increased usage (or are acquired via trade) later in the year will be better than those whose usage remains the same or decreases after April. League-wide, FIP for additions is 3.74, compared to 3.95 for holdovers. That’s a good sign- the gap between Cardinals additions compared to Cardinals holdovers is a little bit higher than league average. What if we look at it by individual team?

Where it loses some steam is their ranking in difference. The Cardinals’ 0.23 FIP gap between additions and holdovers is 17th in the league. And as we can see from the previous FIP/WPA graph, it’s trending downward. Put another way, the Cardinals are generally effective at identifying pitchers who can help the bullpen after April, as judged by FIP. But they aren’t uniquely effective at it compared to the rest of the league. In fact, they’re right around average, and it’s trending downward. I’d love to include WPA here but it would require a massive amount of work. Given that WPA is somewhat volatile anyway, FIP is the better choice.

Putting this puzzle together, the Cardinals’ in-season moves have been effective in upgrading the bullpen after April, though not uniquely effective compared to the rest of the league. We also know that the franchise is tracking around average in WPA and WPA/LI, and well below it in FIP, for post-April bullpens. In other words, their April holdovers have muddied the overall value of the bullpen gains by in-season additions. We know that they’re swapping out 28.8% of innings after April, which is just a tick below league average. All of this implies that the front office may be able to find some marginal value by using a quicker trigger in jettisoning April holdovers sooner in the year, and spackling in with their more effective in-season additions.