The Challenge Rating System
Today's post was written by Solanus, with a little editing by me. I asked him to write up a description of his Challenge Rating system, which he developed as a way to measure strength of schedule. He e-mailed it to me last night, so I'm posting it for him this morning. Thanks, Solanus, for writing this up; the floor is now yours.
We've heard everybody complain that such-and-such team has bloated its record by piling up wins against easy opponents, or that all of our team's opponents get hot just in time to face us. These statements are usually backed up by cherry-picked examples or partially remembered anecdotes. Wanting to get a clearer picture of what was actually happening (and trying to kill significant dead-time at work without resorting to HR-alert-worthy options), I came up with a system to determine the strength of schedule for every team in the majors.
The Challenge Rating (CR) is a weighted combination of two factors: 1) the opponent's overall winning percentage, and 2) the opponent's winning percentage in games played immediately before and after their series against your team. The first factor measures the opponent's overall skill, and the second estimates the level at which the opponent is playing at the time your team happens to run into them. Teams go through hot and cold spells; they endure injuries and make roster moves which can alter their "degree of difficulty" at various points in the season. Just consider the 2006 Cardinals: They had the second-best record in the NL for most of the year, which would make them seem a difficult foe, but if your team caught them in August --- when they were playing without Eckstein and Edmonds, and with the injured Isringhausen still at closer --- they were not too hard to knock off.
An average CR is 1000; CRs higher than 1000 denote difficult opponents, while those lower than 1000 denote easy ones. An opponent with a CR of 1200 might be a .600 team that's playing at its normal level, or it could be a .500 team that's in the midst of a hot streak; even a .450 team can have a CR of 1200 if it's playing well enough. A CR of 800 would be just the opposite --- a lousy team playing at its usual level, an average team that's playing poorly, or a good team that's hit the skids.
I won't bore everyone with all the math here; for those of you who care, the formula is at the bottom of this post.
Over the course of a season, the average CR of a team's opponents will tend to even out quite a bit --- but not entirely. Some teams will get unlucky and run into more hot opponents than cold ones; others will catch a break and tend to run into teams when they're playing at their worst. This can be pretty revealing. When I first described this system at VEB last season, the Cardinals were 15 games over .500, but the Challenge Rating system showed that the Cardinals' schedule in the first two months of the season had been (as I termed it) "criminally easy." That suggested to me that the team's overall record was partly an illusion: "Collectively they're just a .500 team waiting for Albert to do something spectacular to save the day or for Carp to pitch a gem." That turned out to be a pretty accurate description of the 2006 Cardinals.
What does the CR system say this year? Through their first 41 games, the Cardinals had faced opponents with an average Challenge Rating of 1025, making theirs the 7th most difficult schedule among the 16 NL teams --- slightly tougher than average. The toughest belongs to the Nationals, at 1104. By contrast, the Astros have had the easiest road so far at 868. (The best/worst in the AL: Cleveland at 885, Kansas City at 1142.) Looking deeper into the Cardinals' record, it is fairly apparent that we aren't necessarily as terrible as our record suggests, but we're not any good either:
- Against bad opponents (CR of 850 or less), the Cards are 4-4. Could be better, but ... meh.
- Against average opponents (CR between 850 and 1150), the Cards are 11-12. OK, we're an average team playing .500 against other average teams ... meh again.
- Against good opponents (CR of 1150 or higher), the Cards are 1-9. Yeah, you read that right --- one and FRICKIN' nine. That 1-9 came against the Mets in the opening series (0-3), the two sets against the Brewers (1-4), and the two-game sweep against the Giants.
By contrast, when the Astros played them last week, the Giants didn't pose nearly as difficult a challenge; San Fran went 2-3 coming into the series and has gone 1-2 since, so Houston caught them at a good time: The Giants' CR for that series was only 855. The Astros took 2 out of 3.
Here's a complete look at the NL Central to date. The "weak" column covers games against opponents with a CR of 850 or lower; "average" applies to opponents between 850 and 1150; and "hard" covers foes at 1150 or higher:
| AVG CR |
W-L | vs weak | vs avg | vs hard | |
|---|---|---|---|---|---|
| Brewers | 915 | 27-17 | 15-8 | 9-6 | 3-3 |
| Astros | 868 | 21-22 | 10-13 | 7-4 | 4-5 |
| Cubs | 1016 | 20-22 | 9-8 | 4-6 | 7-8 |
| Pirates | 1028 | 19-24 | 2-7 | 15-10 | 2-7 |
| Cardinals | 1025 | 16-25 | 4-4 | 11-12 | 1-9 |
| Reds | 1020 | 17-27 | 2-4 | 11-15 | 4-8 |
The Cards have played only 8 games against easy opponents all year, while the Brewers and Astros have played 23 apiece. The Cards have an easy series this week; the Pirates carry a CR of 796 into the series, the third-lowest CR the Cardinals have run into this season. But the Cardinals' own CR is even worse than Pirates' --- an abysmally low 390 --- thanks to the five-game losing streak we're riding. We're a bad team playing badly; there is no easier challenge. I really hate to say it, but right now the Cardinals are as a big a slump-buster as I've seen in a while. A win against us right now doesn't mean squat. We're playing like crap, so the opposition should expect to win.
The Math:
- Calculate the team's winning percentage in the 5 games before and 5 games after the series in question.
- Add 0.5.
- Multiply the sum by the team's overall winning percentage times 2.
- Multiply by 1000 to get a nice round number.
- Calculate the team's winning percentage in the 5 games before and 5 games after the series in question. The Tigers went 1-4 in the five games before this series, and are 0-0 in the five games after; their winning percentage is .200, or 0.2. (After the Tigers play their next five games, the Challenge Rating score may change.)
- Add 0.5. 0.2 plus 0.5 = 0.7.
- Multiply the sum by the team's overall winning percentage times 2.The Tigers' overall winning percentage is .628, so the equation is 0.7 x (.628 x 2), or 0.7 x 1.256 = .8792.
- Multiply by 1000 to get a nice round number. .8792 x 1000 = 879.2, or 879. That was the Tigers' CR for that series.
0 recs |
46 comments
Comments
Is it an anomaly
Makes me wonder about the stat...
by silent_bob on May 22, 2007 9:36 AM EDT reply actions 0 recs
it makes sense
by tdawg on May 22, 2007 10:03 AM EDT up reply actions 0 recs
that's
by eglasier on May 22, 2007 10:10 AM EDT up reply actions 0 recs
The Challenge Rating that you see there
There is a bit of a correlation, though, between a team's record and their average CR faced. The best teams will tend to have lower CR's simply because they don't have to face themselves; the opposite is true for the bad franchises.
by Solanus on May 22, 2007 10:04 AM EDT up reply actions 0 recs
Ow! My Brain!
by Alxfritz on May 22, 2007 9:56 AM EDT reply actions 0 recs
If
by bobeans on May 22, 2007 9:58 AM EDT reply actions 0 recs
Above my head
by whiteyball on May 22, 2007 10:02 AM EDT reply actions 0 recs
this is pretty cool
Is there any way that home field could be taken into account? Home teams tend to have an advantage over road teams so I wonder what value home field might have in this system.
by chuckb on May 22, 2007 10:14 AM EDT reply actions 0 recs
If
by cardsgirl95 on May 22, 2007 10:27 AM EDT up reply actions 0 recs
Brilliant
It also buys into the fact that baseball players can play with some sort of momentum. Taking a step back, you might say that last series starting pitchers had more to do with the success of the team rather than "momentum" or just the team being "hot". If that's the case, CR is a bad indicator in as you'll probably miss one or two of those starters that made the team "hot" in the first place. Interesting though, very interesting. Gets me thinking if nothing else. Extra Kudos for drawing no soccer parallels.
by Jonathan23 on May 22, 2007 10:16 AM EDT reply actions 0 recs
Brewers and Pirates
by cardsgirl95 on May 22, 2007 10:16 AM EDT reply actions 0 recs
I wouldn't say that
Unless you're saying that they've basically gotten lucky in that they haven't faced very many hot teams...
by taiko on May 22, 2007 1:12 PM EDT up reply actions 0 recs
I think
by cardsgirl95 on May 22, 2007 8:21 PM EDT up reply actions 0 recs
Slump busting
Solanus, what would happen if you looked at the challenge ratings of different teams before and after playing the Cardinals. If we truely were slump busters, would teams improve imediately after playing us?
by Zubin on May 22, 2007 10:28 AM EDT reply actions 0 recs
I like it
here are a few thoughts I had to improve (but it would also take away from the simplicity, which is a very nice aspect)
- instead of win loss, use runs scored and runs allowed.
- somehow incorporate the pitchers faced during the series (i.e. missing oswalt is a big advantage when facing the astros)
by PGeorge on May 22, 2007 10:29 AM EDT reply actions 0 recs
the only change i would suggest
the pirates' overall w-l, for example, would be calculated as 17-21 --- because they're 19-24 overall, minus 2-3 vs the cardinals.
that would eliminate some noise. but overall i like the simplicity of the formula and wouldn't tinker with it too much. as i read this, it's only meant to be a shorthand look at schedule strength, rather than any sort of precisely calibrated instrument. as an "at a glance" tool, i think it functions well and is very useful.
by lboros on May 22, 2007 10:46 AM EDT up reply actions 0 recs
Ultra-simple
If someone wants to run with this, develop some macros to gather all of the necessary info, and produce something truly useful, more power to them. I only have so much time in a given day to goof off and I'm spread terribly thin already.
by Solanus on May 22, 2007 11:21 AM EDT up reply actions 0 recs
A great idea sparks others
As you've insightfully identified, an opponent's W-L record isn't the best measure of the strength of that opponent. Therefore, it seems this metric would also be stronger if instead of actual W-L record something along the lines of Bill James' Pythagorean Therorem, or some other measure of expected wins, was used.
This would partially (not totally) eliminate the variable of a team who happened to play weak competition before and after a given opponent. In other words, a team that was lucky would be less likely to appear as "hot."
I would also dispute Jonathan23's earlier post that this metric buys into the notion of players playing better with momentum. It's really giving you a snapshot of the opponent at the time the games were played. A team that's lousy in April may be very good in August, and this system takes that into account.
Thanks again, Solanus.
by bgodar on May 22, 2007 12:20 PM EDT up reply actions 0 recs
Yadier Molina
Over Yadi's last 200 ABs:
57 Hits
3 HRs
9 2B
22 RBI
21 BB
.285 BA
.375 SLG
.222 OBP
That percentages (SLG and OBP) are fungable because I can't find the total number of plate appearences over that time.
But I think that is damn close.
Anyways, Molina has shown a remarkable improvement over a larger sample size now. Maybe this guy is a legit MLB hitter.
by Hardcore Legend on May 22, 2007 10:58 AM EDT reply actions 0 recs
That OBP can't be right
by Hardcore Legend on May 22, 2007 11:13 AM EDT up reply actions 0 recs
The online calculator I was using is messed up
(H+BB+HBP)(AB+BB+HBP+SF)
Yadi
(57+21+2)(200+21+2+2)
(80)/(225)
.355 OBP
by Hardcore Legend on May 22, 2007 11:21 AM EDT up reply actions 0 recs
I was gonna say...
by Mr Redbird on May 22, 2007 12:24 PM EDT up reply actions 0 recs
Actually...
by Zubin on May 22, 2007 2:55 PM EDT up reply actions 0 recs
Fixed it now
by Hardcore Legend on May 22, 2007 11:32 AM EDT up reply actions 0 recs
no worries
by eglasier on May 22, 2007 11:58 AM EDT up reply actions 0 recs
Prior Years
Also, good teams take advantage of weak and avg opponents, and play around .500 against hard opponents. The Cards obviously arent doing any of that.
by UNCDubya on May 22, 2007 11:13 AM EDT reply actions 0 recs
That was a hallmark
by Mr Redbird on May 22, 2007 12:25 PM EDT up reply actions 0 recs
I have 2005 and 2006
2006 was hit-or-miss: we did fairly well against the average foes (48-37), but stunk against good foes (11-19) and battled bad teams to a draw (24-22). The final tally for CR was 974, 5th easiest in the NL.
Looking at the majors as a whole, you notice some interesting things.
- The Cubs in '05, when facing a CR of 1.28 or higher, were 16-8. A lot of those times it was the Cardinals, but they had a real tendency to play up to their competition. But they also stunk when it came to the rest of the league (63-75).
- In 2006, the Astros played 25 games against tough opponents. The Devil Rays played 74 games the same way, posting an average CR of 1070 for the entire season.
by Solanus on May 22, 2007 1:05 PM EDT up reply actions 0 recs
Prior years?
by MUTiger on May 22, 2007 4:14 PM EDT up reply actions 0 recs
Solanus,
by toris34 on May 22, 2007 11:38 AM EDT reply actions 0 recs
Reyes
I hadn't seen a lot of discussion on "strand rate" before. Is it fair to expect that he will "progress" to the norm going forward? Maybe similar thinking to BABIP, although I would guess that one of the big differences from one pitcher to another is their ability to strand runners. This article would say that it's more random than skill differential.
by wildman on May 22, 2007 12:50 PM EDT reply actions 0 recs
I saw that too, and just posted a diary
by taiko on May 22, 2007 1:09 PM EDT up reply actions 0 recs
Or perhaps young Reyes
by Zubin on May 22, 2007 2:52 PM EDT up reply actions 0 recs
Could be reduced velocity
by Hardcore Legend on May 22, 2007 3:01 PM EDT up reply actions 0 recs
also could be pitch selection
by lboros on May 22, 2007 4:47 PM EDT up reply actions 0 recs
That a very good point
Maybe he gets too nervous and defensive, instead of 'you hit it, you name it'.
by Hardcore Legend on May 22, 2007 5:18 PM EDT up reply actions 0 recs
I'd agree with that.
by Carps on May 22, 2007 5:33 PM EDT up reply actions 0 recs
juan bats cleanup tonight
Duncan
Pujols
Encarnacion
Rolen
Molina
Edmonds
Miles
Wainwright
by jeff abs on May 22, 2007 5:53 PM EDT reply actions 0 recs
Wow, I'd reconstruct that lineup
Eckstein
Edmonds
Pujols
Duncan
Molina
Rolen
Miles
Wainwright
You honestly could flip Rolen and Edmonds, as Rolen actually still is 'seeing' the ball well as far as drawing walks.
Duncan needs to be hitting behind Pujols right now. Juan 'infield single' Encarnacion provides nothing as far as protection.
I hope it all works out, of course, but how many games do we have to lose before our 2nd best power hitter gets moved to the #4 spot?
Are we the only team in baseball that has their best two power hitters hitting #2 and #3 and our other good 'hitter' hitting 6th?
Small sample sizes and all, I know. But, like I pointed out earlier, over his last 200 ABs, Molina has a .285 BA and almost a .730 OPS. I doubt there are 4 guys on this team that over their last 200 ABs have anything like that.
by Hardcore Legend on May 22, 2007 6:32 PM EDT up reply actions 0 recs
I think...
At some point we have to come to grips with the fact that there's no good place in the lineup for a lot of these hitters.
by guayzimi on May 22, 2007 6:41 PM EDT up reply actions 0 recs
Why not try batting Rolen 2nd?
If not 2nd then put the guy 7th and bat Yadi 5th.
by stltrav09 on May 22, 2007 5:59 PM EDT reply actions 0 recs
Craig Wilson
by inwaltwetrust on May 22, 2007 6:09 PM EDT reply actions 0 recs
please no!
by wildman on May 22, 2007 10:17 PM EDT up reply actions 0 recs



















