Viva El Birdos: An SB Nation Community

Navigation: Jump to content areas:


Sports blogs for fans, by fans.
New Blog: Sounder At Heart for Seattle Sounders Fans!

The Challenge Rating System

Today's post was written by Solanus, with a little editing by me. I asked him to write up a description of his Challenge Rating system, which he developed as a way to measure strength of schedule. He e-mailed it to me last night, so I'm posting it for him this morning. Thanks, Solanus, for writing this up; the floor is now yours.

We've heard everybody complain that such-and-such team has bloated its record by piling up wins against easy opponents, or that all of our team's opponents get hot just in time to face us. These statements are usually backed up by cherry-picked examples or partially remembered anecdotes. Wanting to get a clearer picture of what was actually happening (and trying to kill significant dead-time at work without resorting to HR-alert-worthy options), I came up with a system to determine the strength of schedule for every team in the majors.

The Challenge Rating (CR) is a weighted combination of two factors: 1) the opponent's overall winning percentage, and 2) the opponent's winning percentage in games played immediately before and after their series against your team. The first factor measures the opponent's overall skill, and the second estimates the level at which the opponent is playing at the time your team happens to run into them. Teams go through hot and cold spells; they endure injuries and make roster moves which can alter their "degree of difficulty" at various points in the season. Just consider the 2006 Cardinals: They had the second-best record in the NL for most of the year, which would make them seem a difficult foe, but if your team caught them in August --- when they were playing without Eckstein and Edmonds, and with the injured Isringhausen still at closer --- they were not too hard to knock off.

An average CR is 1000; CRs higher than 1000 denote difficult opponents, while those lower than 1000 denote easy ones. An opponent with a CR of 1200 might be a .600 team that's playing at its normal level, or it could be a .500 team that's in the midst of a hot streak; even a .450 team can have a CR of 1200 if it's playing well enough. A CR of 800 would be just the opposite --- a lousy team playing at its usual level, an average team that's playing poorly, or a good team that's hit the skids.

I won't bore everyone with all the math here; for those of you who care, the formula is at the bottom of this post.

Over the course of a season, the average CR of a team's opponents will tend to even out quite a bit --- but not entirely. Some teams will get unlucky and run into more hot opponents than cold ones; others will catch a break and tend to run into teams when they're playing at their worst. This can be pretty revealing. When I first described this system at VEB last season, the Cardinals were 15 games over .500, but the Challenge Rating system showed that the Cardinals' schedule in the first two months of the season had been (as I termed it) "criminally easy." That suggested to me that the team's overall record was partly an illusion: "Collectively they're just a .500 team waiting for Albert to do something spectacular to save the day or for Carp to pitch a gem." That turned out to be a pretty accurate description of the 2006 Cardinals.

What does the CR system say this year? Through their first 41 games, the Cardinals had faced opponents with an average Challenge Rating of 1025, making theirs the 7th most difficult schedule among the 16 NL teams --- slightly tougher than average. The toughest belongs to the Nationals, at 1104. By contrast, the Astros have had the easiest road so far at 868. (The best/worst in the AL: Cleveland at 885, Kansas City at 1142.) Looking deeper into the Cardinals' record, it is fairly apparent that we aren't necessarily as terrible as our record suggests, but we're not any good either:

  • Against bad opponents (CR of 850 or less), the Cards are 4-4. Could be better, but ... meh.
  • Against average opponents (CR between 850 and 1150), the Cards are 11-12. OK, we're an average team playing .500 against other average teams ... meh again.
  • Against good opponents (CR of 1150 or higher), the Cards are 1-9. Yeah, you read that right --- one and FRICKIN' nine. That 1-9 came against the Mets in the opening series (0-3), the two sets against the Brewers (1-4), and the two-game sweep against the Giants.
That's right, the Giants, a 21-22 team overall, were among the Cardinals' toughest challenges this year. That's because they were smoking hot at the time the Cards hit them --- they came into the series on a mild hot streak (won 3 out of 5) and then went 5-0 after the Cardinals left town. Because they were 8-2 before/after they played the Cardinals, the Giants had a high CR of 1270 for that series --- they were an average team that happened to be playing very well when they came up on the Cards' schedule.

By contrast, when the Astros played them last week, the Giants didn't pose nearly as difficult a challenge; San Fran went 2-3 coming into the series and has gone 1-2 since, so Houston caught them at a good time: The Giants' CR for that series was only 855. The Astros took 2 out of 3.

Here's a complete look at the NL Central to date. The "weak" column covers games against opponents with a CR of 850 or lower; "average" applies to opponents between 850 and 1150; and "hard" covers foes at 1150 or higher:

AVG
CR
W-L vs weak vs avg vs hard
Brewers 915 27-17 15-8 9-6 3-3
Astros 868 21-22 10-13 7-4 4-5
Cubs 1016 20-22 9-8 4-6 7-8
Pirates 1028 19-24 2-7 15-10 2-7
Cardinals 1025 16-25 4-4 11-12 1-9
Reds 1020 17-27 2-4 11-15 4-8

The Cards have played only 8 games against easy opponents all year, while the Brewers and Astros have played 23 apiece. The Cards have an easy series this week; the Pirates carry a CR of 796 into the series, the third-lowest CR the Cardinals have run into this season. But the Cardinals' own CR is even worse than Pirates' --- an abysmally low 390 --- thanks to the five-game losing streak we're riding. We're a bad team playing badly; there is no easier challenge. I really hate to say it, but right now the Cardinals are as a big a slump-buster as I've seen in a while. A win against us right now doesn't mean squat. We're playing like crap, so the opposition should expect to win.

The Math:

  1. Calculate the team's winning percentage in the 5 games before and 5 games after the series in question.
  2. Add 0.5.
  3. Multiply the sum by the team's overall winning percentage times 2.
  4. Multiply by 1000 to get a nice round number.
To use an example, let's calculate the CR for the Cards' last opponent, the Tigers:
  1. Calculate the team's winning percentage in the 5 games before and 5 games after the series in question. The Tigers went 1-4 in the five games before this series, and are 0-0 in the five games after; their winning percentage is .200, or 0.2. (After the Tigers play their next five games, the Challenge Rating score may change.)
  2. Add 0.5. 0.2 plus 0.5 = 0.7.
  3. Multiply the sum by the team's overall winning percentage times 2.The Tigers' overall winning percentage is .628, so the equation is 0.7 x (.628 x 2), or 0.7 x 1.256 = .8792.
  4. Multiply by 1000 to get a nice round number. .8792 x 1000 = 879.2, or 879. That was the Tigers' CR for that series.

0 recs  |  Comment 46 comments

Story-email Email Printer Print

Comments

Display:

Is it an anomaly
that the WORST teams have the HIGHEST CR?

Makes me wonder about the stat...

by silent_bob on May 22, 2007 9:36 AM EDT reply actions   0 recs

I think
that this column shows the average CR of the opponents each team has played.  Otherwise it doesn't make any sense.

by eglasier on May 22, 2007 9:53 AM EDT up reply actions   0 recs

it makes sense
KC has the highest CR because they don't get to play themselves

by tdawg on May 22, 2007 10:03 AM EDT up reply actions   0 recs

that's
not necessarily true.  Because in that case, if it just mattered if they played themselves then the Brewers would have the highest CR in the Central.  The point of this is that it includes factors not typically included in strength of schedule; in this case whether a team is facing a "hot" opponent.

by eglasier on May 22, 2007 10:10 AM EDT up reply actions   0 recs

correction
Lower, not higher, duh

by eglasier on May 22, 2007 10:11 AM EDT up reply actions   0 recs

The Challenge Rating that you see there
Is the average level of competition that each team has FACED, not that they presented to other teams. If you wanted to know the average challenge level that they gave their opponents, you would substitute their overall win % for the recent win % in the equation above. So, this year the Cardinals would offer a Challenge Rating of 694 [((.390+.5)*(.390*2))*1000].

There is a bit of a correlation, though, between a team's record and their average CR faced. The best teams will tend to have lower CR's simply because they don't have to face themselves; the opposite is true for the bad franchises.

Everywhere is within walking distance if you have the time.

by Solanus on May 22, 2007 10:04 AM EDT up reply actions   0 recs

Ow! My Brain!
Pretty cool stuff, Solanus. And a reminder why I majored in history.
Everybody off the bandwagon!

by Alxfritz on May 22, 2007 9:56 AM EDT reply actions   0 recs

If
the Cardinals could have just played the Cardinals a few times by now, their average CR would be lower.  And if the Brewers had to play themselves, theirs would be higher, no?

by bobeans on May 22, 2007 9:58 AM EDT reply actions   0 recs

Above my head
this stuff is too heavy for me.  let's just line up between the white lines and play ball.

http://whiteyball.wordpress.com

by whiteyball on May 22, 2007 10:02 AM EDT reply actions   0 recs

this is pretty cool
it's not an exact science of course but it stands to reason that how a team is playing at the time should be factored into schedule difficulty.  I really like this tool.  It also tells me that the Pirates and Nationals are probably looking forward to playing us as much as we're looking forward to playing them.  

Is there any way that home field could be taken into account?  Home teams tend to have an advantage over road teams so I wonder what value home field might have in this system.

by chuckb on May 22, 2007 10:14 AM EDT reply actions   0 recs

If
you check out the Pittsburgh Post-Gazette, you will find out that is true.  

by cardsgirl95 on May 22, 2007 10:27 AM EDT up reply actions   0 recs

Brilliant
because of the simplicity if nothing else.  I have a feeling that the ratings may be self fulfilling to some degree, as alluded to by tdawg.

It also buys into the fact that baseball players can play with some sort of momentum.  Taking a step back, you might say that last series starting pitchers had more to do with the success of the team rather than "momentum" or just the team being "hot".  If that's the case, CR is a bad indicator in as you'll probably miss one or two of those starters that made the team "hot" in the first place.  Interesting though, very interesting.  Gets me thinking if nothing else.  Extra Kudos for drawing no soccer parallels.    

by Jonathan23 on May 22, 2007 10:16 AM EDT reply actions   0 recs

Brewers and Pirates
It says something about the Brewers though.  They have faced 23 teams in the weak column and fared well, while they have only faced six hard opponents and played .500 against them.  That would seem to indicate that their record is indeed inflated and their performance last week bears that out. Interesting that the Pirates have done poorly against weak teams and played well against average teams.  That could make for an interesting series against them this week.  

by cardsgirl95 on May 22, 2007 10:16 AM EDT reply actions   0 recs

I wouldn't say that
Good teams probably should go about .500 against each other, and definitely should beat the bad/ slumping teams. The Brewers may not be quite as good as record was a week ago, but I don't think there's anything in these numbers that suggests a major fall to earth for them.

Unless you're saying that they've basically gotten lucky in that they haven't faced very many hot teams...

by taiko on May 22, 2007 1:12 PM EDT up reply actions   0 recs

I think
that I was saying they have been lucky to feast on weaker teams for so many games and now that the schedule has toughened up for them, their performance reflects that - the Brewers are playing .500.  

by cardsgirl95 on May 22, 2007 8:21 PM EDT up reply actions   0 recs

Slump busting
This quote says it all: "...really hate to say it, but right now the Cardinals are as a big a slump-buster as I've seen in a while."

Solanus, what would happen if you looked at the challenge ratings of different teams before and after playing the Cardinals.  If we truely were slump busters, would teams improve imediately after playing us?

by Zubin on May 22, 2007 10:28 AM EDT reply actions   0 recs

I like it
its still a pretty basic way of looking at the schedule, but I like it.  I think the 10 games around are weighted too much though.  

here are a few thoughts I had to improve (but it would also take away from the simplicity, which is a very nice aspect)

  1. instead of win loss, use runs scored and runs allowed.  
  2. somehow incorporate the pitchers faced during the series (i.e. missing oswalt is a big advantage when facing the astros)

by PGeorge on May 22, 2007 10:29 AM EDT reply actions   0 recs

the only change i would suggest
is this: when calculating the overall won-loss percentage, subtract the games played against your team. so, far example, in calculating the overall w-l for any opponent that's playing the cardinals, subtract the opponent's w-l vs the cardinals.

the pirates' overall w-l, for example, would be calculated as 17-21 --- because they're 19-24 overall, minus 2-3 vs the cardinals.

that would eliminate some noise. but overall i like the simplicity of the formula and wouldn't tinker with it too much. as i read this, it's only meant to be a shorthand look at schedule strength, rather than any sort of precisely calibrated instrument. as an "at a glance" tool, i think it functions well and is very useful.

by lboros on May 22, 2007 10:46 AM EDT up reply actions   0 recs

Ultra-simple
This was never meant as something that was to be a SABR-strength metric, that would stand up to a lot of scrutiny and be a predictive tool. It was just something I did, by hand, to kill some hours at work. I see a lot of things that I could do to make this a lot more useful (home/road, pitchers, runs scored/allowed, etc.), but I don't have the time/drive/energy to do more than this manually or the computer skill/power to collect all the data.

If someone wants to run with this, develop some macros to gather all of the necessary info, and produce something truly useful, more power to them. I only have so much time in a given day to goof off and I'm spread terribly thin already.

Everywhere is within walking distance if you have the time.

by Solanus on May 22, 2007 11:21 AM EDT up reply actions   0 recs

A great idea sparks others
I think the fact that your post has sparked so many of us to speculate about how this metric might be tweaked speaks to what a solid concept it is.  So I hope this and other similar posts are seen in the spirit of shared inquiry, rather than as criticism.

As you've insightfully identified, an opponent's W-L record isn't the best measure of the strength of that opponent.  Therefore, it seems this metric would also be stronger if instead of actual W-L record something along the lines of Bill James' Pythagorean Therorem, or some other measure of expected wins, was used.

This would partially (not totally) eliminate the variable of a team who happened to play weak competition before and after a given opponent.  In other words, a team that was lucky would be less likely to appear as "hot."

I would also dispute Jonathan23's earlier post that this metric buys into the notion of players playing better with momentum.  It's really giving you a snapshot of the opponent at the time the games were played.  A team that's lousy in April may be very good in August, and this system takes that into account.

Thanks again, Solanus.

by bgodar on May 22, 2007 12:20 PM EDT up reply actions   0 recs

Yadier Molina
I meant to do this last week, but I guess more ABs doesn't hurt the same size:

Over Yadi's last 200 ABs:

57 Hits
3 HRs
9 2B
22 RBI
21 BB
.285 BA
.375 SLG
.222 OBP

That percentages (SLG and OBP) are fungable because I can't find the total number of plate appearences over that time.

But I think that is damn close.

Anyways, Molina has shown a remarkable improvement over a larger sample size now.  Maybe this guy is a legit MLB hitter.

We aren't very good.

by Hardcore Legend on May 22, 2007 10:58 AM EDT reply actions   0 recs

That OBP can't be right
because over that span, Yadi has posted OBPs of .279, .424 and .355.
We aren't very good.

by Hardcore Legend on May 22, 2007 11:13 AM EDT up reply actions   0 recs

The online calculator I was using is messed up
OBP:

(H+BB+HBP)(AB+BB+HBP+SF)

Yadi
(57+21+2)
(200+21+2+2)

(80)/(225)

.355 OBP

We aren't very good.

by Hardcore Legend on May 22, 2007 11:21 AM EDT up reply actions   0 recs

I was gonna say...
Impossible for OBP to be lower than BA, right?
In Albert we trust.

by Mr Redbird on May 22, 2007 12:24 PM EDT up reply actions   0 recs

Actually...
no, it is extremely unlikely, but possible.

by Zubin on May 22, 2007 2:55 PM EDT up reply actions   0 recs

your
obp has to be wrong...

by eglasier on May 22, 2007 11:14 AM EDT up reply actions   0 recs

Fixed it now
Should be right.
We aren't very good.

by Hardcore Legend on May 22, 2007 11:32 AM EDT up reply actions   0 recs

no worries
I do the same shit all the time

by eglasier on May 22, 2007 11:58 AM EDT up reply actions   0 recs

Prior Years
Any chance we could see similar numbers for prior years?

Also, good teams take advantage of weak and avg opponents, and play around .500 against hard opponents.  The Cards obviously arent doing any of that.

by UNCDubya on May 22, 2007 11:13 AM EDT reply actions   0 recs

That was a hallmark
of the 2004 and 2005 Cardinals' teams.  How I long for the days...
In Albert we trust.

by Mr Redbird on May 22, 2007 12:25 PM EDT up reply actions   0 recs

I have 2005 and 2006
2005 pretty much followed the winning "formula": 21-18 against good competition, 43-24 vs mids, 36-20 facing the dregs. Their full season average CR was 968, the lowest in the NL, but only slightly easier than SD and MIL, two .500 squads. (The Yankees had a sinfully easy schedule - 937.)
2006 was hit-or-miss: we did fairly well against the average foes (48-37), but stunk against good foes (11-19) and battled bad teams to a draw (24-22). The final tally for CR was 974, 5th easiest in the NL.

Looking at the majors as a whole, you notice some interesting things.

  • The Cubs in '05, when facing a CR of 1.28 or higher, were 16-8. A lot of those times it was the Cardinals, but they had a real tendency to play up to their competition. But they also stunk when it came to the rest of the league (63-75).
  • In 2006, the Astros played 25 games against tough opponents. The Devil Rays played 74 games the same way, posting an average CR of 1070 for the entire season.
There are many other tidbits of info to glean from the total works. If anyone is interested in looking at the source data, just send me an e-mail.
Everywhere is within walking distance if you have the time.

by Solanus on May 22, 2007 1:05 PM EDT up reply actions   0 recs

Prior years?
I would imagine most Prior years would start off promising and end with a complete tanking.
The outfield is deep and playing him straight-away and the infield is the same except first, second, third and short are playing him to pull. -- Mike Shannon

by MUTiger on May 22, 2007 4:14 PM EDT up reply actions   0 recs

Solanus,
I commend the work, I think this is an interesting/useful stat on a fairly basic level.  I understand the "spreading thin" concept.  Last September/October I tried to post an interesting stat occasionaly but so far this yr i've stuck to reading (and attempting to get some work done).

by toris34 on May 22, 2007 11:38 AM EDT reply actions   0 recs

Reyes
Interesting article on his "bad luck" this season:

http://stlouis.cardinals.mlb.com/news/article.jsp?ymd=20070522&content_id=1979280&vkey=news_ stl&fext=.jsp&c_id=stl

I hadn't seen a lot of discussion on "strand rate" before.  Is it fair to expect that he will "progress" to the norm going forward?  Maybe similar thinking to BABIP, although I would guess that one of the big differences from one pitcher to another is their ability to strand runners.  This article would say that it's more random than skill differential.

by wildman on May 22, 2007 12:50 PM EDT reply actions   0 recs

I saw that too, and just posted a diary
The authors basically say that it's a matter of luck, but I wonder if something's wrong with his delivery out of the stretch. His splits between pitching with the bases empty versus with runners on are gaping.

by taiko on May 22, 2007 1:09 PM EDT up reply actions   0 recs

Or perhaps young Reyes
is particularly hittable out of the stretch>

by Zubin on May 22, 2007 2:52 PM EDT up reply actions   0 recs

Could be reduced velocity
from the stretch, not being able to use the big leg kick, and when he tries to throw that 93 mph letter high fastball, it is only 91 mph and closer to the belt.
We aren't very good.

by Hardcore Legend on May 22, 2007 3:01 PM EDT up reply actions   0 recs

also could be pitch selection
we're gathering up our pitch charts for the first 1/4 of the season --- maybe reyes tends to get "cute" with his off-speed stuff when men are on base, falls behind in the count, then has to groove a fastball . . . . . just a theory, but if that's true our charts will bear it out.

by lboros on May 22, 2007 4:47 PM EDT up reply actions   0 recs

That a very good point
in his last start, I remember making a remark in the game thread that after he gives up the runs, or the homerun, or whatever failure he falls into, he usually comes back like a bulldog and fills the strikezone up with good stuff.

Maybe he gets too nervous and defensive, instead of 'you hit it, you name it'.

We aren't very good.

by Hardcore Legend on May 22, 2007 5:18 PM EDT up reply actions   0 recs

I'd agree with that.
Also, maybe it's because when he gets a man on first base he throws the 2 seamer to try to induce a doupble play ball. We all know how well that pitch works for him.

by Carps on May 22, 2007 5:33 PM EDT up reply actions   0 recs

juan bats cleanup tonight
Eckstein
Duncan
Pujols
Encarnacion
Rolen
Molina
Edmonds
Miles
Wainwright
Call up Ankiel!!

by jeff abs on May 22, 2007 5:53 PM EDT reply actions   0 recs

Wow, I'd reconstruct that lineup
based on recent performance like this:

Eckstein
Edmonds
Pujols
Duncan
Molina
Rolen
Miles
Wainwright

You honestly could flip Rolen and Edmonds, as Rolen actually still is 'seeing' the ball well as far as drawing walks.

Duncan needs to be hitting behind Pujols right now.  Juan 'infield single' Encarnacion provides nothing as far as protection.

I hope it all works out, of course, but how many games do we have to lose before our 2nd best power hitter gets moved to the #4 spot?

Are we the only team in baseball that has their best two power hitters hitting #2 and #3 and our other good 'hitter' hitting 6th?

Small sample sizes and all, I know.  But, like I pointed out earlier, over his last 200 ABs, Molina has a .285 BA and almost a .730 OPS.  I doubt there are 4 guys on this team that over their last 200 ABs have anything like that.

We aren't very good.

by Hardcore Legend on May 22, 2007 6:32 PM EDT up reply actions   0 recs

I think...
it has something to do with Juan's past success against Duke.

At some point we have to come to grips with the fact that there's no good place in the lineup for a lot of these hitters.

by guayzimi on May 22, 2007 6:41 PM EDT up reply actions   0 recs

Why not try batting Rolen 2nd?
Maybe hitting in front of Pujols can help boost his average and get his confidence back... still gives you power in the 2nd spot (which TLR likes) and he's  probably a better bunter than Dunc.

If not 2nd then put the guy 7th and bat Yadi 5th.  

by stltrav09 on May 22, 2007 5:59 PM EDT reply actions   0 recs

Craig Wilson
cleared waivers... has there been any talk of interest from Cardinals camp?

by inwaltwetrust on May 22, 2007 6:09 PM EDT reply actions   0 recs

please no!
His numbers with the Yankees and Braves are beyond horrible and his attitude here in Atlanta was poor.  I don't think he adds anything we're looking for - if we're looking for people to return to career norms we already have plenty.

by wildman on May 22, 2007 10:17 PM EDT up reply actions   0 recs

Comments For This Post Are Closed


User Tools

Welcome to the Internet's #1 St. Louis Cardinals blog.
Start posting about the Cardinals »

Join SB Nation and dive into communities focused on all your favorite teams.

FanPosts

Community blog posts and discussion.

Recommended FanPosts

Stl_ia_card_45_-_dark_small
Jeff Pearlman Thinks Of Hair Clumps When He Thinks Of The Thief McGwire
Black-spider-monkey_small
Losing my religion (w/ baseball)

Recent FanPosts

Small
Can someone explain to me...
Knights-09_small
Disenchanted Blue Jays Fan Looking For A New Team
Painterlance_small
The Holliday Dilemma (Rocks Fan Perpsective)
375830-r1-025-11_011_small
Anybody read Bob Gibson's new book yet?
Flanders_small
Yadi2first
Small
40 Man Question..
Cathybachebay1_small
The current Busine$$ of Baseball...how long can it last?
Avatar_small
VEB CheBird T-Shirt for Sale - Red or Powder Blue, CLEARANCE

+ New FanPost All FanPosts >


Managers

Jack_benny_small DanUpBaby

Editors

Bender1_small azruavatar

Adam1_small chuckb

Trigun_001_small the red baron