Quality starts are a pretty good statistic for telling how consistent a pitcher is.  They often tell a more complete picture than rate stats which can be altered by a couple of really bad, or really good starts.  The problem with quality starts are that they don't really tell how well a pitcher pitched.  With new advanced pitcher evaluating statistics, such as FIP, we can get a much better picture of how well a given pitcher actually pitched if we eliminate non-performance based factors like defense, luck and ballpark. Translating FIP to quality starts, we can see what percentage of the time a pitcher was "really" effective.

My first example is Johan Santana, who led the league in ERA and quality starts.  However, his FIP was a full run higher, meaning that he got "lucky" this year.  His .287 BABIP and 22.4 LD rate give credence to that notion.  So I wanted to see how many "real" quality starts he had.  

I went to Santana's Fangraphs game logs and downloaded them to Excel.  Than I imputed the FIP formula to each of his individual games.

IP	HR	SO	BB	IBB	Game FIP
7	1	8	2	0	3.63
7	0	3	0	0	2.34
6.2	3	7	2	0	7.63
7	1	10	0	0	2.20
7	0	4	1	0	2.49
5.2	2	7	3	0	6.20
6	0	8	4	2	3.49
6	1	5	1	0	4.06
7.2	3	5	2	0	8.20
7	0	1	0	0	2.91
7	1	7	2	0	3.91
7.2	0	6	3	0	2.77
6	0	5	1	0	2.20
7	0	10	3	0	1.63
6	1	5	2	0	4.49
7	1	4	2	1	5.20
6	0	8	4	1	3.06
8	0	6	0	0	1.49
5	0	5	3	0	3.06
4	2	2	3	0	7.63
8	1	4	0	0	3.91
9	1	5	1	0	4.06
6.1	0	3	2	0	3.20
7	1	7	3	0	4.34
7	1	6	2	0	4.20
9	0	7	0	0	1.20
7	0	5	1	0	2.20
6	2	6	3	0	6.49
6	0	10	1	0	0.77
7.1	1	6	3	0	4.63
7	0	4	2	0	2.91
7	0	8	2	0	1.77
8	0	10	2	0	1.20
9	0	9	3	1	2.34

Sorry for the lack of a table, I don't know how to do one on a fanpost (help?).  For FIP quality starts, I basically used the same criteria as normal quality start, so at least 6 innings pitched and at least a league average FIP (4.40).  So Santana really had 26 quality starts instead of 28, which was still good for 76% QS which would be third in the league.  He had a great season and was probably underrated by his FIP.  His overall FIP was aggravated by his two 3 homer starts.

Okay, so my first example failed as the FIP-ERA disparity didn't reflect that much in his actual game by game performance.  My next example is Adam Wainwright, who also had a rather large disparity in his ERA vs. FIP.  He finished with 12 quality starts in 20 games or 60%, but how did he really do?

IP	HR	SO	BB	IBB	Game FIP
8	1	4	1	0	4.34
7	0	6	0	0	1.49
7.2	0	6	2	0	2.34
7	0	6	4	1	3.63
9	3	6	2	0	7.91
6.1	0	5	1	0	2.20
7	0	3	2	0	3.20
6	3	3	1	0	8.34
5.2	0	2	1	0	3.06
7	0	5	2	0	2.63
8	1	8	1	0	3.20
8	1	4	2	0	4.77
5	1	4	1	0	4.34
6	1	4	1	0	4.34
6	0	2	0	0	2.63
5.1	0	7	2	0	2.06
8	1	6	1	0	3.77
4	0	3	2	0	3.20
5	0	5	3	0	3.06
6	0	2	5	0	4.77

Wainwright really had 2 more quality starts by FIP.  Like Santana, he had a couple of 3 homer games (although one of them came in a complete game win) that significantly raised his FIP.  His real QS percentage was 70%, which would be 10th in the league.  When watching WW last year, it seemed like always turned in a good start and kept the team in the.  When he pitched, I felt that we were going to win.  A lot of us feel that he is becoming an ace, and judging by his consistency between starts, that might be true.  

Now lets look at Kyle Lohse.  He finished with a small disparity in his FIP vs. ERA and and 20 quality starts in 33 games.  He parlayed that consistency to a 40 million dollar contract extension.  Going by last years numbers, he earned it, but was he really that that consistent?

IP	HR	SO	BB	IBB	Game FIP
5	0	3	3	0	3.63
7	0	2	1	0	3.06
5.1	0	2	1	0	3.06
7	0	3	0	0	2.34
4	0	1	2	0	3.77
6	0	3	1	0	2.77
6	1	0	2	0	5.91
4	0	4	5	0	4.20
7	1	4	1	0	4.34
6	0	5	0	0	1.77
6	0	5	0	0	1.77
6	1	0	1	0	5.49
5	0	2	0	0	2.63
6	1	3	2	0	5.06
8	1	3	2	0	5.06
6	1	4	2	0	4.77
4	1	2	1	0	4.91
7	0	4	2	0	2.91
7	1	6	2	0	4.20
7	0	3	1	0	2.77
7	1	4	2	0	4.77
8	0	7	0	0	1.20
5	3	2	1	0	8.63
5.1	1	6	3	0	4.63
7	1	5	1	0	4.06
4.1	1	5	0	0	3.63
6	0	2	3	1	4.34
6	0	3	2	0	3.20
8	1	6	1	0	3.77
6	0	7	2	0	2.06
5	0	5	2	1	3.06
6.2	0	5	2	0	2.63
6	2	3	1	1	6.91

Overall he only had 16 real quality starts.  4 less than he had with the original measure.  His QS percentage drops to a Jo-El esque 48%.  Yuck.  It turns out that he really wasn't that consistent at all.  You'll notice that he had a lot of ~5 inning games in which he struggled, but was able to not implode.  While that may be an admirable quality in a back of the rotation starter, it simply are not acceptable in your #2.  We re-he-healy need Carp back.

So um... what do you guys think?  Would there be a better way to measure what a quality start instead of 6 innings and above average FIP?  Do you want to see any other pitchers on the staff (It only takes me a couple of minutes to do)?