Thursday, June 10, 2010

The Significance of Strasburg's Debut

There are a couple of easy mistakes to make in forecasting that I often warn about:

- Never be fooled by a small sample.
- A great outcome that gets our attention is not a randomly selected sample.

I'm about to make a couple of exceptions for good reason and I will explain why.  A single game performance can be so outstanding that it is extremely unlikely that it can be accomplished by the vast majority of players.  Also, even a seemingly small sample can be large enough to tell us something in the rarest of cases.

Let me give you an example: Suppose we are standing at a golf course waiting our turn to golf on the first hole.  We are observing a player ahead of us whom we know absolutely nothing about other than what we can visually observe.  He reaches into his bag, pulls out his driver on a hole that's over 300 yards away, takes a practice swing and then confidently swings and we watch as the ball magically soars, lands on the green and rolls right into the hole for a hole in one.

Believe it or not, that sample size of just one means something and tells us quite a bit.  We can't be positive but the chances are extremely high that the golfer we've just seen did not just take the first swing of his life.  It's possible but unlikely.  He may not be a great or a pro or even a top amateur but we've at least narrowed the likely range of skill that he has by observing a single swing.

That the hole in one got our attention is dangerous... We noticed the player because he did something extraordinary the first time we saw him.  Remember Karl "Tuffy" Rhodes hitting 3 home runs on opening day a number of years back?  Better yet, how about Mark Whiten's 4 home run game?  They certainly got our attention and if we were to examine the odds of players doing what these players did, they would be low and we could easily make false conclusions about their ability.

But there is a point where someone does something that is so unlikely that it is more likely that they have unusual ability than unusual luck.  That brings me to Stephen Strasburg's major league debut.

Not only did the debut live up to the hype but it exceeded it.  Strasburg didn't just look fantastic and from a non-statistical perspective, his stuff was electric.  He struck out 14 batters of the 24 he faced but in doing so, walked a grand total of none.  That's extraordinary.  Using tools available at baseball-reference.com (a site I highly recommend and which should be a frequent stop on every reader's Internet browsing), I found a grand total of 20 games since 1968 where a pitcher struck out at least 14 batters without walking a batter.  The first few names since 1968 were Sam McDowell, Luis Tiant, Vida Blue and Frank Tanana, all of whom had had excellent careers.  In the 1980s, the names were Dwight Gooden (twice), Mark Langston and Sid Fernandez, again all pitchers who had strong careers.  Since 1990, the names are those you would expect to see: Pedro Martinez (twice), Randy Johnson (twice), Roger Clemens, Curt Schilling, John Smoltz, Johan Santana and Mike Mussina make the list.  Then there are a couple who may not have had the careers of these others but still managed to accomplish the feat in Eric Bedard and Mark Prior.  That's it since 1968.

But again, we run into the problem of singling out something that got our attention rather than a random sample of a game.  So, let's consider this from a different angle.  What if we could simulate a large number of random games made up of exactly 24 batters faced?  We could try different theoretical levels of strikeout ability and see how often our imaginary pitcher strikes out at least 14 batters.

Let's start with the average National League pitcher from 2009, who would strike out about 18% of the batters he faced.  With our ability to run high speed simulations of blocks of 24 batters faced, I ran 10 million of these and at the end of 10 million "games" the average NL pitcher from 2009 struck out at least 14 out of 24 batters in 149 of the 10 million games or about 1 in every 67,114 games.

In other words, it's possible that Stephen Strasburg is not yet at least an average National League pitcher but based on a single start, an incredibly small sample, we can already say that it's extremely likely that at least where strikeouts are concerned, that one game demonstrates that he's already above average by NL standards.

But you probably already thought that.  Let's raise the bar.  Let's give him the theoretical ability of a pitcher who strikes out about one out every 4.5 batters he faces or about the rate Josh Beckett had in 2009.  When we ran ten million games of that sort of pitcher, the 14 strikeout threshold was met or exceeded 1,491 times or about once every 6,706 games.

Let's go even further: Let's give our imaginary simulation pitcher the ability to strike out about one out of every 3.5 batters he faces or about what Tim Lincecum did in 2009.  When we ran that simulation, again ten million times, our imaginary pitcher of Tim Lincecum's approximate strikeout skill managed to achieve the 14+ strikeouts in 24 batters faced mark 21,881 times or about one out of every 457 times.

I still hesitate if not outright reject making conclusions based on a performance that gets our attention simply because of its excellence but this isn't just one night where a pitcher struck out 14 batters and walked none.  This was the top-ranked pitching prospect in the world making his first major league appearance.  That's not necessarily a random sample and we could have easily resolved to do this sort of analysis on whatever he happened to achieve in his first game.

My suspicion is that Strasburg actually has nearly proved something beyond an acceptable degree of confidence and that is that he is an even better strikeout pitcher already than I expected he would be at this early stage and my next projection revision will reflect this.  His minor league numbers this year didn't even hint at this sort of performance and I'm expecting that both he and his team were (and maybe still will be) controlling his effort enough to not over-exert himself.

One other notion that I think deserves mentioning is that we hear some reminding us that facing the Pittsburgh Pirates did not offer Strasburg an average major league opposition.  While that may be true, the Pirates may be a low-average team (worst in the majors at .238) but they are actually not that bad at making contact, at about a 79% rate, almost exactly the current major league average, which is about 79.6% so far this season.  So, Strasburg's strikeouts achievment is not significantly lessened by the Pirates having been his opposition in his first game.

Anyway, despite everything I often write about small samples and overexcitement about huge single game performances, I believe that Strasburg's first game is that rarest of exceptions.   I don't think it proves that he's going to strike out double digits in every game but it does demonstrate a significantly superior strikeout potential than I had previously expected so early in his development.  If you didn't watch his first game, I highly recommend you tune in to watch his second start.  It's not about whether he can match what he did in game one as much as you really need to see his stuff if you enjoy the game.

Wednesday, June 02, 2010

Trading and the Gambler's Fallacy

We're hitting that point in the year where the contenders and the also-rans in fantasy leagues are becoming clearly separated and teams are making moves to strengthen their run for a championship or, if they think they can't contend this year, building for the long run.

Occasionally, I pop into message forums to see what the chatter is and of course, I often hear from readers asking about whether they should trade for player X despite a lousy start to the season, and at this time of year, I've personally responded by email to a greater percentage of these emails than usual.  One of the trends I've noticed, and it's not the first time I've seen this, is that there remains a trend for some fantasy leaguers to confuse the law of large numbers with the oft-used phrase "the law of averages" and/or committing the gambler's fallacy.

The law of large numbers does not tell us that a player will do more at some later point to offset the results from the period when he did less earlier.  In that same line of thinking, it does not mean that a player who has started any season above his real ability will now necessarily do worse later so that his final totals somehow arrive at his real ability.  The larger the sample size, the more an entity tends to perform closer to its real ability but there is not an offset effect.  In other words, let's return to our favorite coin flipping example.  Assuming for this example a fair coin, let's say the coin goes 7 for 10 in April on tails.  This does not mean that the coin should now go 3 tails for 10 flips in May to "balance out" things.  The coin still has a 50% chance of coming up heads or tails.  If we flipped the coin another billion times or so, that 7 for 10 start will be a long-lost memory because in the grand scheme of tossing the coin a billion times, the sample size of 10 would have a virtually non-existent effect on the final totals.

The gambler's fallacy, as it is often called (and worth Googling if you've never read about it), is the belief that somehow things will balance out as if the deviation from real ability so far that we've witnessed will be offset by a reverse deviation in the opposite direction.  I remember years ago reading an example that went something like this: If Wade Boggs is hitting .200 at the end of April, then you should trade for him because you know his final totals will be around .330-.360 at the end of the season and you'll end up with the portion of the season that helps him achieve these totals.  Wrong.  You should have traded for Wade Boggs because his ability was so good at that time that you'd end up with a lot of value but if he were to hit .200 through the first month and his real ability was .360, it did not mean he was more likely to hit .380 the rest of the way to make up for the slow start.  The most likely outcome would still be .360.  Now, if that .200 start convinced you he was declinined and you decided he was actually becoming a .350 hitter instead of a .360 hitter, well then you would project him to hit .350 the rest of the way, presuming neutral other variables that influence the remaining outcome.  You see this sort of thinking in so many circles about expecting reverse effects to balance things out.  The roulette player who has witnessed five blacks in a row believes that red is now due to come up and when red finally, and inevitably eventually comes up, he exclaims "I told you!" when you can keep doing this forever until you hit red.

I wanted to write about this today not only to deter trading for players because you believe they're about to perform beyond their ability to balance out a slow start but also to deter you from dumping players who have started out hot just because you think the slump is coming that will offset it.  By all means, winning at fantasy baseball absolutely requires that you exploit the incorrect perceptions generated by a player's season-to-date.  If you think a pitcher's real hidden ERA skill is around 5.50 and he has an ERA of 3.00 so far on the season, you have to make your move and maximize your return.  But if you believe his real hidden ERA ability is 3.75 for that same pitcher, it doesn't mean that his hot start should have you believing he's suddenly going to collapse performance-wise just because he's already experienced the good part of his season.

You have to assess the player's remaining expected value but always avoid incorrectly applying the season-to-date numbers to make "reverse calculations" that try to say "okay, because he's already got an ERA of 6.00, that means his ERA will be 2.50 the rest of the way if he's to land at his real 4.00 ability by the end of the season."  It doesn't work that way.

Take the often-discussed Javier Vazquez for example.  He's much younger than you might believe if you read speculation about him being at the end of the line (Vazquez turns thirty-four later this month) and while we've been forced to adjust our forecasts to reflect that his terrible start jeopardizes his spot in the rotation for a while, it also doesn't mean that we expect him to suddenly go on some 1988 Orel Hershisher-like stretch to offset that start just because we believe his ability is better than he's shown.  His season is going to look bad for a long time and there's no erasing what's already in the books.  So, when/if you trade for him, and he's an example of a player who has become surprisingly easy to acquire relative to just a few months ago, don't be banking on him pitching above his projected ability level the rest of the way.  Just trade for him because you share our belief that he's a better pitcher than we've seen and his projected remaining value can be of use to you.
Now, this is where there is another exploitation opportunity in trading right about at this time of year.  That is, there are people who do make the gambler's fallacy and they're playing against you in your fantasy league.  There are fantasy GMs, potential trading partners, who actually expect players to perform outside of the ability that even they project because they commit the gambler's fallacy.  These are the guys who will actually be anxious to trade a Jose Bautista because they think that his unusually strong start is a fluke and that Bautista will come crashing down later in the year.  Well, even if we agree that Bautista is overachieving, that doesn't completely negate his value and we will still be forecasting him to perform according to our revised estimate of his long run ability, not less than that simply because he had a hot start.  Maybe there are fantasty trading partners who don't dump their players like this but who are too smart for their own good and want to trade for players whose decline to date represents a real problem but for whom hope is held out that the rest of the season will be a bounce-back beyond their projected ability to balance things out.

In short, always remember that when you're making a trade, you need to be assessing what you believe the player's real ability is, both for the short-term and long-term.  Certainly, no one can deny that what a player has done so far this season informs our opinion of that but it's crucial to use that season-to-date performance properly, to influence our evaluation of ability and not to make some sort of offset estimate that tries to say "he's already hit 10 of his projected 15 home runs so a big drop-off is coming" or variations along those lines.  Doing that is not only a quick path to frustration but it also will have you waiting for streaks and slumps that even if and when they come, will be fluke more than fact and will further confuse making accurate evaluations of a player's real hidden and long-term sustainable skill level.