Add new comment

First, the positive thing. I agree that it is possible we are not doing a good job understanding how minor leaguers are likely to translate to the majors. There are new discoveries every year regarding things like K rate boundaries (which types of .500 SLG minor leaguers are likely to hit in the big leagues?), contact proficiency (BABIP) and correlation to big league success, which type of power is better (lots of doubles or few doubles and lots of HRs)? And for pitchers, the same story is unfolding, but more slowly due to complications with injury. Many of the current projection schemes are too simplistic and do not account for the real reasons that production changes when you change leagues.
Second thing:
But that still does not mean that major league projections and statistics are no more reliable than minor league. While there remains uncertainty as to how best to project minor leaguers, those projections should still be considered less reliable...and Tom Tango finds that, in fact, they are less reliable. Statistically significantly less reliable to a p-value of basically 0. It may not SEEM like a lot to go from .335 average projection to .319, but that is actually a *MONUMENTAL!!* shift when you consider how many player seasons were in that sample. The weather analogy would be if the US generated global forecast had a global mean sea-level pressure bias of +0.5 millibars. Atmospheric pressure is about 1013 mb, so 0.5 seems tiny, but if you run our model for a year and it's always high by half a millibar by day five, there is something really REEEALLY wrong with the model. Something so wrong that if you just gave the climo projection for pressure, you'd have less bias and thus a better forecast. What Tango is saying is that the 16 point wOBA error is a gigantic bias given the sample size and that you actually do better if you simply assume that all minor leaguers will hit the same in their first big league season as the average minor leaguer does in reality.