Insightology

Why aren't there any more 0.400 hitters in baseball? The eminent evolutionary biologist and baseball fan Stephen Jay Gould answers this great sporting puzzle in his book, Full House: The Spread of Excellence from Plato to Darwin, which is only partly about baseball. It is really about understanding basic statistics and along the way you get a great discourse on the animal world and in particular bacteria. He uses these examples, and a terrifying intellect, to argue that just looking at averages (such as the mean) is not sufficient and one has to look at variation within the system as a whole to fully understand trends. With this approach he is also able to answer the question of whether quality of play overall has declined or improved in baseball over time.  If you are not into animals, you may want to go straight to the baseball portion, which by itself is not at all a hard read. So why aren't there any more 0.400 hitters in baseball?

No less an authority than the great Ted Williams (in his 1986 book, The Science of Hitting) argued that today's batters are just not as good. Other conventional explanations that Gould mentions are, better pitching, better fielding and better managing. He goes on to show that while fielding certainly has become better over time (as primarily measured by the improved fielding average over time), these are not the reasons for the extinction of the 0.400 hitter in baseball. The problem is essentially statistical. The mean batting average over the years (by design to a good extent) has remained largely unchanged (approximately 0.260 in both leagues). However, the variation around the mean (known as variance or standard deviation) has steadily decreased over time. What does that mean?

Consider what the early years of the sport (for convenience, the early 20^th century) were like. There were some terrific players (Ty Cobb, Babe Ruth etc), but there were also many mediocre players. Baseball was not yet the science it is today nor was the equipment (especially fielders' gloves) anywhere close to today's baskets. Tactics such as relief pitching, field placements, strategic pinch hitting etc were not as prominent. Importantly, baseball drew from a much smaller and less diverse pool of players, and they were generally smaller than today's players.

Taken together this means that the best hitters of yesteryears quite often feasted on relative mediocrity. Over time all of these factors have changed to a point where baseball and everything that surrounds it is now a science backed by an industry. Becoming a major league baseball player today is far harder, not least because you have to compete with players from around the world. In other words, mediocrity has been slowly eliminated in the sport. In statistical terms this means that, while the mean batting average has remained the same, the variance around the mean has shrunk substantially. We don't find 0.400 hitters, but we also have to struggle hard to find sub 0.200 hitters. Unlike in the old days, a great fielder with lousy hitting skills cannot survive in the majors.  

This is true of pitching too. If you look at the 100 best ERAs, only a handful (less than 10%) appeared after 1920. If you remove the anomaly of Bob Gibson (1.12 in 1968, a pitching year for some reason), none crack the Top 40. Is it that pitchers have become worse over time? Does it make any sense that Sandy Koufax (who places no higher than 97^th) was only as good as that position indicates? Especially with the development of specialist relief pitchers and the dramatic improvements in conditioning and strategy, it is hard to argue that today's pitchers in general are substantially worse than those from times gone by. In fact it could be argued that pitchers today are on average much better, and fielders far better, than those from the early part of the century. For that reason today's best hitters (or yesteryears best if time traveled to today) are unable to reach the soaring heights of 0.400 hitting. So, yes, Ted Williams was wrong.       

Will we ever see a 0.400 hitter again? Probably, yes, but it would take not just a very good hitter but also substantial luck and circumstances. As sabermetricians have shown, baseball involves a lot of luck and a player needs that to carry him through an entire season. Gould's idea of a constant mean and decreasing variance signifies the development of excellence over time in not just baseball, but other sports too. He postulates that at the end of the right tail there is a "wall" of human limits. In many of the older sports we are close to that wall and hence progress is very slow, if ever. A specimen not seen before is often needed (along with the right circumstances) to break through that wall. It is possible we witnessed that in the 100m dash in the Beijing Olympics in the form of Usain Bolt of Jamaica.    

Stephen Jay Gould died in 2002. Before that he served as a Professor of Zoology and Geology at Harvard University for many years. He has written several popular books on these topics.