It has been called “‘Moneyball’ management.” Employers, especially at larger companies, increasingly are using extensive psychological test results to select candidates to fill management positions. And supplying these tests is a growing industry.
Overall, the process does resemble the data gathering and statistical analysis made famous by the Oakland Athletics baseball team and the book and movie “Moneyball.” There are several differences, though, and they are significant.
In both the baseball and management forms of “Moneyball,” analysts refine a mound of statistics to see how well the subject — potential player or corporate manager — fits the profile of successful individuals. Some tests make extensive use of algorithms in the interpretation process, and in both cases there is a considerable amount of statistical manipulation.
One of the differences is right up front: the system design and purpose. The original “Moneyball” system was devised to help so-called small-market (lower-budget) Major League Baseball teams to compete successfully with the league’s heavyweights. It was designed to identify skilled but undervalued players so they could be acquired at affordable costs.
The management “Moneyball” recruiting system’s purpose is very different. Rather than identifying good prospects from a large pool, it is designed to avoid the expensive mistakes in candidate hiring that result in subsequent poor performance as managers.
Another difference, a crucial one, is in the data itself. In the case of “Moneyball” in baseball, the statistics are compiled from a player’s performance in actual games. The management version, by contrast, uses proxy variables — in this case, statistics derived from responses to questions.
It could turn out that the management system’s proxy variables are a better predictor than the player’s performance statistics. After all, there is a big difference between minor league and major league baseball. Major league history is padded with large numbers of unsuccessful players who couldn’t hit a “big league curveball” and triple-digit pitching “phenoms” who couldn’t keep the ball away from the middle of the plate.
The predictive shortcomings of the baseball system’s performance data aren’t a good reason to justify blind faith in the management “Moneyball” system’s proxy data. We shouldn’t assume that it is better just because it is different.
Relying on psychological profiling to sort out the likely failures and identify “keeper” candidates is not a bad idea at all, but businesses should keep it in perspective. And that means recognizing the risk that the tests aren’t providing the data you need.
In fact, with management “Moneyball” there is a real risk that what is being tested is not the candidate as much as the candidate’s education and his or her ability to navigate and “game” the test. Some management programs, for example, are very closely tuned in to what corporate employers want in their recruits. The result is that candidates who are prepared with that information will adjust their test responses to reflect their knowledge of what is wanted by the hiring companies.
The test results may also be skewed by raising a generation of students that is very experienced at taking and gaming tests — and that would be particularly true of students with post-high-school educations. Skilled test designers and evaluators are very good at anti-gaming tools, but it is something worth discussing with test vendors before adopting a “Moneyball” type of test system.
Another factor that should be taken into account is the effectiveness of baseball’s “Moneyball.” The Oakland Athletics are still a long way from a World Series win, and “Moneyball” has not proven to be better than the human-based scouting system in identifying player potential.
The lesson that businesses should derive from baseball’s “Moneyball” experience is that the combination of statistical manipulation and artificial intelligence can provide some information that is useful in decision making. It just isn’t effective enough to replace that process.
Taking that lesson to heart, many companies, large and small, can save themselves a considerable amount of time and money by thinking through their management problem and its sources. If the issue is the performance and overall quality of new hires for management positions, and there is solid evidence that the candidate selection process is to blame, then some form of management “Moneyball” may help solve the problem.
What is equally likely, though, is that the issue is not all about candidate selection. Part of it is about ill-defined and poorly measured management structure, responsibilities and performance.
If a management “Moneyball” system is focused on a replacement CEO, it could be helpful to the board of directors in their selection process. But widespread psychological testing has a long way to go before it proves itself more effective than an a solid, human-based interviewing system.
James McCusker is a Bothell economist, educator and consultant.
Talk to us
> Give us your news tips.
> Send us a letter to the editor.
> More Herald contact information.
