Educational testing is under attack. In New York City, according to one recent poll, a majority of voters support removing the Specialized High Schools Admissions Test (SHSAT)—the standardized test used for admission to the city’s eight most selective high schools. In higher education, the “test-optional” movement has gained momentum, with prominent schools like the University of Chicago no longer requiring the American College Test (ACT) or Scholastic Aptitude Test (SAT) for admissions. In fact, according to fairtest.org: “More than 1,000 four-year colleges and universities do not use the SAT or ACT to admit substantial numbers of bachelor-degree applications.” And in graduate admissions, the “GRExit” movement continues, with Princeton University announcing that 14 of its graduate programs no longer require the GRE for admission.
Criticism of testing is not new. In 1975, educational psychologist Lee Cronbach observed that, while support for testing would come and go, critics came “largely from those who speak for the poor.” What Cronbach described over 40 years ago is replaying itself today. Testing is being deemphasized or eliminated, though the evidence for its reliability and validity is substantial, especially compared with other selection criteria.
Standardized tests like the SAT have been shown to measure general cognitive ability, or g. By extension, the SHSAT and similar standardized mental or achievement tests likely also measure general mental ability, which has been shown to be predictive of academic performance, career potential, creativity, and job success. Even after accounting for socioeconomic status, standardized tests are predictive of long-term outcomes. They have “predictive validity,” a key statistical measure of the degree to which the test score is associated with some valuable outcome of interest in the future, such as grades or graduation rate.
In undergraduate admissions, the predictive validity of standardized tests ranges from 0.51 to 0.67 (with perfect validity being 1.0) for predicting cumulative GPA and individual course grades. Similar findings are observed in graduate admissions. Results from over 82,000 graduate students revealed validities ranging from 0.34 to 0.41 for cumulative graduate GPA and 0.26 to 0.51 for comprehensive exam performance. Standardized admission tests at both the undergraduate and graduate level are also related to achievement in life, such as high incomes or creation of patents. These effects are statistically significant: a correlation of just 0.30 translates roughly into a 67 percent chance that a student scoring in the top quintile will be successful (versus 33 percent of those in the bottom quintile). A study of more than 22,000 middle school students in New York City found that the SHSAT had a predictive validity of roughly 0.45 for freshmen high school grades. Though still preliminary, these findings suggest that the SHSAT functions similarly to undergraduate and graduate admission tests.
Though critics are often unimpressed by the validity evidence, no method of evaluation currently exists that predicts future success as well as standardized tests. Studies of other admission-related criteria have found substantially lower validities compared with such testing. Letters of recommendation show predictive validities ranging from 0.13 (medical school GPA) to 0.28 (undergraduate GPA). Personal statements fare much worse. Critics point to the effect of social class on test performance as a primary reason for not using the test, but for many talented students from families of low socioeconomic status, the standardized test offers a platform on which to compete with students from wealthier backgrounds.
Yes, being born into a wealthy family brings advantages—but are those advantages most pronounced in proctored, timed tests of logic and reasoning? The evidence on the predictive validity of the tests, even after accounting for socioeconomic status, suggests not. An individual born into a working-class family of five who works extra jobs to help with the bills will be at a much worse disadvantage if the admissions process places more weight on extracurricular involvement or volunteer-research experience. If standardized tests are removed, even more talented and capable students without the means to obtain the credentials of “well-rounded” students—on factors with low to zero predictive validity—may simply be overlooked.
Standardized tests are by no means perfect. Financially disadvantaged students might be strained for finding money and time to take the tests, but more equitable solutions exist than to throw them out. Universal screening, for example, may remove the undue financial burden of the test on disadvantaged students. Despite what appears to be a fundamental disagreement between test critics and proponents, we believe that common ground is evident. The key is to seek fair and equitable admission processes that best evaluate human potential from all backgrounds. Despite their imperfections, standardized admission tests remain an effective way of doing this.
Photo: Kali9/iStock