More than just test scores

Continuing the theme of my last post, I want to reference Henry M. Levin’s paper “More than just test scores,” which can be accessed at http://roundtheinkwell.files.wordpress.com/2012/09/more-than-just-test-scores-sept2012-2.pdf.  I’ve followed Dr. Levin’s work for several years and I’ve enjoyed his contributions to the knowledge base on assessment issues.  In this brief, published online on August 30, 2012, Levin provides an outstanding and thorough discussion, supported by extensive research citations, on the relationship between test scores and earnings.  He notes “…the measured relationships between test scores and earnings or productivity are modest and explain a relatively small share of the larger link between educational attainment and economic outcomes.”

Non-cognitive skills mean career readiness

Much of his paper discusses “the effects that education has on the development of interpersonal and intrapersonal skills and capabilities that affect the quality and productivity of the labour force.”  It was Levin’s work here that prompted me to write this post and my last post on non-cognitive skills, starting with the SCANS work from 13 years ago, which offers substantial background to the reader for Levin’s research analysis.  Just reading Levin’s abstract should be enough to keep you reading to the end of the paper, and then checking out some of the references.

Nowhere does Levin suggest that cognitive skills, as measure by test scores, are not important contributors to economic success later in life.  But throughout the research cited one reads repeatedly that academic achievement is not the only measure of economic success.  One must conclude that focusing 100% on improving test scores as the be-all and end-all for education is a narrow and likely misguided agenda.  Coupled with the SCANS report, it’s clear that a focus on test scores and the corresponding narrowing of curriculum to cognitive issues only is not the best means to improve America’s economic competitiveness or the earnings potential of American youth.

Noting that the discussion of world-class educational standards, and comparisons of nations on various international tests leaves out “other attributes of school or student performances,…” Levin argues that “…basing world-class standards on measures of student achievement is a very limited approach to evaluating any educational system.  To meet the economic, political, social, and personal demand for competency, much more is required of students and adults than just cognitive proficiencies as measured by test scores.”  And he echoes the SCANS work when he suggests that interpersonal and intrapersonal skills are dimensions that can’t be ignored.

Test scores don’t explain earnings

By reviewing numerous studies on the connection between academic achievement and later earnings. Levin adds depth to the common understanding that more schooling equates to higher incomes.  This is correct, but the assumption that the higher earnings are associated with cognitive skills is overstated.  He identifies numerous studies that include test scores in their analysis of schooling and earnings, and they find that the “…impact of schooling on earnings retains more than 80% of its estimated effect on earnings, even after accounting for student achievement.”  (See Bowles, Gintis, and Osborne, 2001, “The determinants of earnings: A behavioral approach,” Journal of Economic Literature, 39(4), 137-1176.)  In plain English, test scores account for “a minority fraction” of the relationship between earnings and schooling, and other elements of schooling “must be accounted for.” Taking on the claim that test scores have predictive validity for worker productivity, Levin cites research that shows scores account for only 6% of the variance in productivity, “leaving 94% to be explained by factors other than the test result.”

So, it seems to me, the research isn’t particularly supportive of pushing exclusively for high academic achievement to return America to world-class productivity.  And besides, the US worker is already among the most productive anywhere, even when our test performances appear mid-range.

Graduation rates do equate with economic performance

The relationship between finishing high school with a diploma and increased lifetime earnings is well-established.  Levin offers some depth to this discussion by pointing to research on five educational interventions to prevent dropouts, which offer net benefits to taxpayers through public heath savings, crime savings, and increased exhale revenues from earnings.  Interestingly, he also notes that GED graduates do not show the same benefits–their test scores are similar to graduates, but their earnings were inferior.

Having shown that cognitive skills and tests scores are not very good predictors of economic performance, but high school completion is, there is something other than academics of value for students who stay in school and complete a diploma other than compiling academic skills.  And these factors, not currently tested, play as important a role, or a greater role, in helping students become ready for the world of work.  I strongly suggest you read the rest of Levin’s article, in which he reviews what those characteristics are, and how schools can improve the way they are supported.  In fact, Levin identifies several programs that teach the social and emotional skills, and cites  research supporting the efficacy of doing so.  He spends some time on the SEL movement–social and emotional learning–and goes beyond identifying the employment benefits by identifying research that shows SEL program intervention “…were comparable to or exceeded the results found in the literature for improving student achievement.”  This is a really significant finding–“…interventions to develop social and emotional skills have particular salience for improving student achievement.”

I’ll end with a few sentences from Levin that should inspire you to read this piece several times, and read some of the back-up research he has so capably assembled.  “The almost singular focus on test score performance in educational assessment at both domestic and international levels does rest on some foundation…. But the evidence does not support the assumption that cognitive skills are all that counts and that they alone can produce healthy and productive adult personalities.”   “Far from being harmless, the focus on test scores and the omission of the non-cognitive impact of schools can create far-reaching damage.”   “Other goals may be as, or more, important in the long run in terms of creating productive, equitable, and socially cohesive societies and economic growth.” (See Gradstein and Justman, 2002, “Education, social cohesion, and economic growth.”  American Economic Review, 92)4_ 1192-1204.

As educators and activists for meaningful educational policy, we have to stop simply looking at easy solutions and start listening to what employers say they want.  Listening to employers was a novel idea when the SCANs report came out in 2000, and it’s likely that those who publish achievement tests and those who push them as easy measures of future productivity will continue to ignore employers in the future.  Real educators should speak up.