Bruce Baker on Value Added and Student Growth Percentiles

I’ve lauded Bruce Baker’s blog once already, but day by day I’m becoming an even more devoted follower of his wit and his brilliant exposés of fallacious research, bogus claims, and shallow thinking.  I’ve learned/confirmed so much by subscribing to School Finance 101, particularly about value added models and student growth percentile models, as Dr. Baker reexamines the research on these topics and highlights the shortcomings and the unintended consequences of using these measures in states all over the country.

I’m very interested in his work on student growth percentiles, first created by Damien Betebrenner at the National Center for the Improvement of Educational Assessment, www.nciea.org. First used in Colorado, and now spreading to Massachusetts, New Jersey and New York, the proponents of SGPs suggest they fix the problems inherent in value added measures.  Baker emphatically says not so, and uses the writings of the fathers of SGPs to highlight how SGPs are being misused.

To get up to speed on VAMs and SGPs and their problems as Baker sees them, go through the blog references found in his Value Added Teacher Evaluation category here, and as you read the articles, click on through to the links within each one.  You’ll get quite a wonderful look at the limitations of these statistical models as they relate to teacher evaluation.  Baker also takes on state education officials in several states, including Colorado, New Jersey and New York, with his irrevent commentary.

Of particular interest for New Yorkers is his analysis of the preliminary technical reports on the results of NY’s first SGP assessment results.  This analysis can be seen in the entry entitled “AIR Pollution in NY State…”  He offers graphs showing how factors that cannot be attributed to teachers have an effect on the patterns of SGP scores–students in low income schools generally underperform students in higher income schools, as one already knows, but the SGPs of these students also lag behind higher income schools.  This demonstrates that SGPs do not, in fact, account for effects of peer groups, of school effects, or of poverty, as some suggest.

Read Baker and weep about how assessments are regularly being misused by people who should know better,

Race to the Top – A “Race … in the Right Direction?”

New York schools, despite having one of the largest percentages of students in poverty, consistently rate highly in Quality Counts, from Education Week.  The state was a second round winner of $700 million in Race to the Top funds from the Feds, and aggressively rammed new legislation through the state legislature to qualify.  The unintended consequences are now being felt by those responsible for implementation.  Teacher morale is down, the costs of implementation are proving to be unfunded mandates several times greater than what was gained from ‘winning’ the Race, and a tax cap pushed through by the Governor is wiping out district capacity to raise taxes even where the public would be willing to pay more to maintain quality schools.  Moreover, many are estimating that 40% of New York districts face bankruptcy in the next few years.

Dr. Kenneth Mitchell, the Superintendent of the South Orangetown Central School District in Rockland County, NY, contributed a wonderful analysis of the unintended consequences of Race to the Top in New York, published by the Center for Research, Regional Education and Outreach at the State University of New York at New Paltz.  Open ‘Discussion Brief 8’ from this link to read the full article.  With the cooperation of 18 school districts in Rockland, Westchester and Putnam Counties, the three counties bordering New York City to the North, Dr. Mitchell produced an eye-opening analysis of the impact of RTTT in this relatively wealthy region.  These three counties are among the richest in New York–and the 18 districts are representative of the range of wealth found in the 60 districts in the region, though, I think, they would average somewhat above the mean in community wealth because some of the larger and lower income districts did not contribute data to the study.

Financially, Dr. Mitchell reports that the four year return to the 18 districts from RTTT funds will be $520,415, while the expense to districts is estimated to be $6,472,166, for a deficit of $5,951,751, an unfunded mandate of nearly $400 per pupil, to be funded by local taxpayers.  And because of the tax cap, which now requires a 60% super majority to pass a budget above the 2% cap, very few districts will be able to fund the mandates without substantial cuts in programs somewhere.  The cost of implementing the state’s new Annual Professional Performance Review – APPR  (teacher and principal evaluation) by itself represents a 3% increase in local costs to pay for the testing and evaluation training required for implementation without considering the costs of redesigning instruction to move to the Common Core curriculum, which requires new texts, instructional materials, shifts of content from grade to grade, and additional teacher professional development for implementation.

The effects, as gleaned from the 18 districts responses to the program, are disturbing.  Despite Race to the Top’s promotion as a school improvement program, districts will see staff cuts and larger class sizes, and non-mandated programs will be cut.  Districts are cutting maintenance despite this being a wonderful opportunity to get cost effective pricing during an otherwise slow economy.  Priorities are shifting away from instructional services that districts have developed over years of successful local control in order to fund the external mandates.  Internal professional development and staff to provide it is being cut, and training is focusing on teacher evaluation requirements, not on classroom instruction. Some districts will be required to hire additional supervisory staff to complete the evaluation mandates, which require more time than existing programs.  Curriculum is narrowing as districts prepare for extensive testing that will be used for teacher accountability.  And finally, quoting directly from the brief, “..the hidden costs may be greater than the outlay in dollars.  Teachers and administrators, stressed by the rapid change, the demand for accountability via the new testing and observation requirements, and anxieties about receiving low scores, are very likely to abandon initiatives that may be innovative and beneficial for preparing the next generation, but are out of alignment with a narrowed professional agenda for staying within the ‘Effective” range on the APPR.”

Dr. Mitchell proposes some reasonable shifts that New York politicians could make to back off and redirect this initiative.  He also identifies several sophisticated research groups that question the use of tests for teacher evaluation as Race to the Top demanded, and question the massive national shift to the Common Core curriculum.  Both these shifts are untested and lack a research base to justify the depth of the changes.  And since the changes were imposed by the legislature in New York, and by legislatures racing to get Federal funds across the nation during the economic slow-down, the changes are out of the hands of state education departments even if those leaders were inclined to back off in the first place.  Since they were generally the designers of the system, they are unlikely to be responsive–it’s full steam ahead, into the chaos that will ensure.

Download the article pdf at the link above, and see the research citations in the “Works Cited” link as well.