Score card

It's the last week of June, and I just wrapped up another school year.  My 29th overall, which still seems kind of impossible to me until I realize that a child of a former student graduated from high school this year.  Then it seems pretty real, along with a realization of "Good lord, I'm getting old."

So I've been at this for a long time, and with, I think, some measure of success.  Which is why I read my letter from the school district awarding me my numerical grade for the school year with a mixture of amusement and irritation.

I won't leave you hanging; I got an 81.  I got an 84 last year and a 91 the year before that, so according to the state rating scale, I'm becoming incrementally less competent.  It can't, of course, be because the metric is flawed, that the three grades are comparing different assessments of different students put together in different ways.  No, in the minds of the geniuses at NYSED, this number means something fundamental about my effectiveness as a teacher.

In fact, that's what an 81 gets you; a designation of "Effective."  You have to have a 92 to be "Highly Effective."  If you're below 75, you're "Developing."  I'm glad I didn't land in that category.  If after 29 years at this game, I'm not "Developed," I don't hold out much hope.

What amused me most about all of this nonsense was the paragraph that said, and I quote:

Please remember that your scores are confidential and should not be shared in any way.  In accordance with state regulations, the parent of a child in your class may request your composite score and rating as well as that of the principal.  For your own protection, teachers are strongly discouraged from sharing their own scores outside of the district process.

Which is a recommendation I'm happy to toss to the wind (along with the aforementioned letter).  If we keep our scores and the way they were generated under wraps, it allows the statistics gurus at the State Education Department to keep everyone under the impression that they actually know what they're doing.

[image courtesy of the Wikimedia Commons]

Let me get specific, here.  My 91 two years ago was based upon the scores of my Critical Thinking classes and my AP Biology class.  Critical Thinking is an elective, and while the day-to-day work is difficult (requiring a lot of thinking, surprisingly) the material that is suitable for an exam at the end of the year is actually quite easy.  So my students performed brilliantly, as I would expect.  Additionally, that year's AP class was an extremely talented group who knocked the final exam clear out of the park.

Fast forward to last year.  My score last year was based on a combination of my Regents (Introductory) Biology class and my AP Biology classes.  Because of a strange policy of piling students who are classified as learning disabled into the same class, last year's Regents Biology was half composed of students who have been identified with learning disabilities.  Many of these students were hard-working and wonderful to teach, but it's unsurprising that that part of my grade went down.  My two AP classes last year were a friendly, cheerful lot who also happened to be somewhat motivationally challenged, and who by the end of the school year were far more invested in playing Cards Against Humanity than they were in studying for my final.  So that accounts for the remainder of the decline in my score.

This year, my score was a composite once again between Regents and AP Biology, but this time my Regents classes were among the most talented, hardest-working freshman and sophomores I've ever had.  My AP class was small but outstanding, but because of the way the scoring is done, they would have to score on my (very difficult) final exam higher than a target determined by their score on the (far easier) Regents Biology exam for me to have that student's score count in my favor.  On the part of my assessment that came from my AP class, I got a grand total of three points of of a possible twenty -- mostly because of students who got an 81 or 82 on an exam where their target was 85.

So my three scores in three consecutive years have absolutely nothing to do with one another, and (I would argue) nothing whatsoever to do with my competence as a teacher.  But because there's no idea that is so stupid that someone can't tinker with it to make it even stupider, next year the State Department of Education has informed us that we'll be assessed a different way.  Our joy at hearing this pronouncement was short-lived, because once we heard how they're going to score us, we all rolled our eyes so hard it looked like the email was inducing grand mal seizures.

Next year, unless over half of your students are in classes that take a mandated state exam at the end of the year, 50% of your score will be based on an average of the "Big Five" exams, the ones that all students have to take to graduate -- English, US History, Algebra I, Global History, and Biology.   (The other half, fortunately, will be based on evaluation by an administrator.)  If you think you can't have read that correctly, you did; the half of the high school band teacher's grade (for example) will come from students' scores on exams that she had absolutely nothing to do with.  Even for me, who teaches one of the "big five" -- less than half of my students next year will be in Regents Biology, so I'll be getting the composite score, too.

But don't worry!  Because students mostly score pretty well on these exams, and the score will be calculated using the time-honored statistical technique of averaging averages, we'll all look like we're brilliant.  So in effect, they took an evaluation metric that was almost completely meaningless, and changed it so as to make it completely meaningless.

Because that's clearly how you want an evaluation system to work.

All of this, it must be said, comes from the drive toward "data-driven instruction" -- converting every damn thing we do into numbers.  Couple this with a push toward tying those numbers to tenure, retention, and merit pay, along with a fundamental distrust of the teachers themselves, and we now have a system that is so far removed from any measure of reliability that it's almost funny.

Almost.  Because NYSED, and other state educational agencies, look upon all of this as being deadly serious.  It's all very well for me -- a veteran teacher of nearly three decades who is looking to retire in the next few years -- to laugh about this.  I wouldn't be laughing if I were a new teacher, however, and I'd be laughing even less if I were a college student considering education as a profession.

In fact, it'd make me look closely at what other career options I had.

Knowing the score

New from the "Merciful Heavens, Please Tell Me We're Not Fighting This Nonsense Again" department, officials at the New York State Department of Education are proposing using student scores on AP exams and the SAT test as a means for evaluating teachers and administrators.

How many times does it need to be said?  Standardized test scores are not a measure of teacher effectiveness.  Okay, if I was completely incompetent, my AP Biology students would probably all tank the exam.  But beyond that, my students' scores are far more indicative of their ability to comprehend technical material, their curiosity, and their work ethic than it is of anything I happen to be doing.  I have had years where every single student in my class has gotten a score of 3 or above (usually sufficient to obtain college credit).  Other years, I have not had a single 5 (the highest score) and a commensurately high number of 1s and 2s.  What happened?  Was I competent one year and completely ineffective the next?

Add to that the fact that the College Board, in their infinite wisdom, completely restructured the exam four years ago, and I don't think the scores actually mean much of anything from the standpoint of what I am doing in class.

The SATs are even worse.  I used to teach SAT math prep courses in the evening until I became so frustrated by the "learn how to game the test" approach of most of the curricula we used that I decided to make extra pocket money a different way.  My considered opinion is that your SAT exam score tells you exactly one thing -- how well you did on the SAT test.  A study two years ago found little correlation between SAT score and success in college.  More troubling still is the fact that the one of the strongest correlations of SAT exam scores is with parental income; on average, students from the wealthiest families outperformed the students from the poorest families...

... by 400 points.

So is the idea here to further penalize teachers and administrators who work in schools in high-poverty areas?  Because that's sure as hell what it sounds like.

[image courtesy of the Wikimedia Commons]

New York State education officials are either unaware of these problems or else are ignoring them.  Ira Schwartz, Assistant Commissioner of Education, said in a memo, "The proposed changes would recognize efforts to encourage student participation and success in college preparation courses."

And unfairly penalize schools and teachers where other factors interfere with student success in these measures.  So Schwartz and Mary Ellen Elia, the State Commissioner, are either being disingenuous or else are once again proposing using standardized test scores as a way of instituting a top-down micromanagement approach that stifles creativity, destroys morale, and virtually eliminates local control.

"In December, the state’s education policymaking body suspended the use of those tests in teacher evaluations for the next four years," wrote Monica Disare in Chalkbeat New York.  "The moratorium is meant to give the education department time to redo the evaluation system.  This announcement, especially the references to SAT, AP, and other exams, offer early signs of how state officials will sort out that task and which new metrics they are exploring."

Also some early signs that what we're looking at is more of the same.  Evidently the Test 'Em Till They Can't See Straight approach, both here and in other states, has not been diminished despite objections from educators and the increasingly powerful opt-out movement.  You have to wonder what would make a difference.  Perhaps when they realize that they're driving experienced teachers from the profession, and discouraging college students from pursuing education as a career.

Or maybe that will just give them the impetus to gut the public school system completely, and replace it with corporate-run for-profit schools designed on the factory model.  Which is increasingly seeming like what they want.

Stop asking questions

I had a conversation with a student yesterday that illustrated all that is wrong-headed about public education.

Our school is fortunate enough to have an awesome selection of science electives, thanks to a forward-thinking principal we had ten years ago.  I teach an introductory neuroscience class, which is a great deal of fun for me and (I hope) the students -- we get to spend a semester looking at the intricacies and peculiarities of our brain and sensory processing systems.  The subject is one that seems to have a lot of intrinsic interest for high schoolers, and it's often the genesis of some pretty cool class discussions.

Last week I assigned them to read an article called "The Future of Brain-controlled Devices," about the possibility of brain-machine interfaces, and we discussed it in class yesterday.  What I had anticipated would be a ten-minute discussion of the scientific and ethical questions raised in the article turned into a whole period's worth.  The topics jumped, free-association-style, away from the subject at hand -- sensory enhancement, brain-to-brain connections ("machine-mediated telepathy"), virtual reality interfaces, restoration of sensorimotor abilities in the disabled -- and soon we found ourselves discussing the nature of seizure disorders, the role of sleep in memory consolidation, how the pleasure-reward circuit in the brain works, what happens when someone has a migraine, how visual pattern recognition works.

When the bell rang, one of my students chuckled, and said, in an amiable sort of fashion, "Wow.  We really got nothing accomplished this period."

I asked her what she meant.

She indicated the blank page in her science notebook.  "We didn't even write down any notes.  After we turned in the responses to the article, we just spent the rest of the period talking about random stuff."

I smiled and shrugged -- as I said, she's a nice kid and a good student, and didn't mean it as any kind of serious criticism -- but inwardly, I was a little appalled.  Here we have a senior in high school who has been taught, in her thirteen years in public schools, that a wide-ranging class discussion driven by the students' own curiosity, which never leaves the purview of the class's curriculum, somehow doesn't count unless they are made to write down lists of vocabulary words so they can study it later for the test.

These kids were focused and engaged, actively pursuing questions that they were interested in, driving their own learning and using me as a resource and a facilitator.  Not a single one tried to derail the conversation into other subjects; no one said, "So Mr. Bonnet, how do you think the New Orleans Saints are gonna do this year?"  We may have wandered off of the topic of brain-machine interfaces -- but would I really have been doing a better job as a teacher had I halted the discussion, and said, "Okay, stop asking questions.  On to the next topic, which is neurological disorders.  Get your notebooks out...."?

I think public schools, despite amazing obstacles, do a pretty damn good job of educating children.  But we do teach them an unintended lesson, one which some of them never unlearn.  It's the lesson that education is a passive enterprise, with the teacher as the knowledge donor and the student as recipient.  We tacitly pass along the message that if the information isn't on the test, they don't have to think about it, that it isn't worth knowing.  That we'll tell them what to write down, we'll decide for them what counts, that learning consists only of copying everything we write on our white boards into their notebooks.

And it's a notion that is fostered at every level.  If it's not a Quantifiable Outcome, if you can't measure it on a standardized test, to the federal and state departments of education, it does not exist.

[image courtesy of the Wikimedia Commons]

And just last week, the New York State Board of Regents voted to increase the impact of standardized tests by raising to 50% the contribution of exam scores to a teacher's end-of-year evaluation.  But we're apparently not supposed to ask questions about that, either.

You have to wonder why the powers-that-be in education seem so dead set against fostering creative, out-of-the-box thinking in children... or in the teachers themselves.

What does it mean to be truly educated?  It means having some kind of knowledge base, some fundamental set of facts at your disposal, sure.  But it's far more than that.  Education should foster creativity, drive, the ability to make new connections, the confidence and skill to be the author of your own understanding.  The If-It's-Not-On-The-Test-Don't-Waste-Your-Time mentality has warped what should be the true mission of schools -- to give children not only a set of tools, but a passion that will push them never to stop questioning, never to stop learning about the world around them.

And that memorize-and-test attitude has poisoned the children themselves.  I see it especially amongst the best and brightest, in the anxiety over scores, the fretting over learning every last definition, date, and detail.  Synthesis and questioning become a distraction.  They've learned their lessons well, and come away with the impression that minutiae are more important than curiosity.

We've come a long way from the original meaning of the word "education," which comes from a Latin verb meaning "to draw out of."  We've come to think of it as stuffing facts into children's minds, and after that, more facts still, and judge our success by how many of those facts they can successfully regurgitate on the end of the year tests.  How many opportunities for questioning, how many "teachable moments," are lost because we are chained to Student Learning Objectives and Measurable Outcomes?

I'll end with a quote from Socrates which I think sums the whole thing up:  "Education is not the filling of a vessel, it is the kindling of a flame."

Rules, ethics, and opting out

In the last few days, I've been prepping for the start of another school year.  My 29th, which boggles my mind a little.  And although I wouldn't turn down another three months of vacation, there's a part of me that's enjoying getting my classroom cleaned and ready, going through lessons and support materials, and wondering who's going to be in my classes this year and what joys and challenges they will bring along with them.

And as New York State teachers head toward the on-ramp, our Commissioner of Education is already beginning to polish up her own rhetoric in support of the Common Core and standardized exams, and against the opt-out movement.

This will be Commissioner MaryEllen Elia's first full school year in New York.  She comes to us from Florida, replacing Commissioner John King, who brilliantly illustrated the Peter Principle when he was promoted to the position of senior advisor to federal Secretary of Education Arne Duncan.  King, a toe-the-line demagogue who wouldn't hear any criticism of the haphazard fashion in which the Common Core and its attendant exams were rolled out in New York, is now in the position of seeing to it that the entire nation goes the same way.

Elia, unfortunately, seems cut from the same mold.

Three days ago she launched a campaign to fight the opt-out movement, which last year saw over 1.1 million participants across the state.  But instead of admitting that if the parents of over a million children are objecting to a policy, it might be time to reconsider it, she doubled down on her own stance -- and implied that the parents in the opt-out movement were simply uninformed.

"As you get more people involved in the process, you have more people understanding what’s going on and why you have assessments," Elia said.  "There are a lot of people that don’t know what the Common Core is...  We’re trying to pull together a tool kit, if you will, to support superintendents in how we can communicate in a much more effective way to people across the state.  I want the superintendents to understand the reflections and law that they can use as an information piece when they talk to people in their community … It’s important for them to be able to say, ‘Listen, it’s the law.’"

The problem is, it's not the law.  There is no law that mandates that students take tests, standardized or otherwise.  Republican Assemblyman Jim Tedisco, a vocal opponent of the standardized test movement, made this abundantly clear last year.  "They [NYSED and school districts] should be providing parents with the truths and the facts and their rights," Tedisco said in an interview.  "And their rights are yes, they can opt out of something they haven’t opted into. They can refuse something for their kids they’ve never opted into."

[image courtesy of the Wikimedia Commons]

As far as the parents who choose opt-out being uninformed, Elia may have stirred up a hornet's nest.  Jessica McNair, co-founder of the advocacy group Opt Out CNY, said, "I think she has a lot to learn about the parents in New York State.  We’re not going to back down until we see tests that are developmentally appropriate, and tests that are decoupled from the teacher evaluations."

But for the teachers who are participating in the pushback, Elia had even harsher words.  Such behavior, she said, was unethical.

"I think opt-out is something that is not reasonable," she said, at a meeting of Educators4Excellence.  "I am absolutely shocked if, and I don’t know that this happened, but if any educators supported and encouraged opt-outs, I think it’s unethical."  She has even hinted that teachers who recommend opting out to students or parents could be charged with insubordination.

It's unethical to follow the deepest core value of education -- to do what's best for children?  I have been unequivocal in my support for the opt-out movement; at this point, it's the only leverage parents and educators have against an upper administration that has a long history of being blind and deaf to the concerns of the rank-and-filers who spend nine months of every year on the front lines, and who know best the needs of their students.  They have chosen instead to take away the rights of the local districts to oversee their own assessments and teacher evaluations, and ceded that power to corporations like Pearson Education, who have over and over demonstrated that they are incapable of providing metrics that mean anything.

As Carol Corbett Burris, former principal of South Side High School in Rockville Centre School District and winner of the 2010 New York State Educator of the Year Award, put it:

(T)here comes a time when rules must be broken — when adults, after exhausting all remedies, must be willing to break ranks and not comply.  That time is now.  The promise of a public school system, however imperfectly realized, is at risk of being destroyed.  The future of our children is hanging from testing’s high stakes.  The time to opt out is now.

In other words, if Ms. Elia believes that such actions are unethical, then we as educators should welcome that label as a badge of honor.

If that makes us insubordinate, so be it.

And to Ms. Elia, I can only give a warning.  If you think that by demeaning teachers and parents as unethical and uninformed you can break our resolve, you have a lot to learn.  You think 1.1 million non-compliant children is a lot?

If you don't back down with the rhetoric, and look at how the system itself is failing children, you haven't seen anything yet.

Blocking the paradigm shift

Yesterday, something rather unusual happened; I left an educational staff development day without feeling like I needed to kick a small innocent furry woodland animal out of sheer frustration.

Usually, calling these things "a waste of time" is a tremendous understatement, but in yesterday's installment we were given a talk by Dr. Willard Daggett, the CEO of the International Center for Leadership in Education, and I left with a great many things to think about.

Daggett himself has not been immune from controversy, something about which you can form your own opinions after an easy Google search.  But that's not what I want to look at here.  His talk yesterday centered around moving from what he calls the "Urgent Issues" -- things like the Common Core Standards, APPR (Annual Professional Performance Review, the "teacher grading" system), state-mandated standardized tests, and budget cuts -- to looking at what most of us think of as the second-tier "Important Issues."

Those "Important Issues," Daggett says, are the ones that should be moved to the front of the line.  They include: the increasing gap between what schools do and what colleges and business leaders want; the rate at which colleges report students underperforming, needing remediation, and dropping out before graduating; and competition in the global marketplace with college graduates in technical fields who are better educated than the ones that the United States is producing.

In dealing with all of these, Daggett says, educators have been more reactive than proactive.  We keep doing things the same old way, even if the "same old way" isn't working so well.  We have ignored the research about how children learn, about what makes the content rigorous and relevant, about how to increase literacy and mathematical ability.

But the most important thing he said, in my mind, was when he started talking about how we teach our content areas as a disconnected series of facts.  I'm guilty of this myself; I've heard that in an introductory biology class, students learn more new vocabulary than they do in the first year of a foreign language.  We're shy on application and problem solving, and focus instead on teaching (and testing students on) a fact salad that has little connection to the real world.

It's time, he told us, to look at other ways of doing what we do.

I found myself unable to argue with much of what he said.  And this is despite the reputation I have for being something of an opinionated gadfly.  He showed a slide that he said represented how people feel who propose making major changes to schools:

The cat, of course, is the one who is proposing the changes, and the line of German Shepherds the school staff.  He used his laser pointer to point out the dog in the middle who looks like he's about to run out and eat the cat for dinner, and said, "And I'm sure that everyone in this room knows which faculty member's face should be on that dog."  At which point more than one of my colleagues looked in my direction.

So as I said, it's a minor wonder that I didn't get angry.  Because much of what he was saying is a stinging indictment of what I've spent the last 28 years doing, and Dr. Daggett had facts and statistics to bolster his contentions.

But here's the problem, something that I mentioned to him during a break, and for which he didn't have a very good answer.

How can teachers, administrators, and school boards institute a major paradigm shift when (1) students and teachers are still being evaluated on the same old metric of regurgitation-based standardized tests, and (2) legislators are still tying the funding of schools to students' scores on those tests?

Most of us recognized the problem before yesterday; the reason that Dr. Daggett's speech didn't raise more hackles is that the majority of the people in the room already know the scope of the problem (even if they may not have known the specifics he brought up).  But we're caught between knowing that the world is changing, and that we're not meeting student needs very well, and a leadership that demands that we assess student achievement the same way we've been doing for the past fifty years.

It's not the teachers who are blocking the paradigm shift.  It's the people in the State Department of Education who are designing all of the assessments, and the state legislature that is holding the purse strings.

But it was one of the last things he said that made me sit up and frown a little.  He told a personal anecdote about a family member who had had emergency surgery for a life-threatening injury, and compared the way surgeons are treated in hospitals with the way teachers are treated in schools.  "Teachers," he concluded, "are the equivalent of the front-line surgeons in hospitals.  We should treat teachers like surgeons."

All well and good to say.  We have a governor here in New York State who trusts teachers so little that we're not even allowed to grade our own final exams, because of worry that we'll cheat.  We have to fight tooth and nail for every pay increase we get, while the state aid that pays our salaries is cut every year, and the governor has mandated a property tax cap so that school boards couldn't even raise the tax levy if they wanted to.  We're at the whim of a Board of Regents that is so out of touch with what is happening in schools that they have publicly stated that standardized test scores should comprise 50% of a teacher's final "grade," and that if the teacher doesn't meet the benchmark on that 50%, the other 50% -- evaluation by administrators, classroom visits, and so on -- doesn't matter.

Hell, they don't even trust the administrators.  The latest proposal is to have classroom observations done by outside evaluators, to keep the principals from cheating.

Treat us like surgeons?  We're so far from that level of respect and (dare I mention it) salary that the comparison almost made me laugh out loud.  The distrust and disrespect current government leaders have for the teaching profession, and the resultant stress on teachers, is one major reason why we're hemorrhaging talent -- the best and brightest are finding other careers.  Consider, for example, the loss of Stacie Starr, winner of a 2014 Teacher of the Year award in Ohio, who in her resignation speech said, "I can’t do it anymore, not in this ‘drill ‘em and kill ‘em’ atmosphere.  I don’t think anyone understands that in this environment if your child cannot quickly grasp material, study like a robot and pass all of these tests, they will not survive."

So while I thought Dr. Daggett had some good ideas, we were left with no real direction for solutions.  The situation won't change until the leadership does, and I don't see that happening any time soon.  Until then, we're doing just what the State Departments of Education are mandating that the children do; focusing on disconnected details, and avoiding any application of what we know to the real world.

Teacher scores and error bars

One of the first rules of handling data that students learn in science classes is the concept of "significant figures."  Although the rules for determining whether a particular digit in a measurement or calculation is significant (i.e. reliably accurate) are a little complicated, the whole idea boils down to a simplistic concept:

When you do a calculation that combines various pieces of measured data, the result cannot be any more accurate than the least accurate piece of data that went into the calculation.

It's why you see "error bars" around data points in scientific papers.  You have to keep in mind how precise the data is, so that when you combine different measurements, you know how accurate the result is.  And the difficulty is that error is cumulative; the more pieces of data you combine, the greater the cumulative error becomes, and the lower the confidence that the outcome is actually right.

Which brings me to how teachers' grades are being calculated in New York state.

Our grades this year are a composite of three measures.  60% of our grade comes from numerical scores assigned by our principal from classroom observations; 40% comes from the outcome of our students' performance on tests (20% each from two different sets of tests).  This year, my two blocks of twenty percentage points each came from my AP Biology exam results, and the total of my student's results on my in-class final exams.  So, here are my results:

I got 60/60 on classroom observations.  I got 20/20 on my AP Biology exam results, which is mystifying for two reasons: (1) the exam itself was a poorly-designed exercise in frustration, as I described in a previous blog post; and (2) three of my 27 students got a 2 on the exam, which is below the benchmark, so my score should have been knocked down a peg because of that.

I got a 10/20 on my in-class final exam results.

Why?  A combination of reasons.  The state, in their desperation to pretend that all outcomes are quantifiable, required that for the purposes of calculating our "teacher grade," the exit exam score had to be compared to a "pre-test."  My pre-test, in AP Biology, was the combination of the students' Regents (Introductory) Biology and Regents Chemistry final exams -- both markedly easier tests.  Every student in my class scored below their pre-test score on my rigorous, college-level final, so in the state's eyes it looks like the year they spent in my class actively made them stupider.

I also got graded down because of the three students in my elective who chose not to take the final exam.  You might ask yourself why the teacher should be blamed for a student's choice to skip the day of the final.  The state has a ready answer: "It is the teacher's responsibility to make certain that all students complete the requirements of the course."  (That's a direct quote, folks.)

So, my overall grade this year is a 90, which you'd think I'd be pretty pleased with.  Actually, I'm not, because my grade -- supposedly, a measure of my effectiveness as a teacher -- isn't a 90 at all.  What should it be, then?  Damned if I know.  We've combined three measurements to get that score that were all measuring different things, at different accuracies.

Remember error bars?

Were my classroom observation scores accurate?  I'd say so, and I'm not just saying that because I scored well.  The principal I work for is outstanding, and has a real sense of what good classroom teaching is.  Of the three measures, I'd say that this is the one I'm the most confident of.

How about the 40% that came from test scores?  Honestly, I'd say that number has a wobble factor of at least ten points either way.  In part, the test score outcomes are due to my effectiveness as a teacher; it'd be a sad state of affairs if how my students performed had nothing to do with me at all.  But are there other factors involved?

Of course.  On the plus side, there's the hard work the students put in.  Dedication to a class they've enjoyed.  Good study skills.  Raw intelligence.

On the minus side, there's poverty.  Cognitive disabilities.  Lack of parental support.  Bad attitude.  Frustration.  Laziness. 

To name a few.

So, really, how confident are you that my grade of 90 is actually a reflection of my effectiveness as a teacher?  Because that confidence can't be any higher than the least accurate measure that went into calculating it.

The funny thing is, this statistical concept is one that is taught in every Educational Statistics class in the world, and yet the powers-that-be in the State Department of Education have been completely unresponsive to claims that the way they're handling numbers is spurious.  Of course, I don't know why we should expect any different; the way they handle scaling final exams in New York state is also spurious, and they have feigned deafness to objections from teachers on that count, too.

As an example, on the state biology final, students have consistently needed to get 46% of the answers correct to score a scaled score of 65 [passing], while on the physics exam, the fraction of correct answers students need to score a 65 has varied from 59% to 67%.  Yes, that's correct; there have been years where exam scores in physics have been scaled downward.  When questioned about how this can possibly be fair, Carl Preske, Education Specialist at the New York State Department of Education, responded (this is a direct quote):

I promised myself that I would not join in any discussion of negative curve and the quality of the questions.  So much for promises, unless you personally have a degree in tests and measurements  I doubt that you have the expertise that the twenty teachers who have worked on each question.  Secondly if you lack a degree in psychometrics than [sic] comments on negative curves are useless. That being said,  each subject area established their own cut points for 65 and 85 more than 10 years ago: we (those constructing the physics exam) have the advantage of having a much larger number of difficult questions to place on each exam than does Chemistry  and with that greater number of difficult questions we are able to avoid what you prefer to call a negative.  Since we have about 20-25 questions above the 65 cut point we are able to stretch out the top 35 scaled credits,  Chemistry has between12 and 18 questions above the cut point over which they may scale the 35 credits.   If you wish to remove the "negative curve" than [sic] please find a way to generate 20 difficult questions to give to the test writing group each year.

Well, that was lucid.

So, we're basing teachers' scores on a combination of metrics based on the scaled scores of flawed tests.

Remember the idea of error being cumulative?  ("Your score is a 90!  ± 50 points!")

Now, you may be thinking, what real difference does a teacher's score make?  How can it be used against them?  My own opinion is that we are, country-wide, moving toward using teachers' end-of-the year scores for purposes of awarding (or revoking) tenure, job retention, and (ultimately) raises and salary.  None of that has happened yet.  But already, these scores are being considered reliable enough that they are being used as a criterion for the awarding grant money.  I just saw last week an offer of research grant money that was open to teachers -- but only if you were considered "Highly Effective," that is, you scored a 91 or higher for the year.

That's right, folks.  If I'd gotten one point higher, I would be able to apply for a four-year research grant worth $15,000/year.  But I'm only "Effective," not "Highly Effective," so there you are.

The whole thing is intensely frustrating, because it seems like all of the rank-and-file teachers grasp the problem with this immediately, and none of the higher-ups in the State Department of Education are even willing to admit that what they're doing is statistically invalid.  Their attitude seems to be that if it can be converted to numbers, it's real.  And if it's real, it can be converted to numbers.

Oh, and if it can be converted to numbers, it's valid.  Right?

Of course right.

Me, I'm just going to keep loping along doing what I've always done, teacher score be damned.  I told a colleague this year that I didn't care what I got as long as it was above a 65, because if I "failed" I'd have to do more paperwork, which makes me sound like one of my less-motivated students.  But I know that what I do in the classroom works; I know I'm effective.  Whether I got a 90, or a 100, or a 72, means absolutely nothing, neither in the statistical sense nor in any other sense.  What we do as teachers has an inherently unquantifiable aspect to it.  How can you measure students' excitement?  Or creativity?  Or the sense of wonder they get at learning about the world?  Or the moment that a kid decides, "I love this subject.  I want to spend the rest of my life doing this?"

But the b-b stackers in the state capitol don't, apparently, recognize any of that as valuable.  It's a good thing that most of us teachers still do.

How not to evaluate educators

In his talk "The Surprising Truth About What Motivates Us" (which you can, and should, watch in its entirety here), Daniel Pink describes how the micromanagement and punishment-and-reward system built into the American workplace model simply don't work.  Pink, who worked as an aide to Labor Secretary Robert Reich and who was listed as one of the fifty most influential management thinkers in the world, describes several studies that have findings that are astonishing -- including one that found that in tasks that require engagement and creativity, reward-and-punishment strategies lead to poorer performance.

But what interests me most comes in the middle of the talk, where he describes an Australian software company, Atlassian, that instituted a policy that runs completely counter to the usual business model.  Once a month, every employee at Atlassian is given an entire day to do whatever they'd like to -- in Pink's words, "you can work on whatever you want, with whoever you want, but you have to show us the result in 24 hours."

"In that one day of pure, undiluted autonomy," Pink states, "this has led to a whole array of fixes for existing software, and a whole array of ideas for new products that otherwise would never have emerged."

Autonomy, then -- up to a point, certainly, but far beyond what most businesses are willing to try -- increases productivity, engagement, motivation, and morale.  We do our best, most creative work without someone breathing down our necks.

Which brings me to the latest from the New York State Department of Education.

Some of you may remember from previous posts that the most recent brainstorm from NYSED is that this year the teachers are getting numerical grades.  Yes, folks, in a month or so, I'm going to be getting a report card!

Only fair, you might say, given that I do the same to my students.  And honestly, I have no problem with accountability; I welcome being observed and evaluated.  There is nothing to be gained by a policy that allows inadequate teachers to continue indefinitely in the same job.

The problem is the way the grade is calculated.  Part of it is based on observations by an administrator; but 40% comes from how well students meet "SLOs" -- "Student Learning Objectives" -- based on their performance on standardized tests.  To determine whether the students met their SLOs, an exit exam is administered, and used to see if the student met or exceeded a "target" score based on scores on a pre-assessment.

Let me give you an example from my own class that will illustrate why this is a statistically spurious method.

I teach, amongst other things, AP Biology.  It's a notoriously tough subject, full of technical terms and difficult concepts.  For this class, the "pre-assessment" used is the average of the students' scores on the Regents (Introductory) Biology and Regents Chemistry exams.  Now, the problem is, these are both dramatically easier classes and exams; most of the students who made it to AP Biology scored in the 90s on the Regents Biology exam and at least in the 80s on the Regents Chemistry exam.  So the students walk in with, most of them, a "pre-assessment" score of around 90.

Then, at the end of the year, they hit my cumulative, college-level final.

I can say that almost without exception, everyone scores lower on that exam than they did on their "pre-assessment."  In fact, this year, 12 out of 23 students who took my final did not meet their "state target scores" -- in other words, in New York State's eyes, the students, and I, have failed.  From the point of view of the people at NYSED, it looks like the year my students spent in my class actively made them stupider.

But it gets better.  Because just yesterday, we received a communiqué from Ken Slentz, the Deputy Commissioner of Education in New York State.  The message stated that students who are not eligible to take the science Regents exams because they did not turn in enough labs during the school year -- New York has a requirement that high-school-level science students do a minimum of thirty hours of labs to get credit -- are counted as zeroes against the teacher's score.

You read that right.  A student who was out for medical reasons, and who is on home tutoring and cannot do labs -- which three of my students were, this year, for months at a time -- not only is penalized by not being allowed to take the final exam, but that zero is counted as a failure of the teacher's.  Students who simply disengage, and decide that they have no particular interest in turning in anything, are also counted against the teacher's evaluation score.  "It is the responsibility of the teacher to ensure that all students meet lab requirements so that they are able to sit for the Regents exam," Slentz wrote in an email that was sent to every high school science teacher in New York.  "The Department recommends that districts... create processes that ensure students have opportunities to make up lab requirements."

As a student put it, when I discussed this policy in my Critical Thinking classes, "Wow.  First, teachers have an incentive to give easy final exams.  Now they have an incentive to lie about whether we turned in our lab reports."

In the case of the teachers in my school, I have to say that virtually all of them have responded with indignation.  "I'll be damned if I'll compromise the integrity of my course for some bullshit rule," one said.  "If the state wants to grade me down because of things that are outside of my control, they can knock themselves out."

But what it has done is to destroy morale.  More and more teachers I know are actively looking for other jobs.  One, a friend of long standing who has young children in the school, is looking into finding a way to take her own children out of the public school she teaches in -- a stinging vote of no confidence in the direction public education is going.

My fear, though, is that this trend of turning everything -- students and teachers alike -- into numbers is only beginning.  Micromanaging b-b stackers like Deputy Commissioner Slentz, who evidently don't have the vaguest idea of the reality of classroom teaching, will accomplish nothing by these new mandates but driving those of us who actually care about educating children into other jobs.

As for me: I'm not too far from retirement, and would be eligible for a buy-out (should one be offered) in three years.  I can stick it out that long.  How much longer I'll be able to keep my morale up is another matter.  But after all, given the mediocre grade I'm likely to get on my Report Card this year, it's probably better that I start thinking about getting a job where my evaluation is actually based on my level of performance.

If you can't teach...

Last week, I was the unfortunate victim of a science department meeting at my school.  I loathe meetings to begin with, so I already had a bad attitude going in.  (Although in the interest of fairness I will admit that I have attended interesting, productive meetings.  I believe there was once in 1988, and that one time in 1997.)  The focus of this particular meeting was Reworking the Standards for Science Education in New York State.

The very title given to this meeting set my teeth on edge.  It's not that I'm against high standards; it's just that every time I have been part of a campaign on the part of the State Education Department to rework the standards, the standards have gotten weaker.  The last big overhaul of the science standards in New York State resulted in the New and Improved Regents Exam, which features questions like:

42.  In what organ in the human body does the embryo develop?  (a) liver (b) pancreas (c) stomach (d) uterus

What always makes me laugh, in a bitter and cynical fashion, is the way that NYSED and other government agencies give these initiatives catchy,  buzzwordy names, presumably so as to trick all the teachers into supporting it because we don't know how to read the fine print, or presumably, to think.  The last one was "Raising the Bar."  Given that the "Raising the Bar" initiative generated questions like the aforementioned, it seems like they Raised the Bar High Enough for Everyone to Walk Under It.  The current push is part of the federal government's Race to the Top, which mandates (among other things) using increased standardized testing as a way of improving student learning, because we all know how well that works.  To make someone a better driver (for example), all you have to do is administer the written driver's exam over and over.  Right?  Of course right.

The Race to the Top initiative is a competitive game (thus the "race" analogy) to garner funding by acquiescing to whatever damnfool thing has become the Flavor of the Month at the US Department of Education.  And desperate for funding, we've all rushed headlong into the race, accepting whatever deals with the devil we've had to make along the way, thinking, "It'll be all right in the long run.  We need money."  And after all that, I found out that the federal grant money our district got from winning the Race to the Top was... $41,000.  Only a little more than the starting salary of a single first-year teacher.  For this, we compromised our own standards, accepted the standardized testing, the tying of teacher evaluation to student test scores, and the rest?

Of course we did.  We were cornered.  Our superintendent, for whom I have a great amount of respect, said to me, "If we don't join RTTT, we're going to end up having to acquiesce to the new rules anyway, and get no money at all."  I think she's right.

In the end, of course, all of these initiatives end the same way; life goes on.  We go on teaching what we've taught, in the way we've always done.  Why?  Because, for the most part, it works.  All of the Vertical and Horizontal Alignment, and so forth, that NYSED is currently requiring us to do, will make not a point's worth of difference to student learning.  What makes the most difference to student learning is, and always has been, having a passionate, committed teacher, who knows the subject thoroughly and cares about it deeply, and can instill a sense of enthusiasm into his/her students.  These aren't my words; these are the words of the students in my Critical Thinking class last year, when we discussed the educational system, where it works, and where it falls down.  And they were unanimous in agreement that the first and foremost determiner of student success in a course is having an outstanding instructor.  And if you can show me how Raising the Bar and Race to the Top accomplishes that end, I would be much obliged.

It puts me in mind of the old quip.  You know it?  "If you can, do; if you can't, teach."  Understandably, I've never agreed with that.  But if you add a third clause, I'm more likely to agree:  "If you can't teach, join the Department of Education and tell the teachers how to do their jobs."