Skeptophilia (skep-to-fil-i-a) (n.) - the love of logical thought, skepticism, and thinking critically. Being an exploration of the applications of skeptical thinking to the world at large, with periodic excursions into linguistics, music, politics, cryptozoology, and why people keep seeing the face of Jesus on grilled cheese sandwiches.
Showing posts with label Pearson Education. Show all posts
Showing posts with label Pearson Education. Show all posts

Friday, June 26, 2015

Plan of attack

The time has come to ask what exactly the people in charge of overseeing public education are trying to accomplish.

And I'm sorry, "Improving public education" isn't a good enough answer.  Nor is "making sure we have teacher accountability."  I want to know, specifically, why our elected officials and educational leaders are moving in the direction they are, along with evidence of how their decisions will work to accomplish their goals.

Because at the moment, a lot of it looks like a carefully-designed program to tear down the entire edifice.

[image courtesy of photographer Svetlana Miljkovic and the Wikimedia Commons]

Here are a couple of the latest salvos.  So let me tell you about those, and see if you agree with me.

Let's start, as so many attacks on education do, in the state of Texas, where Governor Greg Abbott has appointed Donna Bahorich as the chairperson of the State Board of Education.

Bahorich is an ultraconservative who backed last year's decision to approve history texts that claimed that the United States Constitution is based on the bible, and that the American system of democracy was inspired by Moses.  She is strongly in favor of using money from taxpayers to support private religious schools.  Her qualifications?  She was a manager of a telephone company, campaign treasurer for Senator Dan Patrick (who himself has gone on record as stating that creationism should be taught in public schools), and a member of the pastor's council at Houston's Vineyard Church.  Her M.A. is in counseling from Jerry Falwell's fundamentalist bastion, Liberty University.

She chose to homeschool her own children.

So to summarize: Texas now has a woman in charge of their educational system who has never taught, did not send her own children to public school, seems to have no qualifications for the position whatsoever, and has shown herself to be an ideologue who would love to see free secular public education replaced by publicly-funded religious schools.

Even some of Abbott's Republican supporters think this is a misstep.  State Board member Thomas Ratliff said, "Public school isn’t for everybody, but when 94 percent of our students in Texas attend public schools I think it ought to be a baseline requirement that the chair of the State Board of Education have at least some experience in that realm, as a parent, teacher, something."

Yeah.  You'd think so.  But Texas isn't the only one.  Right here in my home state of New York, we have Merryl Tisch as chancellor of the State Department of Education, a woman whose sole experience with teaching was seven years in two wealthy private Jewish schools.

Is it becoming a requirement that in order to lead public education, you need to have no experience with public education?

Then, we have the ongoing attacks on teachers via the reliance on standardized tests to measure not only student progress, but teacher competence.  So given how much is resting on the outcome of those tests, you'd think that there'd be a great deal of emphasis on having qualified scorers, right?

But according to an exposé in The New York Times, the high-stakes exams from Pearson Education and other testing-for-profit corporations are being graded by people who not only have no teaching experience, but no background in pedagogy whatsoever:
There was a onetime wedding planner, a retired medical technologist and a former Pearson saleswoman with a master’s degree in marital counseling.  To get the job, like other scorers nationwide, they needed a four-year college degree with relevant coursework, but no teaching experience.  They earned $12 to $14 an hour, with the possibility of small bonuses if they hit daily quality and volume targets.
Bob Sanders, vice president for Content and Scoring Management at Pearson, said that none of that mattered.  "From the standpoint of comparing us to a Starbucks or McDonald’s, where you go into those places you know exactly what you’re going to get.  McDonald’s has a process in place to make sure they put two patties on that Big Mac.  We do that exact same thing. We have processes to oversee our processes, and to make sure they are being followed."

You know, Mr. Sanders, if you're trying to make an argument that you're dedicated to quality, comparing yourself to McDonalds might not be your best choice.

And then, we've got the more fundamental problem that the evaluation system itself is faulty.  Let's take my own situation as a case-in-point, because I just got my "final grade" for this school year yesterday.  The numerical grading system in New York has been in place for three years.  Two years ago, I scored a 92.  Last year, I scored a 91, missing the "highly effective" designation by one point.

This year, I scored an 80.

So according to New York State, I became 11% worse at teaching between last year and this one.  Good thing I didn't drop more than that; another 5% downward, and I'll be classified as "Developing," which is kind of funny given that I've taught for 28 years.  If I haven't "Developed" by now, I don't think it's gonna happen.

And the tailspin in my score is, of course, being laid at my feet, because there can't be any other contributing causes, right?  It couldn't be because of things like differences in student effort and work ethic from year to year, changes in the exam scaling, or the fact that 50% of one of the classes I was evaluated on this year were special needs students, including two tenth-graders who read at a fourth-grade level.

Nope, the drop in scores was clearly my fault.

What the hell is going on here?

Elected officials are appointing people to leadership roles in education who have little to no experience in public education.  A teacher evaluation system has been put in place that is not only so faulty that a student in a freshman statistics class could see the flaws, but has been handed over to for-profit corporations who farm out the actual scoring to people who have never spent a day working in a classroom.

You know, I don't tend to buy into conspiracy theories, but this is more and more looking like a well thought out strategy to destroy public education from the ground up.

I hope I'm wrong.  Incompetence and mismanagement are easier to forgive than deliberate, calculated malfeasance.  But as the problems pile up, and the solutions appear designed to make things worse, and the people appointed to be in charge continue to be selected from the ranks of anti-education demagoguery, my confidence that we're not seeing some kind of coordinated attack is becoming weaker and weaker.

Thursday, April 30, 2015

A conspiracy of engineered failure

Robert Heinlein said, "Never attribute to conspiracy what is adequately explained by stupidity."  I think this is a pretty good rule of thumb.  It's not that conspiracies don't exist; it's more that humans aren't very good at them (e.g. Watergate), and we're much more likely to be acting from venial motives -- greed, duplicity, desire for power -- than we are to be engaging in some kind of deep and sinister plot.

That said, I'm beginning to wonder if the implementation of the Common Core isn't some kind of conspiracy.

Let's start with the fact that the Chief Operating Officer of the South Carolina Department of Education, Elizabeth Carpentier, is threatening parents who allow their children to opt out of state tests with thirty days in jail.

State Superintendent of Education Molly Spearman, asked about Carpentier's statements, declined to comment other than saying that there is "no statutory provision for parents to opt their children out of testing."

[image courtesy of the Wikimedia Commons]

Arne Duncan, Secretary of Education, has been making some veiled threats of his own.  "It’s just part of most kids’ education growing up,” he said, regarding the implementation of the Common Core standardized exams.  "Sometimes the adults make a big deal and that creates some trauma for the kids...  We think most states will do [meet their targets for participation].  If states don’t do that, then we have an obligation to step in."

So the implication is that everything is just hunky-dory with the Common Core curriculum and the tests, and the only parents that will opt out are overprotective mommies and daddies who can't stand to see their kids work hard.

Oh, and those mommies and daddies might have to go to jail if they allow their kids to opt out.

So we have threats coming from the powers-that-be, but there's a flipside to this.  If the curriculum and the tests were fair measures of student achievement or teacher competence, the opt-out movement might have less of a basis for their argument.  But the implementation of these exams, and their content, has been riddled with problems.  Psychologist Dr. Charlene Williams writes:
The 6th grade ELA practice performance task for the Smarter Balance was completely inappropriate for 11-12 year olds, requiring them to toggle between several screens (on small Ipad screens), and choose multiple pieces of evidence to evaluate, select, paraphrase, compare and contrast, as well as write a multiparagraph essay. Never mind that while practicing, toggling back to the articles caused the students’ written work on the essay to be erased (lost).
Williams goes on to challenge the exams' validity on every level:
1) There is no proven Construct Validity (does your test measure what you think it measures). 
2) Cut scores are determined by an unknown (arbitrary) process - labeling children as proficient or failing appears to not be based on any scientific process.  It is not scientific to arbitrarily decide what levels of your test scores actually mean in the real world.  Scientific measurement requires cross-validation with external measures that provide evidence for your claims (like grades, or independent in-depth measures of children’s educational achievement in a a smaller sample with highly experienced evaluators). 
3) Computer adaptive tests - there have been many concerns raised about how item difficulty has been decided. Children continue to progress on these tests if they continue to get a certain number the most recent answers correct. Educational measurement specialists (true academically trained professionals) and parents and children have observed that very often items following very difficult questions are significantly easier. This raises concerns that children’s scores are artificially deflated by unscientifically determined item difficulty determinations. 
4) Inter-rater reliability - No checks exist to independently determine whether the scoring administered by these testing companies has truly reliable and valid measurements of children’s answers... The assessments are not verifiable, because they are not permitted to be subject to independent scientific evaluation.
This last point is especially troubling.  Anything we get to hear about the exam content has to be "leaked," because the people who see the exams are prohibited from discussing them.  This mandate comes not from state Departments of Education, but from Pearson Education, the corporation who designs the tests.  Educator Elizabeth Phillips, in an op-ed piece for The New York Times, writes:
I’d like to tell you what was wrong with the tests my students took last week, but I can’t. Pearson’s $32 million contract with New York State to design the exams prohibits the state from making the tests public and imposes a gag order on educators who administer them.  So teachers watched hundreds of thousands of children in grades 3 to 8 sit for between 70 and 180 minutes per day for three days taking a state English Language Arts exam that does a poor job of testing reading comprehension, and yet we’re not allowed to point out what the problems were.
But some educators care more about their students than they do about threats of repercussions by a rich corporation who is pulling the strings of upper-level administration in every state in the United States... and those educators have uncovered some frightening facts.  For example, a teacher who (understandably) wants to remain anonymous performed a reading-level analysis on a passage from the 4th grade ELA assessment, and found that it had a lexile score of 1140 -- corresponding to appropriateness for the average reading ability of an 8th grader.

The skew between the difficulty level of the exam material, and the grade for which it was targeted, prompted New York educator Stephanie Santagada to write a letter to Governor Andrew Cuomo using vocabulary culled directly from the 4th, 6th, and 8th grade reading assessments:
There is a man in Albany, who I surmise, by his clamorous paroxysms, has an extreme aversion to educators.  He sees teachers as curs, or likens them to mangy dogs.  Methinks he suffers from a rare form of psychopathology in which he absconds with our dignity by enacting laws counterintuitive to the orthodoxy of educational leadership.  We have given him sufferance for far too long.  He’s currently taking a circuitous path to DC, but he will no doubt soon find himself in litigious waters.  The time has come to bowdlerize his posits, send him many furlongs away, and maroon him there, maybe Cuba?
So yeah, I'm beginning to think there's more here than simple incompetence.  The people in charge are not stupid, and one thing these people excel at is number crunching.  I flatly refuse to believe that the inclusion of a passage that is four grade levels too hard in the ELA assessment was a simple blunder.  Which leads us into into some scary territory, because that implies that the other problems may not be accidents, either.

But why would Federal and State Departments of Education, with the collusion of a lot of elected officials like our own aforementioned governor, do this?  Cuomo himself tipped their hand earlier this year, when he said that he wanted to change the teacher evaluation system -- because too many teachers were achieving high scores by the previous metric.

So the reason for all this, I believe, is that the powers-that-be are deliberately setting students up to fail, in order to show up public education itself as a failure -- in an effort to destroy the entire edifice. Replacing it, more than likely, with a network of charter schools that are run by privately-chosen (i.e. not elected) boards, and which will have a vested interest in buying in to programs, curricula, and assessments created by for-profit corporations like Pearson.

The whole thing, I think, boils down to money, and who is lining whose pockets.  So in the end, it does turn out to be venial motives -- greed and political power.

I could be wrong.  It might be that what we're looking at is the educational equivalent of the Keystone Kops, running around frantically and bonking into walls and falling over.  That's certainly what it looked like at first.  But now, with the time we've had to smooth over problems, develop exams, streamline administration, we shouldn't be seeing these kind of mistakes.  Giving so many tests might still be a mistake, of course; but the fact that the exams themselves are so deeply, fundamentally, and obviously flawed, coupled with the gag order against discussing them, has the hallmark of deliberate downward manipulation of the scores.

Which means that it is even more important for parents to defy the threats, and opt their children out, and for educators to come forward with the content and administration problems of the exams themselves.

Public education itself might be at stake.

Friday, May 2, 2014

Computerized essay, computerized grade

Honestly, I didn't need another reason to hate the increasing barrage of standardized tests that has come to characterize the American approach to public education.

I've seen enough of its ill effects already.  Demoralized kids, who daily face curricula that have turned into a hodgepodge of minutiae and generalities, with little emphasis on connections or critical thinking.  The "teach-to-the-test" mentality becoming abundant amongst teachers and administrators -- driven, it must be said, not by laziness or ineptitude, but because they are now being evaluated by how well the students perform on these metrics.  Writing that is graded on meeting a set of bullet-point rubrics that often have little to do with depth of understanding, creativity, nuance.

But just yesterday, I found yet another reason to despise the direction our educational system is going.  Because apparently, the latest push in the educational assessment world is to take essays -- the last bastion of expressive thought in an increasingly fill-in-the-bubbles world -- and score them by computer.

[image courtesy of the Wikimedia Commons]

I'm not making this up.  There is now software out there -- Intellimetric, eRater, and Project Essay Grade, for example -- that developers claim can take an essay written by a student on a computer and come up with a score that matches to a high degree the score that would be given by a trained human reader.  There's also "WriteToLearn Automated Language Assessment" -- offered by none other than Pearson Education, who seems to be becoming to the educational world what Monsanto is to the environmentalists.

Proponents say that humans are fallible, biased, tire easily, can be sloppy, can cheat.  Which is all true, of course.  But the people who are using machine scoring of essays are confident to the point of hubris: "ETS has been at the forefront of research in automated scoring of open-ended items for over two decades," reads the description of the use of automated scoring protocols on the Educational Testing Service website, "with a long list of significant, peer-reviewed research publications as evidence of our activity in the field.  ETS scientists have published on automated scoring issues in the major journals of the educational measurement, computational linguistics and language testing fields.  Their work has also resulted in 19 U.S. patents related to applying NLP in assessment, significantly more than any other organization."

And it's already in use.  The GMAT (Graduate Management Admission Test) essays are at least in part machine scored, and the PARCC (Partnership for Assessment of Readiness for College and Careers) assessments are supposed to be following suit in the 2014-2015 school year.  So evidently, the ETS people and their pals at Pearson and other educational software development corporations have done their public relations jobs well.  The educational establishment, it seems, is sold on automated essay scoring.

Which puts them in a kind of awkward position apropos of a piece of research published in Phys.org just yesterday, which showed that a simple piece of software called "BABEL" (Basic Automated B.S. Essay Language), developed by programmer Les Perelman at MIT, can produce a high-scoring essay, according to the automated scoring software -- despite the fact that the output of BABEL is meaningless gibberish.

Which implies that students could do the same.  Further implying that what the automated scoring programs are detecting is not writing quality.

Given the prompt to write about "privacy," BABEL produced an essay that scored a 5.4 out of a possible 6, according to the automated scoring software, despite the fact that it contained the sentence, "Privateness has not been and undoubtedly never will be lauded, precarious, and decent."  The whole essay was written that way, i.e., complete and utter bullshit, composed of random words strung together into fancy-sounding complex sentences with lots of commas and subordinate clauses. Since the automated scoring software was looking for complexity of sentence structure, word length, and word commonness as some of its criteria for the overall score, and could not actually discern the meaning (or lack thereof) of the passage, the program was fooled.

Which means, of course, that there's no reason that humans couldn't similarly game the program.  Learn how to string some ten-dollar words together, put in a few cool phrases like "will certainly be, despite suggestions to the contrary," and figure out how to do parallel construction, and apparently it doesn't matter if you're saying anything that's meaningful.

Look, I know I'm a bit of a Luddite, but it's not that I don't trust technology per se.  I just think that thus far, it has some significant limitations.  We are not yet -- and chances are, won't be for some time -- within hailing distance of a sentient computer, that would be able to understand the nuance and connotation of written or spoken language.  All of the apps and programs and bells and whistles that seem to be taking the educational world by storm are no replacement for a truly engaging teacher.  Even if the software improves dramatically, I would question its utility as anything more than a clever teaching tool, and something that a skilled teacher really can get along just fine without.

But I find the idea that we are further mechanizing the act of teaching -- an act that is, when done well, far more an art form than it is a science -- profoundly repulsive.  As of two years ago, we were told by the state of New York that we teachers are not trusted sufficiently to grade our own final exams, so we have to give the exams to other teachers to score.  Now, apparently, we're moving to taking the assessment of our students out of human hands entirely.

Next, I fear, we will see the teachers themselves replaced by software -- with the ETS and Pearson and so on lauding the changes as visionary, and describing the "peer-reviewed research" the scientists on their payroll are doing, that shows how effective it all is.  "Students learn best with an interactive computer-based tutorial," I can see the press release saying.  "We have been at the forefront of non-teacher-based instruction for decades!"

No teachers necessary, right?  Just some low-paid aides to keep the kids pointed at the computer screens.  Consider the savings to the taxpayer!

More and more we are seeing an emphasis on processing children through factory-model schools, as if they were little automata that could be tweaked and turned and programmed and all come out identically "career and college ready."  There is scant emphasis on creative, original thought, because, after all, how could you assess that, turn it into a number?  And you know, if you can't quantify it, it doesn't exist.

And I suspect that Perelman's result with BABEL will be met with a thunderous silence.  The educational establishment has a sorry history of ignoring research that would cause them to have to make a shift in the status quo, especially when said status quo is making a lot of money for the corporations that are now holding the purse strings.

Easier, apparently, to brush off a 5.4/6 on a nonsense essay than it is to admit that the entire system is headed in exactly the wrong direction.

Monday, April 28, 2014

The Common Core product placement conspiracy

I've been asked, at times, if I think it's possible that some conspiracy theories may be true.

And my answer is: of course.  Humans conspire.  It's the evil side of being a social animal; we sometimes use our social proclivities for immoral reasons.  So clearly there are some cases where "conspiracy theory" is actually "conspiracy fact."

It's just that I don't think most of the best-known ones -- the 9/11 inside job/controlled demolition claim, the Sandy Hook crisis actors claim, the HAARP weather modification claim -- have a scrap of evidence in their favor.  But other ones?  They have to be evaluated on their own merits, or lack thereof.

This comes up because I ran into a story yesterday that has all of the hallmarks of a conspiracy -- and yet I find it entirely believable.  I will say up front that I have no hard evidence that my claim is true.  It may be that what occurred here was entirely innocent, and the individuals involved are being straightforward and aboveboard.

But my opinion is that this stinks to high heaven.

I found the article because I was doing some reading about the latest on the Common Core, the ill-conceived and poorly-executed fad of the month for fixing public education by administering more standardized tests.  Oh, the standards themselves look okay: more understanding of math holistically, with less focus on minute details of symbol manipulation; more reading from source materials and writing to express understanding.  But the way it has been implemented has been haphazard at best, with an emphasis on test preparation, leading up to nationally-administered exams that put millions of dollars in the pockets of corporations like Pearson Education.

There has been a groundswell of anger about the whole thing, not only from teachers and administrators, but from parents.  This has led to a growing opt-out-of-the-tests movement, which is gaining traction across the country.  Oh, the fourth grade reading exam is tomorrow?  I'm sorry, my son is sick.  The make-up exam is in three weeks?  Oh, darn, I'm sorry.  He's going to be sick then, too.


But right here in my home state of New York, there's been a revelation of an even darker side of the whole thing.  A story in the Syracuse Post-Standard ran a few days ago that described the fact that the Common Core exams, provided by Pearson, include multiple examples of...

... product placement.

Yes, you read that right.  This has gotten out despite Pearson's tight control on the exam content -- teachers and students are expressly forbidden to reveal, much less copy and distribute, test materials, even once the test has been administered.  But you can't keep something this egregious a secret.  So the 3rd through 8th grade assessments were barely collected and scored when it came out that Barbie, Nike, iPod, Life Savers, and Mug Root Beer had all made their way into the exams.

And not in subtle ways, either.  The Nike brand and slogan appeared in a paragraph about risk-taking.  A story about a busboy cleaning up a root beer spill was specified as cleaning up Mug (a trademark owned by PepsiCo).  They weren't even taking the trouble to hide the product placement, the way that television sitcoms do -- such as placing a two liter bottle of Coca-Cola on a counter in the background of a scene.  This was blatant.

Everyone involved, of course, has denied that Pearson is receiving kickbacks from the corporations that produce the products mentioned.  "There are no product placement deals between us, Pearson or anyone else," Tom Dunn, an Education Department spokesman, said in an interview on Fox News. "No deals.  No money.  We use authentic texts.  If the author chose to use a brand name in the original, we don’t edit."

Mmm-hmm.  And if several of those texts mention products from multi-million dollar corporations, that's purely a coincidence.

Right.

The growing influence of corporations on education is a horrifying trend.  Kevin Kumashiro, writing for the American Association of University Professors, writes:
Current reforms are allowing certain individuals with neither scholarly nor practical expertise in education to exert significant influence over educational policy for communities and children other than their own.  They, the millionaires and billionaires from the philanthropic and corporate sectors, are experimenting in urban school districts with educational reform initiatives that are not grounded in sound research and often fail to produce results.  And yet, with funding for public education shrinking, the influence of these wealthy reformers is growing...  The result is a philanthropic sector that is inseparable from the business sector, advancing school reforms that cannot help but to be framed by corporate profitability.
And that's just it, isn't it?  The corporate leaders are not specialists in education; they have never been teachers, administrators, served on school boards.  It is just that by virtue of the money they have, they feel entitled to direct educational policy.  And the sad fact is, because of the increasing desperation of school districts to stay afloat financially, the corporations are succeeding.  More and more, state educational systems are making deals with the devil, and sacrificing our children as a consequence.

So could the appearance of product placement in Common Core-based exams be an accident, a result of random choice of textual material that mentioned modern corporate products that are appealing to children?  Could it have nothing to do with the ongoing attack on the public school system by special interest groups and money-hungry boards of directors?  Could Tom Dunn have been telling the truth when he said that Pearson got nothing for their blatant product placement?

Sure.  It could be.

But I don't believe it for a moment.