Skeptophilia (skep-to-fil-i-a) (n.) - the love of logical thought, skepticism, and thinking critically. Being an exploration of the applications of skeptical thinking to the world at large, with periodic excursions into linguistics, music, politics, cryptozoology, and why people keep seeing the face of Jesus on grilled cheese sandwiches.

Friday, May 17, 2013

The new, eviscerated AP Biology exam

You would think that, after spending 26 years as an educator, I would have figured out that whenever an educational oversight group says, "We are restructuring and reconfiguring this, for sound pedagogical reasons, in order to improve it," this really means, "We are going to scramble this for no good reason whatsoever, and the result will be something far worse than what you started with."

It happened with the New York State Regents Examinations, which (for those of you who do not live in New York) are the high-school-level course exit exams.  The rallying cry was "Raising the Bar," which makes it a little hard to explain why the biological sciences Regents examination is now so easy that it can be passed by anyone who has three working brain cells.  This exam was passed by a student who, on a quiz on human anatomy, incorrectly labeled the "anus" as being on the left arm.

Oh, but there was one major outcome of the exam restructuring: they changed the name of the course from "Regents Biology" to "Regents Living Environment," which raises the bar by virtue of having more letters.

So, when the College Board decided to restructure the curriculum and examination for AP Biology -- a course I've taught for twenty years -- I should have expected the same to happen.  Here's their rationale, as per the 2011 announcement of the planned changes on the College Board website:
“The revisions were enacted to address a challenging situation in science education at a critical juncture for American competitiveness,” said Gaston Caperton, president of the College Board. “The body of scientific knowledge is constantly expanding. The revisions will help science educators ensure that their instruction is fresh and current and that students develop not just a solid knowledge of the facts but also the ability to practice science and think critically about scientific issues.”

The revised AP Biology course and exam align with the knowledge and skills that many rigorous college-level introductory biology courses now seek to nurture, emphasizing the development of scientific inquiry and reasoning skills. Lab work is a critical component of the course, requiring students to master such skills as posing questions; collecting, analyzing and evaluating data; connecting fundamental concepts; and then defending their conclusions based on experiments.

“The revised course objectives will enable teachers and students to explore key topics in depth and will help students learn to reason with the rigor and objectivity of scientists,” said Trevor Packer, vice president of the Advanced Placement Program at the College Board.
So, I dutifully resubmitted my curriculum, and had it approved by the powers-that-be at the AP Central.  "How different can the exam be?" I thought.  "Biology is what biology is; the concepts are the same.  I already teach a rigorous course, that helps students to draw connections between disparate fields of science, that has an emphasis on application, reasoning, and synthesis, and that uses a strong lab component.  I'm told every year by students returning to visit from college that my class was a good preparation for college-level science.  It will be fine."

Optimism is a losing proposition, sometimes.

My students sat for the exam on Monday.  As you are undoubtedly predicting by now, I was not only wrong, I was so far wrong that it has made me question whether I should even offer this course next year.

The New and Improved AP Biology exam -- of which I only ever get to see the free-response section, the multiple choice section is hardly ever released to teachers for analysis -- was, in my opinion, a vague, confusing muddle that left students wondering, "is that all they're looking for?"  Here are four examples, which I will describe rather than quote in their entirety, for the sake of brevity:
1) A question in which fruit flies are placed in a "choice chamber" and given the choice of flying toward a dry cotton ball or one soaked in glucose solution.  Students are asked to "predict the distribution of the flies after ten minutes, and justify your prediction."  (As a student said to me after the exam, "A central principle of the animal world is that 'some food is better than no food.'")

2)  A question showing a "simplified carbon cycle" that looked, more or less, like this (but without the words in blue):


Students had to correctly label the arrows with "photosynthesis" and "respiration," and state that an example of an organism that does both processes is "a plant."

For reference, the unsimplified carbon cycle I use in my class is shown below:



3)  A question regarding the evolution of the earliest amphibians (363 million years ago) from lobe-finned fish (observed in rocks that are 380 million years old), that asked students to predict when, in geological history, you would might find fossils of the transitional species between the two groups.
4)  A question describing an experiment in which rats are given alcohol, and it is found that their urine output increases over rats that are given water.  Students were asked to "pose one scientific question that the researchers were most likely investigating with this experiment" and then "describe the effect of ethyl alcohol on urine production."
And so on.

Understandably, my students were pretty frustrated by gearing up for an examination that turned out to have been eviscerated of virtually all of its technical content.  And when you have a bunch of students who are pissed off because an exam isn't hard enough, you know there's something wrong.

Here are some direct quotes from some of my students:
"You needed to have barely any actual knowledge of biology in order to take this exam.  A few of the terms from Regents Biology would have been enough to get by on."

"I had worked hard and prepared for this exam.  I'd read the new curriculum and course outline, and I worked hard in class.  I felt like I had this material down.  This test was an insult to all of the hard work I put in."

"The reading passages and experimental design descriptions were too long to justify the extremely simple questions we were asked about them."

"The rat pee question wasn't even at the Regents level, it was below Regents level.  We already knew alcohol is a diuretic -- we discussed it in class when we were learning about the kidney.  If I had proposed to my Regents Biology teacher to do this as a final project, she would have said, 'You can do better than that.'"

"My brother is in college, and is taking biology.  I've looked at his textbook and lab manual.  And if I'd taken a course that prepared me to be successful on this exam, that course wouldn't have prepared me to be successful in the college biology course he's taking."

"I felt like even though they were shooting for a more conceptual approach, I wasn't being asked to apply concepts at a very high level.  The carbon cycle question, in particular, was not at a college level.  We knew that amount of detail in seventh grade."
Allow me to interject at this point that this group of 29 young people ranks amongst the top three AP biology classes I've ever taught in terms of drive, curiosity, and depth of understanding.  We're not talking about a bunch of slackers, here.  They had good mastery of the material, and are ready to make the jump to college science classes.  The fact that they ended the school year this way is a crashing letdown, and will remain that regardless of what their scores turn out to be.  (And interesting, too, that one of my best and brightest, who aced damn near every quiz and test I gave her this year, when I asked her what score she thought she got, replied, "I could have gotten a one.  I could have gotten a five.  I could have gotten a three.  I really, honestly have no way to tell how successful I was on this exam.")

So, there you have it.  The College Board has fulfilled its educational goal of taking a test that was, on the whole, rigorous but fair, and turning it into a hash.  Again, I shouldn't be surprised; that's been the result of virtually every educational shift I've seen in the last twenty years.  Oh, and one other thing I'm expecting: not only do educational oversight agencies take their Great Leaps Forward by mucking things up royally, they never admit afterwards that they screwed up.  So expect to see press releases soon from the College Board about how wonderful their New and Improved exam was, and how teachers and students everywhere are singing its praises to the skies.  Look, too, for them to begin to "improve" the exams in all of the other AP courses.

I hope I'm retired by then.

14 comments:

  1. this premise of constantly "upgrading" tests really strikes me as that sort of mindless progress for the sake of progress. In this case, backwards progress. The College Board has taken over more and more aspects of the state of testing in this country. What would it take to consider them a monopoly?

    ReplyDelete
  2. ...seriously...
    I am glad I had the chance to take AP Bio last year...In fact AP Bio is one of the few classes I have been able to practically apply information from. Do not teach to their test. Keep on teaching the course I had the privilege to take.

    ReplyDelete
  3. Have to wonder how much of this is deliberate.

    I must note that I consider
    "We know we're making kids dumber, but we don't care"
    or
    "We know we're making kids dumber, but it's an occupational hazard of pleasing the powers that be"
    or even
    "We know we're making kids dumber, but critical thinkers are a threat to the powers that be"

    ...as deliberate.

    Dropping a line in the woo-woo pond? Maybe. However the "powers that be" are smart enough to understand the ramifications of all of this, and proceed the way they do anyhow.

    ReplyDelete
  4. To those who these tests and corresponding curriculum's matter most to- that is to say, those who do not make money off of its distribution- they continue to represent a dim moment every year for those students and teachers who have to slug through them and for obvious reasons other than it being the "big, difficult final exam". The name change to "living environment" is the most obvious and simple example of this disconnection.
    As someone who has recently started working in a lab, it is almost hilarious to the extent of how useless the AP exams that I took were. I remember nothing from them and now I see for all their structured principles they fail to address what Colleges and businesses are looking for.

    I'll tell you what I remember from a class that impresses absolutely everyone for being a young person in a lab- College Chemistry. Labeling my chemicals and eliminating fractions, understanding how to mix them without cross contamination and an understanding of the most important and basic aspects of chemicals, such as atomic structures and reactions, all are basic foundations that were taught in this non-AP course. A lot of the coursework in AP Physics helped to cement that, but the AP test left me muddled and confused on where I stood. Especially in College I felt like I had ended up taking these AP exams, particularly the AP Physics exam, only to have them taught piece by piece to me in smaller bites so you could actually learn the material.

    I can't reflect on the current change in AP Biology, I never had the pleasure of taking your course. However, as for this change representing a positive move towards this "critical juncture for American competitiveness", it sounds like malarkey from a company that is under political pressure to improve its methods and help towards the goal of moving more students into college.

    ReplyDelete
  5. "The revisions were enacted to address a challenging situation in science education at a critical juncture for American competitiveness" So it would appear that the revisions enacted were due to the complete and total lack of American competitiveness. “The body of scientific knowledge is constantly expanding. The revisions will help science educators ensure that their instruction is fresh and current and that students develop not just a solid knowledge of the facts but also the ability to practice science and think critically about scientific issues.” Hmm well isn’t it just fascinating how “a solid knowledge of the facts but also the ability to practice science and think critically about scientific issues” apparently now means that by having a “solid” knowledge is to know simply the basics. This absolutely makes me feel pride in my country. However I must say that the rallying cry of “raising the bar” was not inaccurate, but rather it was incredibly misleading. By administering this test that supposedly raises the bar, the only bar the College Board is raising is the number of people, the people who incorrectly label the left arm as the anus, now, thanks to that test, have the chance to become a college student. Way to go Divided States of Embarrassment, you have officially cut down your own education system by trying to make it better. Good Job.

    ReplyDelete
  6. I took off teaching it last year for a few reasons and was very nervous about trying to alter my curriculum to meet the new test. Looks like I can take a step back and relax. I heard last year that part of the reason for the change is that students in lower socio-economic inner city districts (I believe LAUSD wasmentioned) weren't able to pass the test so basically they were revamping all of the AP tests to not really raise the bar but do the complete opposite (though its not being sold that way). I didn't want to believe it but if those are the questions asked as FRQs that's what has happened. I'll hope that it was just a transition year test and this year's will be more rigorous but if not then its just not fair to my past students that had to study their butts off to get their 3s, 4s and 5s when the future students just need to take a regular bio class to pass the test. Its also unfair to the colleges that these students will be give possible college credit for taking a test that is inferior to college exams. Regardless, I'll keep my rigor and hope for the best.

    Another thought I had is if the change was also done to require all the AP science teachers to go out and buy new kits and materials. Though there are a couple that are the same or can easily be altered there is the rest that are all new requiring hundreds of dollars to purchase kits or supplies from the chemical/biological supply copanies....
    I hope this isn't the case either........

    ReplyDelete
    Replies
    1. Thank you for the thoughtful response, which I agree with completely. Unfortunately, the people who are in charge of overseeing the exit exams (both state and national) have sold out to corporate concerns and the pressure from educational reformers who seem to care for nothing but quantifying anything that stands still long enough. Rigor, at least so far as the exams go, has gone out the window. Kudos to you for keeping it in your classroom.

      cheers,

      Gordon

      Delete
  7. I'm glad there's a teacher who agrees with me (a former AP Biology student). The test was a complete joke and an insult to all the hard work I put into that class. Granted, I was glad to hear we weren't going to cram 56 chapters of our textbook into a year, but then again, I was disappointed to see that stuff I WANTED to learn was cut out to make the test easier. It seems like the College Board had two options: 1) Leave the test the way it was and keep it hard or 2) Dilute the curriculum and cut out a chunk of biology to make the test easy. In the end, I got a 3 (I was cheated out of a 4/5). Even the laziest students noted how easy the test was, and that says a lot. And it seems you were right about the "new and improved exam" receiving praise, despite the fact that the percentage of 5s fell from 19.7 (2012) to 5.5 (2013). Way to go College Board!

    ReplyDelete
    Replies
    1. Thanks for your response, and I'm glad to hear I wasn't off base as seen from a student's perspective. What bothers me about all of these changes, both at the national and at the state levels, is how little the powers-that-be listen to the stakeholders -- the teachers and (especially!) the students. It's sad.

      Delete
  8. A spirited debate has arisen on our yahoo listserve related to this blog post, so I'll pass along one of the questions I have for you. BTW, we are a nationwide group of homeschoolers who are sharing experiences in preparing our kids for college, hence the name of the group, "hs2coll." Within the longstanding debate about whether ANY AP test fairly measures knowledge, there is recent concern about the revised AP Bio exam, as you reported here. My question for you is, now that scores have likely been reported for the first test session of the revised AP Bio exam, and given how "easy" it seems to have been, what do you know about the scores? Two specific questions: (1) did your high-achiever students score high (as expected) or low? We have speculated the revisions may have disadvantaged students with solid knowledge of the material tested, so do the scores bear that out? And (2) did your lower-achieving students, or more of those students, score high (higher than expected), suggesting the revisions made the test easier? Any other insights on scoring you can offer?

    ReplyDelete
    Replies
    1. Hi... Thanks for the comment! I'm glad that the post is provoking discussion, which it precisely what I intended. In answer to your questions -- my high-achieving students, by and large, scored lower than expected. A brilliant young lady who damn near aced the entire course, and whose high grades were based not only upon hard work but upon a natural gift for mastering technical material, got a 4. Several other students who should have been 4s or 5s got at least one point lower than expected. (In fact, I only had one score of 5 last year, a surprise since I've usually seen about 20% score 5 and this was an unusually motivated, bright class.)

      Likewise -- the low-scorers moved up. Kids who should have been 1s or 2s mostly scored in the 3 range.

      What this shows me is that the revisions on the test don't make the exam a fairer measure of understanding; they push all of the scores toward the center. In one of the first educational statistics classes I ever took, I learned that highly homogeneous score distributions are a hallmark of a poorly-constructed exam.

      So there you are...

      Delete
    2. You're pretty much right about the whole "pushing scores to the center"; if you compare the percentage of people getting 3s from 2012 (14.3%) to 2013 (36.2%), there's quite a big increase.

      Totally unrelated: it's cool to see a teacher who's an outspoken skeptic! I think AP Biology is one of the reasons I'm an atheist today (equally unrelated lol).

      Delete
  9. Its first wish of every student to get good marks in all test but problems is that sat prep boca raton which books are Good for test preparation and helpful for getting good grades. And some perfect books give the Best results.

    ReplyDelete