Posted on

3+ Quick- Birthday, (grading) scale matters, exposing climate fraud, debunking handbook

These aren't brand new items, as they're things I came across awhile ago and am just getting around to posting now. In addition, I realized that the anniversary of this blog just passed. My first post was published January 12, 2008. As I look back at my first posts, it's clear that I've come a long way (hopefully for the better)- in my location, in my career, and in my thinking. So, in celebration of the 4th anniversary of this blog, let me present you with the following interesting tidbits:

Scale matters (Rick Wormelli)

Thanks to the ActiveGrade blog for bringing this to my attention. I don't know how many times I've had discussions with other teachers on the topic of what constitutes fair and effective grading. Often the most heated topic (where I never made any headway) involved the giving out of zeroes for either missing or poorly done classwork. Rick Wormelli gives a great explanation of why grading scales matter- and specifically why zeroes are no good. It's long for YouTube at 8+ minutes, but it's worth it:

Exposing a climate science fraud (Ethan Siegel)

The post is ostensibly a take down of Judith Curry's claim's that recent studies and reports on the topic of climate change are "hiding the decline1." However, the real appeal of this post (for me) is how it so effectively describes how science and scientists work. He goes through the data, the uncertainties in measurement, and explains how exactly it is that scientists determine that some effect is real and not just a statistical fluke.

The Debunking Handbook (Skeptical Science)

Somewhat related, the Skeptical Science blog (one of the best places to find science-based information about climate science) released The Debunking Handbook a while ago and just recently updated it. The Handbook provides guidelines for communicating about misinformation and gives tips to avoid falling into common pitfalls. In their own words, "The Handbook explores the surprising fact that debunking myths can sometimes reinforce the myth in peoples' minds. Communicators need to be aware of the various backfire effects and how to avoid them..." The handbook is a free PDF download available at their website.

  1. "Hiding the decline" is the (totally false) idea that climate scientists are tweaking their graphs to make it seem like the Earth is getting warmer, when it really has been cooling the last decade (which it hasn't). Read the full article for more details. [↩]
Posted on

Exams: SBG-style

The goal of any exam, ideally, is to assess how much students have learned over the course of a semester or school year. I changed the focus of grading in my classes from counting points to counting progress towards specific learning goals, I knew my exams needed to reflect that change as well.

This summer I had initially thought I might design some sort of alternate, performance-based exam that would mesh well with the tenets of standards-based grading. However, this year all exams for the same class were required to be exactly the same regardless of teacher. Since I'm currently one of four teachers who teach the 9th grade Integrated Science course and the only one using standards-based grading, I knew I had to take our common exam and make the best of it.

So, the exams had to have the same questions, but they didn't need to be in the exact same order, right? I reordered all the questions on the exam based on the learning goal they assessed.

Multiple choice section, SBG exam

This process uncovered several questions which didn't address any of the learning goals, so these "others" were grouped together to make their own section.

Overall, I wasn't thrilled with the exam, but I think it was quite good given the requirements it had to meet.

Assessment

Breaking down the exam into its composite learning goals allowed me to assess each learning goal on the exam individually. It took decently longer to grade the exams in this way, but it also provided me and my students with a wealth of information about their learning throughout the first semester.

I created a Google Spreadsheet that automatically calculated the individual scores for each learning goal and the overall exam grade. Once the grading was done, I shared each student's spreadsheet with them through Google Docs.

Below is an example of a filled out scoresheet (and here's a blank calculation sheet if you're interested):

Example Exam Calculation Spreadsheet

Details

Overall grades. You may notice I calculated two "overall" grades. I told students their overall grade on the exam would be the average of their scores on each learning goal (giving each learning goal equal weight), but I wasn't sure if that might result in some odd effects on the overall grade due to some flaw I hadn't planned for. As a check, I also calculated the exam's score "traditionally," or simply by dividing the total points possible by the total points earned. Interestingly these two scores were almost always ridiculously close to each other (for most students it was <1%). I'm not sure exactly what that means, but it was interesting nonetheless.

Unfinished long answer questions. The exam had 6 long answer questions and students were required to complete at least 4 of them. I had a few students who either skipped the long answer questions entirely or did fewer than were required. It didn't make sense to penalize any one learning goal for not doing all the long answer questions (since, after all, simply not doing the long answer questions didn't necessarily mean they didn't understand the content of the learning goals). However, I felt that there should be some penalty for doing fewer than required1.  As a result, I calculated what percentage one long answer question was of the entire exam and divided that by 2- which gave me 1.84% in this case. For each required long answer question that was not completed, I took 1.84% off their overall exam grade.

Spreadsheet-fu. I honed some serious "if-then" formula skills in the process- an area of serious spreadsheet-fu weakness before this spreadsheet. Despite the time it took me to figure out how to make the spreadsheet do what I want, I'm still pretty sure using the spreadsheet instead of calculating everything by hand saved me several hours. Plus, now I have another formula type under my belt.

Final thoughts

Perhaps unsurprisingly, my predictions about what learning goals would be problematic for students on the exam were dead-on. They were the same learning goals that more students struggled with during the course of the semester. There really weren't any surprises on the mid-term.

What then, is the purpose of an exam in a SBG classroom? Exams are meant to assess how well students know the material that has been presented throughout the semester. However, if I am regularly assessing students' understanding of learning goals throughout the semester is there any benefit to a final, summative exam? Most students' exam grades were eerily close to their grades for the rest of the semester2.

If we're doing SBG well, it seems to me the final exam is unnecessary. We should already have a good understanding of exactly what students know, so why bother with a big test at the end of the semester?

Should the exam in an SBG classroom be something totally different than what we've traditionally come to think of exams as being? Or should they just be done away with?

_____

  1. At first I really balked at penalizing students for not completing the required long answer questions. However, after thinking about it for a bit, I came to the conclusion that to some degree the decision of a student to skip one or more of the long answer questions  was indicative of a lack of understanding of the content at least to some degree.     [↩]
  2. On average, the exam grades were just a bit lower than grades for the rest of the semester. I can rationalize that in several ways: additional anxiety due to it being an exam, or a less than perfect exam design, etc.     [↩]
Posted on

SBG: One Quarter Down

This Friday marks the end of the 1st Quarter of the school year. At this point I'm totally a SBG n00b. For the standard, "I can successfully implement standard-based grading into the 9th grade Integrated Science classroom," I'd rate myself at the "basic" level. I've got the basic idea, I've got the basic setup, it's going basically well, but it's a long way from where I hope it will be by the end of the year.

Reflections

Students don't get it

Students understand that their overall performance in class is based on their scores for the learning goals we've gone over in class. They understand that only their most recent score for each learning goal counts. Unfortunately they have at least 8 solid years of being conditioned point-grubbers. The whole concept seems totally foreign to their entire school experience. It saddens me that explaining to a student their grade is based on their actual understanding of the content draws a blank "I don't get it" look. I keep telling myself that by frequently explaining the basic tenets of SBG and sticking to my guns students will eventually reach the point where understanding smacks them upside the head and they spend the rest of the year walking around school demanding that all their teachers do it this way. However, I'd be willing to bet a big part of the problem is the fact that...

I don't get it

Well, I get it, but I'm not sure I get how to implement it. I'm not sure I get how to communicate it. I'm not sure what I'm doing day to day supports the "radical" mandate of SBG1. There have been several changes to my school life this year that have left me time-strapped and feeling I just don't have time to go through my curriculum with a fine-tooth comb and tweak it to fit the SBG mandate. Part of the issue is my understanding of...

Qualitative SBG

Many of the SBG Titans out there teach quantitative subjects such as Math or Physics. I'm teaching a much more qualitative 9th grade Integrated Science. Conceptually, I understand how SBG works within a qualitative course. On the implementation side I'm not as comfortable. Great inquiry-based activities focused on the life cycle of stars are a little trickier for me to design than those around the work-energy theorem. I'm not trying to cop out of providing a curiosity-rich learning environment here; some topics are just harder for me to design great stuff around. Which leads to the complication of the...

State curriculum

The Connecticut State Curriculum Standards for 9th grade Integrated Science aren't that bad. Sure, they're often poorly worded and overly expansive,2 but there are a lot of interesting and relevant topics in there. I'm not one to worry about skipping a standard or six, but there are people (generally the people that fill out my evaluations) who think it's best that I not miss any.

Yesterday I had a crazy daydream about a place where there weren't oh-so-specific standards for each class and I could really let students' questions and curiosity drive what we cover when. I get why we have state standards and think it's generally a positive thing, but I dislike their specificity. We keep forgetting to leave room for curiosity and the pursuit of interesting questions. I need to find a balance between keeping up with the other Integrated Science teachers and making sure I'm putting student learning at the forefront, which is much more difficult because I'm...

Going it alone

I'm the only teacher at my school using SBG. I've pitched it to my Integrated Science colleagues and explained its wonders to my principal, but they didn't seem too interested3. I'd like to work with them to puzzle through how we'll deal with the state standards while doing SBG, or share the effort of designing great activities and projects that keep curiosity and discovery at their center. Even trickier: we were given a mandate that our mid-term and final exams must be exactly the same. That wouldn't be a big deal if we were all on the SBG Express. Since I'm riding solo the common exams probably won't live up to my expectations of what an SBG exam should look like. It certainly won't be focused solely around the learning goals I've developed, which is a major bummer.

Some Questions

2nd Quarter

In the traditional points-driven system, the points simply reset to zero at the beginning of each quarter. Students start fresh. In my understanding, that doesn't really jive with the SBG system. At this point, I'm planning on bringing over all the learning goals and scores from the 1st Quarter into the 2nd and not reset student scores until the end of the semester. How do you SBG wizards out there handle this? I'm not sure if holding over grades from quarter to quarter is technically "allowed," which might make that decision for me.

Assessment routines

While there's no one right way to implement SBG, I'm always looking to make my implementation higher-impact while remaining easy to understand. Here's how things have gone down so far:

  • I give frequent small quizzes over a learning goal or two that we've been talking about in class.
    • If there is an obvious deficiency in student understanding, we take some time in class focused on the weaknesses and do an in-class reassessment later. If the vast majority of students understand the topic it becomes the responsibility of individual students to reassess before or after school.
  • I do frequent projects or activities that cover a couple to several learning goals. Usually there are at least a couple content-based learning goals and a few skill-based learning goals.
  • I've been pretty formal about letting students know when I'm assessing a learning goal. I'm not sure if this is the best method- especially for learning goals in the vein of, "I can effectively communicate and collaborate with others to complete a task." I'd like that to simply be an "always on" learning goal that can be assessed anytime they work in a group. However,  I'm not quite sure how to communicate that assessment in the midst of group work, or whether it'll cause a problem to not assess every student on that learning goal for each group activity. For example, it's easy to pick out students who aren't doing well on that learning goal while it often isn't as attention grabbing when they're doing well. As a result I worry about assessing the negative instances more than the positive, thus artificially driving that score down.

How do you handle "on the fly" assessment?

______

  1. "Learning is King"       [↩]
  2. There are at least 5 standards I could envision being semester long courses by themselves. [↩]
  3. On a positive note, my SBG implementation came up in a meeting where the Asst. Superintendent of Curriculum & Instruction was present and she seemed interested in hearing more.     [↩]
Posted on

Citations & tracked classes: SBG questions

We're now 8 days into the new school year & standards-based grading has officially been introduced and implemented (though we don't yet have much in the way of assessments in the book). I really like how the use of SBG has required me to rethink how I present a topic and how we spend our time in class1.

However, a couple issues have popped up where I could use a little guidance from some SBG-brethren (or sistren):

Problem 1: Citations & plagiarism

In the past, if students failed to cite their sources or plagiarized, I wouldn't accept their project/assignment/what-have you. I would give them an adequate amount of time to make the necessary changes and re-submit it without penalty, but if they didn't fix it up they wouldn't get credit.

As I was thinking through the SBG system, I realized that if I have a standard for properly citing sources and not plagiarizing information I could be opening a loop-hole. I did a twitter shout out on the issue, and the SBG-Jedi @mctownsley, responded to my question with a question:

Is citing sources an important issue you want all of your students to demonstrate?

Well, yes. I believe it's a very important skill to cite your sources- both for academic integrity and to point any readers toward your sources so they can read them and see if they agree with your interpretation of them. However, imagine a student really hates citations (let's face it, they are a pain) and decides to the play the system. They realize that as long as they use citations properly for the last assessment that requires them, they really don't need to do citations for any other previous assessments. This doesn't seem ideal.

My solution as of now: I have a standard for citations. In addition, if a student turns in a project or activity that is missing citations when it should have them or is plagiarized, then I'll give it back, tell them to fix it up, and not change any grades on any standards (except for the citation standard). While this technically leaves a loop-hole intact, I believe it'll prevent too much monkeying around.

Problem 2: Tracked classes

I teach 9th grade Integrated Science all day, every day. However, there are three(!) levels of Integrated Science: Honors, regular, and Foundations. Let's ignore issues with tracking students since it's an issue beyond my control at the moment2.

Should all Integrated Science classes share the same standards? Should achieving mastery be defined the same for all classes? My school weights honors classes more heavily (to prevent students taking low-level classes from becoming valedictorian, presumably), which seems to suggest there's a belief that the class requires less effort3.

My solution as of now: (1)The standards for all levels of Integrated Science are the same, but may be adjusted as I see necessary. If one level is showing a lack of knowledge I feel is important, I'll feel free to add a standard in for just that level (and vice-versa for removal of standards). I'm trying to be flexible and provide the best learning opportunities for all students. (2) I'm really not sure about this one. Right now I'm going to expect students in all levels to demonstrate similar levels of knowledge or skill to achieve mastery. Since I'm flexible on how much time I spend on standards in different classes, I'm willing to spend extra time if needed to get all students to mastery level.

Whatchoo think?

I know there are many people out there who have already dealt with similar issues. I'd love to hear your own solutions to these problems as well as insights into my "solutions as of now."

_____

  1. I really like the way it allows me to focus in on areas of student weaknesses and differentiate instruction with super-laser-guided-satellite-gps precision.     [↩]
  2. For the record, I find it's 95% a bad thing- including some pretty serious (but never mentioned aloud) issues with minorities being over-represented in Foundations and under-represented in Honors. There's an unspoken message being given to our minority population...     [↩]
  3. Not an assertion I agree with, but thems the facts.     [↩]
Posted on

SBG Express: Details

The basic idea of standards-based grading is simple: Grade students on their understanding of specific learning goals. It's the details of that implementation that are devilish. In honor of the "publish, then filter" idea, writing this post is my way of working through (and hopefully solidifying) those details.

What standards?

I've started making a list of standards. I keep oscillating between thinking, "These standards are way too specific!" to "These standards are way too broad!" I'm taking that as a sign that they're about where I want them. This is a list in progress. As of this typing the standards cover the first several mini-units of 9th grade Integrated Science. I'm open to any insights, questions, or comments you have concerning the standards. If you missed the subtle hyperlink earlier, CLICK HERE TO VIEW STANDARDS!

Grading

When the rubber hits the road, I need a specific way to calculate a student's letter grade at any point in time. Figuring this part out is spending more mental energy than anything else. An incorrect implementation might make SBG no better than old-fashioned grades by cumulative points- and in face could be worse. I'd like to avoid that.

  1. Each standard is worth 10 points.
    • Points translate directly to % and grades, so 9.5 = 95% = A
  2. The overall grade is calculated by averaging student scores on all the standards that have been assessed.
    • Some SBG'ers don't like the averaging method since some poorly understood standards might be covered up by a few well understood standards. Conjunctive scoring would get around this (Jason Buell gives a nice overview of conjunctive scoring here), but I worry that conjunctive scoring is a bit too "out there" for administrators, teachers, or students to get behind, and furthermore I'm not sure PowerSchool (our student information system) can handle it. I've put conjunctive scoring on the "possible future enhancements" list.
  3. Students may re-assess on any standard on any day.
    • Limits:
      • 1 standard per day, per student (the Cornally Corollary)
      • Students must know what standard they want to re-assess
      • Students can get help from me or re-assess, but not both on the same day (the Nowak Limit)
  4. Mid-terms and finals are summative
    • Meaning these grades can't change with reassessment. Total value of both combined is 20% of the overall course.
  5. I'll be using the SBGradebook along with PowerSchool to record & report student progress.
    • I'm not going to lie, I'm a little worried about how much time it'll take to enter grades in twice. However, the SBGradebook looks like such an exercise in graphy-awesomeness I couldn't not use it. Plus, it should help students track their own progress more effectively.

I'm pretty sure if you've written about SBG in the past 12 months you'll see something of your system here. Hopefully you view it as flattery and not me biting your awesome ideas.

I'm pretty sure writing this post helped me more than it will help any reader. I needed to hash out several competing ideas I had floating around my head. As always, if you see something glaringly obvious that will sink this SBG ship, let me know.

Posted on

SBG Express: I've got a ticket to ride

I mentioned it in my last post, and I'm officially announcing it here. My ticket is punched and I'm on board the SBG Express1 for the 2010-2011 school year!

I've spent the last few weeks reading and rereading several teachers' explanations and reflections on standards-based grading (including, but not limited to Shawn Cornally, Jason Buell, Frank Noschese, Matt Townsley, and several others who will be mad at me for not giving them a shout out). The more I read, the more I knew that standards-based grading was something that in some sort of sideways, subconscious way I've been working towards implementing the last several years even though I didn't even know what "SBG" stood for until May of this year.

Here's my basic understanding of SBG to date:

  • Assessment and grades should accurately reflect student learning (not just student homework-turning-in abilities)
  • Instead of using cumulative-points-earned as the basis for student grades, use progress towards a set of "standards (or "learning goals", or "knowledge criteria," or "whatever you'd like to call them")."
    • These standards describe specific areas of knowledge or expertise that students should gain. For example, "I can explain the law of gravity and understand what factors affect the strength of gravitational force."
  • Grades in your gradebook should help students realize where their understanding is great and where it's lacking.
    • Knowing they flunked "Quiz: Chapter 7" isn't helpful. Knowing they got 6 out of 10 on "I can explain why stars transition from one stage to another as they progress through their life cycle" gives the student valuable information that allows them to focus their remediation.
  • A grade on a standard is not set in stone (until exam time). Students can re-assess on any standard at any point in the school year. Grades can go down if the student shows a lack of understanding later in the course.
    • This should allow a students' grade to more accurately reflect their actual learning rather than be punished for not learning something before a big test when they knew it by the end of the course. Likewise, the student who crams successfully for the big test then forgets it all should have a grade that better reflects actual understanding.

I know! Sweet, right?

Fortunately, I've been blessed with a personality that's totally fine jumping into a project without having worked out all the details ahead of time. Unfortunately, I'm going to have to explain this whole SBG thing to quite a few students, parents, teachers, et cetera, in just a few days.

Tomorrow I'll share what I've got so far in the "details" folder.

_____

  1. copyright, 2010, Shawn Cornally [↩]