What Should A Grade Measure?

A grade should quantify, with reasonable accuracy, how much of the course content a student has learned by the end of the course.

Weighted Average — A Typical Method

• Exam 1: 20%
• Exam 2: 20%
• Exam 3: 20%
• Final: 30%
• Homework: 10%

Case 1 — What’s Student A’s Grade?

Student A

• Exam 1: 20% → 50/100
• Exam 2: 20% → 40/100
• Exam 3: 20% → 80/100
• Final: 30% → 95/100
• Homework: 10% → 75/100
• Course Grade: → 70 (D/C-)

Case 1 — Is This A Good Measure?

• Exam 1 → 50/100 → slow start
• Exam 2 → 40/100 → “falling behind”
• Exam 3 → 80/100 → begins to understand
• Final → 95/100 → demonstrates mastery
• Homework → 75/100 → gives no information
• Course Grade: → 70 (D/C-) → inaccurate, harmful

Case 2 — What’s Student B’s Grade?

Student B

• Exam 1: 20% → 95/100
• Exam 2: 20% → 80/100
• Exam 3: 20% → 40/100
• Final: 30% → 50/100
• Homework: 10% → 75/100
• Course Grade: → 65.5 (D-/D/D+)

Case 2 — Is This A Good Measure?

• Exam 1 → 95/100 → great start
• Exam 2 → 80/100 → still reasonable
• Exam 3 → 40/100 → things are difficult
• Final → 50/100 → little knowledge retention
• Homework → 75/100 → meaningless tally
• Course Grade: → 65.5 (D-/D/D+) → inaccurate, generous

Problem #1

• Averages are good when we have data points that are independent
• Computing a class average on a given exam can be meaningful
• Time-series data is a different ball game
• Knowledge acquisition (or loss) depends necessarily on the past

Problem #2

• Averaging exam scores together for one student penalizes the student who is slow to start
• Averaging exam scores together for one student favors the student who knows the basics but does not grasp the remainder of the course

It’s Not Synchronized Swimming!

• Everyone learns at a different pace
• The expectation that students should learn the concepts taught for Exam 1 by Exam 1, is heroic
• This is true on the other end as well, a student may grasp the basics easily, but have a difficult time extrapolating to the more advanced topics

(Weighted) Average Is A Busted Metric

• Works well for “low variance” data
• Inaccurate measure for the student who does not learn at the “designated” pace

What Could Be Done

• Wish to measure “aggregate knowledge acquired”
• Need a metric that is closer to “max” rather than “average”
• Assessment should occur towards the end of semester
• Assessments also need to be comprehensive

A Possible Solution (1/2)

• In a typical 15 week semester
  • No exams for the first 12 weeks
  • Ungraded assignments (“formative”) only for first 12 weeks
  • Final three weeks is “exam month”

A Possible Solution (2/2)

• Have exams (“summative”) be comprehensive
• Allow multiple retakes (perhaps up to a limit for logistical reasons)
• Use latest score as a proxy for “aggregate knowledge acquired” (weight of 100%)

Benefits

• No material changes to curriculum or pedagogy
• Only real change is when we assess
• Students get 12-15 weeks to master material
• Can “intervene” without impacting grade / GPA
• Final grade actually is a reasonable approximation for “aggregate knowledge acquired”

Challenges

• Instructors have to make sure to follow student progress regularly / constantly
• Synchronized swimming effect still exists in that the expectation is that students should learn material over 15 weeks, however this is an improvement to the existing testing environment

What Are Your Thoughts?

• Do you already do this? If you do, how has it worked out?
• If you don’t already do this? Do you think you would?
• What are other benefits / challenges that you see?
• Leave your thoughts!