Testing Based on Small Samples

Certainly this is a point of contention. But (in reference to the previous post) we’re not talking about driving cars here. We’re talking about student success.

I don’t think anyone would argue with the fact that a handshake doesn’t tell all about a person. Neither does a meal they can cook. Or how fast they can change a tire. Or how high they can climb a tree. One test does not provide sufficient data for a student.

Regardless of how states test (even these are taken over several days and in different content areas–but I’ll save that argument for another day), our role as administrators and educators and parents is to appeal to multiple intelligences, extracting all the abilities of every student. It is a lazy person indeed who would be content to give one test or assignment and expect that to be enough data to “grade” the student.

It is never ok to ‘evaluate learners’ based on a limited sample of work. It is always appropriate to give those learners a wide array of experiences and opportunities to display their abilities in a testable way. After all, it is the test which should, to an extent, conform to the student in such a way as to provide the learner a chance to show what they know, even if it is done in a non-traditional way.

Evaluation vs. Assessment

There is much discussion on whether or not educators appropriately evaluate the learners entrusted to them. Usually we like to think of “evaluation” as “grading.” While grading is often a result of evaluation, it is not an all-encompassing reflection of the learner. It is a simple snapshot. It is this truth that administrators, parents, educators, and learners themselves get all wrong.

I’ve talked with concerned parents and teachers, demanding to know why their child or student received a grade based on a small sample of work as opposed to incorporating everything that child has ever done. Let’s imagine a scenario:

At about 16 years old, the average teen is able to get a permit which they can use to learn and practice safe driving skills. As a driving licensor, you see this particular teen driving in your neighborhood, around your church, in the grocer’s parking lot. Why? Let’s say it’s because you live near to him, so you constantly see him practicing. And he’s safe! He seems to do a great job controlling his vehicle.

Finally–it’s time for him to go for his test. You’re the licensing agent who is going to sit in the passenger’s seat, allowing this particular young man to show you how he can drive. Now, you have already informally evaluated his driving habits, and feel confident that he’ll have no trouble passing this test.

(You can probably guess where this is going.)

You direct him to put the vehicle in reverse to get out of the parking space, and head on toward the designated route. As he reverses, he loses control and, as a result of too much acceleration or a failure to find the break, or a hallucination–whatever–he slams into a USPS mailbox on a sidewalk.

Now–although you’ve seen this teen drive safely and exercise control over the vehicle in the past, and evaluated his performance based on observation, his “grade” for the test is one which will not allow him to pass.

This was a snapshot. A benchmark, of sorts, which the teen must pass in order to be proficient enough to be entrusted with a driving license. You cannot, in good faith, pass him based on your evaluations throughout the past several months. His “grade” has ultimately been determined by his performance on this test. You wouldn’t say “well, he’s been proficient in the past so we’ll just pass him anyway.” That would do the teen a disservice, not to mention the public!

Although evaluations are important for educators and allow us to give feedback on progress, there are standards (based on a small sample set) which must be met in order to show proficiency. Does it mean that these “grades” are equivalent to proficiency? Certainly not. I’m sure we can all think of people and experiences on the road when you wish that someone had not been allowed to drive based on a faulty reaction or inappropriate behavior.

So we see that even “grades” may very well fall short of true proficiency, but they do give us a baseline. If a learner doesn’t meet the minimum goal even one time, they have a chance to remediate themselves (often with the help of educators) and try again.

This goes with standardized state tests, district benchmarks, report cards, and assignments. It is educationally sound for educators to evaluate learners based on a very limited sample of work as long as the proficiency level is standardized for all the learners enduring the given assessment.