Focus on Assessment – Again! This past Friday we had a Faculty Talking Circle on Online Assessment. It was a rich discussion and a very honest conversation! During the weekend, I did a bit more thinking about what we discussed. Here are a few thoughts. Some of you may know that our family recently added two new members to it! Edgar and Daisy (perky ears!), two precocious little border collies. Right now they are about 12 weeks old and are still learning many things - so are we. Just after we took the siblings home we arranged a visit with the veterinarian for a check-up. One of the things we mentioned was that Edgar had the habit of bolting his food while Daisy would take a mouthful, walk a bit of a distance away and carefully chew what she had before returning to get another mouthful. By that time, Edgar may have eaten all of his portion and most of hers. It didn’t seem fair or equal! We didn’t want Edgar to end up being a fat little dog, especially at Daisy’s expense, so we thought about limiting his food intake or feeding Daisy separately. The vet told us to take a deep breath and relax. Just keep adding food to the dish; puppies self-regulate and limiting Edgar’s food intake might be harmful. He is likely moving through a growth spurt and, judging from his body type, he will likely be a bigger dog. Trying to be equal wouldn’t be fair; especially to Edgar. Yet, as educators we consistently fall into the trap of “fair must be equal”. We don’t want to be seen as favoring particular students and we sometimes hesitate about providing extra resources, support, time or opportunity, because “it wouldn’t be fair to the rest of the class”. Why should a particular student get an extension on their project? They had better have some dire circumstances! How will they understand about deadlines and the “real world”? But fair isn’t equal. Fair is about giving every student the chance to thrive and succeed, and some students will need different kinds of support than others. Another example. Although I don’t often think of it as such, I have a learning accommodation. I wear eyeglasses. Without my beloved blue spectacles, I would be hard pressed to complete most tasks. If I had been taught in an environment where decisions were based solely on keeping all things equal, I should not have been permitted to wear them. Why should I be able to wear glasses while others did not? The fact that the others did not need glasses is immaterial. It’s not fair! It’s not equal! But, as educators, we think little of having students with glasses or hearing aids. They don’t get accommodations, they have aids that do little to interfere with our planning and practice. And, we do have many students who receive accommodations; and we do our best to support them. We have students who need extra time to process, students who need a scribe (graphomotor issues), students who need readers, students who need interpretation, and students who need different colour papers. We are trying to be fair by removing some of the barriers that might interfere with their learning. Their extra time, prepared notes, relaxed deadline, reader, or scribe is my glasses. Friday’s Discussion That brings me back to the Friday Talking Circle on Assessment. Originally I had intended the session to be a bit of an idea exchange. I invited a colleague from Lakeland College, Mabyn Grinde, to join us for the session, as she is working through many of the same challenges with the faculty there. Over twenty instructors were able to attend, as well as Joan Wall and Kerry Taillefer, who would help field questions! The plan was simple. Share some of the assessment principles seek input from instructors. Ask instructors to share some of their current challenges related to online assessment. Elicit a few suggestions and successes from each other that might improve our practices. Oh, and yes, maybe also talk about the new UDL 1.5 time requirement for conducting assessments. However, the 1.5 UDL (Universal Design for Learning) requirement quickly became the major focus of our discussion. The discussion became passionate and, at times, pointed. It was an honest conversation! We were concerned about what is fair, what is equal and whether students might get an unfair advantage. From my understanding the move to universal 1.5 timing for assessments was intended to allow students who may not have had learning challenges formally assessed, to be given the extra time. Since the onset of the Covid-19 pandemic, many of our students may have struggled in isolation, with barriers to achievement such as inadequate technology or Wi-Fi access, anxiety or uncertainly related to the pandemic, childcare and other familial or relational obligations, shared use of technology (one computer for a whole family to do their work on), employment responsibilities and other stressors. Students who do not have these stressors would likely not take advantage of the extra time, but those who need the extra accommodation would be able to use it. Many of the faculty in the talking circle were, for the most part, very sympathetic to the concept of time and a half. Our students are under a lot of pressure and this decision seemed to come from a caring place. Never the less, there were a lot of questions and concerns related to this move.
Compounding the issues raised by the faculty is the fact that many of the established, closed-answer assessments (especially those with right/wrong answers) no longer work in an online world where students can screen capture and instant message. Open-ended assessments require more time to ideate, build, design effective scoring guides for, and mark! Getting More Context As special guests, I invited William Hamilton (Assistive Technology) and Kerry Taillefer (Testing Centre & Learning Support) to help address some of the emergent questions that might bubble up in the talking circle. Recently, William, Kerry, Joan and I were charged with helping the faculty learn about the 1.5 UDL requirement and it has been a bit of a challenge. The decision has also affected our roles and department staffing! William unfortunately could not attend the Friday talking circle. He did send along a few messages though:
Kerry provided some excellent points to consider as we work our way through this change. She has been looking into research on this matter and found out that:
Kerry suggested that we should work with our students to prepare them for the exams, make them aware of test-taking strategies and help them avoid stressing before or over-thinking during assessments. She also suggested talking with students about on how cheating in the short term can have long-term repercussions when they are asked to apply information with an incomplete understanding. I was extremely glad to have Kerry there. A number of instructors in the circle affirmed her observations about students using or abusing extra time. At the same time, we could all agree that: “there are no magic wands” and one-size does not fit all. However, as instructional faculty and faculty support, we needed to “find ways forward”. Then we shifted gears to discuss some solutions. Finding Ways Forward
At this point Mabyn was able to share some of the ways that the faculty at Lakeland have responded to the need reality of online assessment. Some instructors at Lakeland had success with open-book and oral exams. It was a big shift away from the typical battery of 200 multiple-choice questions. The success of open book exams had much to do with the structure of them and the challenge found in them. Students were given a set block of time (a day?) to explore questions that spoke to the big questions of the course and their program. In oral exams, with some carefully crafted questions and strategic probing, instructors could find out very quickly who knew and could make connections, and who could not. Some of the instructors Mabyn worked with also had success in constructing case-study types of questions that required longer answer responses. In all three cases, making sure that the marking criteria (rubrics) are clear was essential. One instructor shared that, after conducting these new assessments, he had a much better picture of “who got it” than he did from the old multiple-choice exams. He could see how students were intersecting ideas and applying them. Some of the other suggestions shared during the session for addressing the challenge of online assessment (and even the UDL 1.5 requirement) included:
Last week I shared some handy websites from Taylor Institute (U of C). Here are a couple of others that people shared on Friday. Websites: https://learninginnovation.ca/student-cc/ (Lethbridge College Centre for Teaching and Learning) https://sites.tufts.edu/teaching/2020/05/08/integrating-inclusive-and-sustainable-assessments-in-your-online-teaching-from-beginning-to-end/ (Teaching and Learning at Tufts University) https://teaching.uwo.ca/elearning/student_assessment/alt-assessment-ideas.html (Centre for Teaching and Learning – Western Ontario) Have a wonderful week. I’m going out to play with my puppies! Jeff
0 Comments
Looking at Assessment!
This week I wanted to talk about assessment. It is the most significant factor in encouraging and sustaining student success. Formative assessment (Assessment For and As Learning) helps to provide students and instructors with an opportunity to check on understanding, clarify learning targets, re-calibrate, and build for success. As many leading educators have pointed out, this kind of assessment needs to happen minute by minute, throughout lessons and activities; there needs to be a constant going back and forth of checking and rechecking. Summative Assessment (Assessment Of Learning) usually comes at the end of a cycle, and it helps students and instructors to measure success against clearly defined parameters. Summative assessment requires educators to establish routines and procedures that promote fairness, reliability, and validity. However, while few educators would debate the impact and role of quality assessment practices, there has been hot debate about what these practices are. A number of years ago I was involved in a district-wide assessment reform that called on K-12 teachers to re-examine their assessment practices. Questions were asked like:
That year we tackled some very prickly pears. There were parents who were very upset that their child’s progress was no longer indicated by percentile (“How can I compare my child to the rest of the class? What does “proficient” really mean? Is it a 63% or 75%? There’s a big difference!”), and teachers who bristled at the idea of no zeros and second chances (“How can I ensure that they learn their lesson about tardiness?”). Not every issue had an easy answer. Nonetheless, as a district, we learned a great deal about assessment. It was a great leap forward for all of us. The present circumstances, the transition to whole scale online learning prompted by the Covid-19 pandemic, has brought on a similar kind of college-wide assessment reform for NorQuest. This time the questions are a little different though:
Since NorQuest moved to exclusively online instruction (with some obvious exceptions), I could feel the tension mounting around just these questions (and many more). Programs that relied on 2-4 big, multiple-choice assessments to provide evidence of understanding and achievement were suddenly faced with the challenges of conducting these exams online. Other programs that required physical demonstration of situational competency, practical skills, or professional judgement were also left struggling. One recent change to how we assess at NorQuest College has been with respect to exam accommodations and to the disability intake process:
These changes are more fully explained in a couple of emails that were sent out earlier this month. If you missed the emails, have a search through your inbox, check with your chair or associate chair, or contact William Hamilton. In the next few weeks William Hamilton, Alan Jeans, Kerry Taillefer, Joan Wall and I will be meeting with departmental groups to help clarify what this means for your students and for you. With all of these assessment pressures and changes in mind, Norma Schneider met with the Deans, Chairs, and Associates during the Week of Welcome to discuss what we, as NorQuesters, see as the most important components and principles in effective online assessment. Some of the frequently mentioned terms included: authenticity, alignment, relevance, choice, reliability, rehearsal, “aha” moments, variation, growth opportunity, transparency, and ownership. Norma urged us to look past assessments that rely solely on rote and recall, and think of ways to assess that will challenge students to analyze, apply and construct based upon scenarios or problems. It reminded me of what one of my Education professors used to say to us, when I was taking my undergraduate degree many moons ago (almost 40 years!). Dr. Glenn Martin (U of A) asked us if we planned to use assessment as an affirmation to find out what students know and can do, or as a “gotcha” to find out what they missed or failed to master. All too often, in an effort to “separate the wheat from the chaff”, we build assessments that test for perfection and not for proficiency. Such a practice makes it easier to rank and sort the students, based on where they fit on the curve (normative-based). These kinds of assessments also favor those students who are very good at memorizing or at taking multiple-choice tests. Dr. Martin advocated for a range of assessment types, that might include multiple choice but also include short and long answer tests, portfolios, projects, forums, problem-based learning, role-plays and many other ways of finding out what your students know and can do. Till next week! Jeff |
AuthorJeff Kuntz Ph.D. ImagesExcept where indicated, images used in the blog posts are personal photos, images from NorQuest College or images from Pixabay. Pixabay is a vibrant community of creatives, sharing copyright free images, videos and music. https://pixabay.com/ Archives
March 2024
Categories
All
|