Musings on Assessment The question of whether instructors should reward participation/conscientiousness is one that has provoked a lot of debate, especially between educational experts (often academics) and those charged with teaching (hard working practitioners!). When I have talked to NorQuest instructors about this issue, they have shared several reasons for using “participation marks”:
Instead, these educators suggest making a concerted effort to focus on competence by:
0 Comments
Assessment Musings In the past few weeks our team has been asked to review assessments (mostly quizzes and tests) with an eye to clarity and fairness. Instructors were worried about question phrasing, multiple choice construction, weightings, time limits among other things. It was wonderful work and we learned much from working with our colleagues across the faculties.
In our interactions with individual instructors, we also discovered that sometimes the assessment is superficial and has only a tenuous link to the course outcomes. Sometimes it is because the instructor has been put into an awkward circumstance; they were handed a course and they are just trying to make a go of it (And might not get that course ever again!). Previous instructors made the text their curriculum and adopted it as “the source of truth”. The result is many “legacy” assessments that merely ask students to recall information. But, “List the four different kinds of organizational structure that the author identifies in the text” is not a great question. Questions like that reward “rightness” rather than comprehension. I’ll give you an example. Years ago, I was visiting a Social Studies class where students had been asked to work with terms associated with the Industrial Revolution. They were given a list of terms on one side of a page and a flowchart on the other, asking them to place the term where it might seem best. I sidled up to one student and asked him how it was going. He said “fine”, as he copied out definitions from the back of the textbook onto the paper he was given. I decided to find out “just how fine it was” going by giving him a quick quiz. I pointed to his paper and asked him to explain what capitalism was. “It is an economic system, based upon supply and demand whereby an entrepreneur controls the means of production.” He shared. “Aha”, said I, “So what is an economic system and what is an entrepreneur?” “Well, it’s got something to do with money. Not sure who this entrepreneur guy is either, but he controls the means of production!” He shrugged. I won’t bother you with the rest of the conversation. Suffice it to say, that student had a very superficial understanding of many of the terms on his sheet. However, if his instructor gave the student a short answer test asking him to define capitalism and a few other terms, he might attain full marks, if he had managed to memorize these definitions. And the instructor might be happy with this; a short answer/recall type of test would be easy to mark, provide full set of class marks and reward those students who spent time in studying (memorizing). But did real learning actually happen? I’m not so sure. That’s why our assessments should align with course GLOs and SLOs (general and specific learning outcomes). These outcomes have been crafted with an eye to essential understandings and skill development that we need to focus on in our teaching and assessment. Readings and resources, video clips and teacher lectures are used to reinforce these outcomes and not to replace them. So, our challenge is in designing assessments that truly assess the knowledge, skills, and attitudes that our students need, and that can be hard work. Related article: A Table of Contents is not a Curriculum Several times I’ve mentioned the Think-Pair-Share (TPS) strategy as one way to allow students time to process their thoughts and consolidate learning. This week I’d like to dive a little deeper into this strategy. The TPS is a very well-known strategy and one that is used extensively across the college, but some newer instructors might not have heard of it and some of the more experienced instructors may have forgotten just how powerful it can be and use it sparingly or ineffectively. The Think-Pair-Share teaching strategy is a very simple collaborative learning technique used to encourage active engagement and critical thinking. It consists of three distinct phases: Think, Pair, and Share. 1. Think: In this phase, students are given a specific question, prompt, or problem to consider individually. They are encouraged to reflect on the topic and generate their own thoughts or ideas without discussion with others. This step allows students to process information independently and develop their own understanding of the topic. Example: In a psychology class discussing theories of motivation, the instructor might ask students to think silently for a few minutes about what factors they believe motivate people to achieve their goals. What is important in this step is that students are given enough think time (Remember last week’s article?). Hurrying this phase will result in truncated thinking where students will simply take quick and superficial stabs at the problem and fail to unpack assumptions or discover complexities. 2. Pair: After the thinking phase, students are then paired up with a classmate to discuss their thoughts and ideas. This phase provides an opportunity for students to articulate their own understanding of the topic and to hear perspectives from their peers. By engaging in dialogue with a partner, students can clarify their own thoughts, gain new insights, and practice communication skills. Example: Continuing with the psychology class example, students might pair up and discuss their thoughts on motivation theories with a partner, comparing and contrasting different perspectives they came up with during the thinking phase. In this phase the quieter, more introverted students are given a safe place to share their ideas. Sharing thoughts will a class of seventy can be daunting, but talking to your elbow partner is a little less intimidating. This step is important because every student is expected to participate, not just the five or six that continually give feedback to the instructor. And it is helpful to remind students that listening is just as important as sharing! 3. Share: Finally, students are given the opportunity to share their ideas with the entire class. This phase allows for the consolidation of learning by synthesizing individual ideas into a collective understanding. It also provides an opportunity for the instructor to clarify any misconceptions and facilitate further discussion or exploration of the topic. Example: In the psychology class, each pair of students might share a summary of their discussion with the rest of the class, highlighting key points and insights. The instructor could then lead a class-wide discussion, drawing connections between different perspectives and providing additional context or examples. Often, I will ask students to share an insight that they heard from their elbow partner. This practice can tighten up the discussion (Students must remember and summarize, not pontificate!). Applying the Think-Pair-Share strategy in an online context requires some adjustments but can still be effective:
The Think-Pair-Share strategy encourages active learning, collaboration, and critical thinking, making it a valuable tool for both traditional and online instruction. NB: This article was a collaborative effort between Jeff and Chat GPT! The Central Purpose Question or Comprehension Purpose Question strategy (CPQ) challenges students to respond to an important and often complex question, to connect this question to their own background knowledge and beliefs, and then to read a relevant text or watch a short video with the CPQ in mind. This strategy presses students to read, listen, or watch with purpose and helps them to find, select and remember important details. Instructors may choose to deal with the CPQ in a variety of ways. One approach is to share the question on the whiteboard or a PPT slide and spend some time as a class talking about it before tackling the task (possibly as a think/pair/share activity). Other approaches might involve incorporating the CPQ into an anticipation guide, a quick write, a discussion web, a role play, a visualization exercise, or a case study. Regardless of how it is advanced, the CPQ is very important for the teacher as well as the student. It lends focus to the lesson and helps to provide relevancy. Some teachers have formalised the CPQ by developing an organizer and leading their students through the three stages in an active learning process (anticipation, consolidation, and extension). In this approach:
N.B. If the CPQ is one that can evoke a yes/no response that asks for reasons for and against (e.g., “Should 16-year-olds be given the opportunity to vote in municipal, provincial and federal elections?”), teachers should consider using a Discussion Web. Providing students with a central purpose question prior to assigning a short reading or video presentation serves several important functions:
One of the questions I frequently get asked is, “So just what is the difference between Applied Research and SoTL?” Well, this question is not that easy! Never-the-less I’ll give it my best shot in this latest installment of the SoTL Corner. And I had my friend ChatGPT to give me a little help with this. Applied Research I asked ChatGPT to tell me what conducting Applied Research is like at the college level: Engaging in applied research at a college level typically involves conducting research that aims to solve practical problems or address real-world issues within a specific field of study. This type of research often involves the application of theoretical knowledge to practical situations, with the goal of producing actionable solutions or innovations. Applied research projects at the college level may involve collaborating with external partners such as industry organizations, government agencies, or community groups to address relevant challenges or opportunities. The focus is on generating new knowledge or insights that can be directly applied to improve practices, policies, or technologies in a particular domain. At NorQuest College, we encourage our instructors and students to engage in a variety of Applied Research projects. Some examples of this are:
An important characteristic of Applied Research work is that it focuses on real world problems that often take place outside the confines of the college. It implies a partnership with an outside agency (like Capital Health, the Mustard Seed, or the City of Edmonton) and it often seeks to provide practical insights and possible solutions or efficiencies. For instance, in Early Learning and Child Care, faculty and student researchers might collaborate with daycare centers to evaluate the effectiveness of a new play-based learning curriculum in promoting cognitive development and school readiness among preschoolers. Or in the field of Nursing, nursing students might collaborate with a local hospital to study the effectiveness of a new patient care protocol in reducing hospital-acquired infections. Applied Research is attractive to college instructors because:
The Scholarship of Teaching and Learning
Here’s what ChatGPT told me about conducting SoTL research at the college level: Completing a Scholarship of Teaching and Learning (SoTL) research project at the college level involves conducting research that specifically focuses on enhancing teaching and learning practices within higher education settings. SoTL research seeks to investigate and improve the effectiveness of instructional strategies, curriculum design, assessment methods, and other aspects of teaching and learning. This type of research often involves studying the impact of innovative teaching approaches, interventions, or educational technologies on student learning outcomes, engagement, and retention. SoTL projects typically aim to contribute to the scholarship of teaching and learning by generating evidence-based insights and recommendations that can inform teaching practice and educational policy at the college level and beyond. Again, NorQuest faculty have also been very much engaged in this kind of research. Examples of this might include Sheryl Boisvert’s inquiry into “flipping the classroom”, Karen Taylor’s work on using Indigenous story telling as an important pedagogical practice, the research that Viola Manokore and I did on the transition to Emergency Remote Teaching Practice, and Dustin Grue’s ongoing inquiry into the use of a generative AI tool as a writing tutor. In each of these cases the focus is on understanding and hopefully improving the student learning experience. At the moment, we are conducting three rather ambitious college wide SoTL projects for Reimagine Higher Education… I’ll write about these in later installment of SoTL Corner. In contrast to Applied Research, the focus of SoTL is on what happens within the college, often right in the classroom. So, while an Applied Research project might study the impact of a certain care model for dementia in several local long-term care centers, a SoTL project would examine the best approach for teaching Practical Nursing and Health Care Aide students about dementia. Would the optimal approach involve readings, simulations, role plays, videos and recorded interviews, literature reviews, or guest speakers? And would the aim of the project be on cultivating empathy, learning boundaries and self-care, changing perceptions and orientations, or understanding the medical conditions and considerations? However, SoTL inquiries are not limited only to studying classroom interventions. When I asked Chat GPT to provide a few examples of this, it said that SoTL researchers might also look at:
SoTL doesn’t always seek solutions, according to Hutchings (2000) it can describe a current state of teaching and learning, it can compare teaching approaches, conditions, or interventions, it can show relationships between various factors in teaching (e.g., workload and student performance) or it can seek to explain (e.g., why are students dropping out?) or advance a model of what could be. SoTL is attractive to college instructors because:
So, could an Applied Research Project also be a SoTL Inquiry? Well, yes, sometimes, if… but that is a topic for another installment of SoTL Corner! Reference: Hutchings, Pat. (2000). Introduction: Approaching the scholarship of teaching and learning. In Pat Hutchings (Ed.), Opening lines: Approaches to the scholarship of teaching and learning (pp. 1-10). Menlo Park, CA: The Carnegie Foundation for the Advancement of Teaching. If you are anything like me, you can become very focused on making sure that, within allocated class time, you get through the central concepts in a timely fashion. Because of this pressure, you might hurry through classroom questions, waiting only for a quick response that confirms the answer so you can get on with the lesson. And, since you already know the answer, you are ready to jump in just as soon as you feel the students have had time enough. Trouble is, it usually isn’t time enough. What can seem like an eternity of waiting can sometimes be only 10-15 seconds. This is barely enough time for an online student to hear and understand the question and perhaps raise an online hand! (If you think I am overstating this, re-watch some of your recent recordings! I did… I am very impatient!) Giving think time when asking questions is important. Students need the opportunity to process questions thoroughly and formulate thoughtful responses. If we fail to give enough think time, we will end up with very little interaction–only the very quick, auditory processors will be volunteering answers. The rest of the class may disengage or simply let others do the thinking for them. A class of seventy will quickly devolve into a back and forth from one teacher with 5 or 6 keener beaner students. Ultimately, we want to encourage deeper engagement with the material and critical thinking. We would also like it if all students had an equal opportunity to participate and contribute meaningfully to the discussion! So how do we go about doing this? Well, one way is by setting time limits and conditions. When you present your question, let the students know that they are not to answer immediately, but that you are giving them a certain amount of time to think about their response. Depending on the question, this could range from a minute to four or five minutes. During that set time, students might jot down their thoughts, make an illustration, or formulate an argument. The hardest part in giving think time is keeping yourself from interrupting! That’s why you need a timer! When the time elapses, you might engage the students in a think/pair/share activity (in-person) so that every student gets a chance to share their thoughts with someone before you pull back to a general classroom discussion (More on TPS in another musing!). Or, if you are working in an online environment like Class Collaborate (BBCU) or Kaltura, you might ask the students to type their thoughts into the chat feed and then “flood the chat” when the time limit elapses. Having students press submit at the same time means that everyone can contribute and prevents students from piggy backing on responses from those quick processors. Flood the chat also provides a rich trove to use as you further discuss the topic. Often students provide perspectives and insights that we never thought of. However, you need to be careful with this strategy because the chat is not anonymous. If you are discussing sensitive topics and want to safeguard students, you might choose to get their ideas through an online polling or posting tool (Padlet, Jamboard) that allows for anonymity. Giving think time fosters a supportive and inclusive learning environment by reducing pressure and anxiety associated with rapid-fire questioning. We’d like our students to be confident and willing to participate! We also want them to engage in more than just passive listening, we want them to do some of the work! |
AuthorJeff Kuntz Ph.D. ImagesExcept where indicated, images used in the blog posts are personal photos, images from NorQuest College or images from Pixabay. Pixabay is a vibrant community of creatives, sharing copyright free images, videos and music. https://pixabay.com/ Archives
March 2024
Categories
All
|