Musings on Assessment The question of whether instructors should reward participation/conscientiousness is one that has provoked a lot of debate, especially between educational experts (often academics) and those charged with teaching (hard working practitioners!). When I have talked to NorQuest instructors about this issue, they have shared several reasons for using “participation marks”:
Instead, these educators suggest making a concerted effort to focus on competence by:
0 Comments
Assessment Musings In the past few weeks our team has been asked to review assessments (mostly quizzes and tests) with an eye to clarity and fairness. Instructors were worried about question phrasing, multiple choice construction, weightings, time limits among other things. It was wonderful work and we learned much from working with our colleagues across the faculties.
In our interactions with individual instructors, we also discovered that sometimes the assessment is superficial and has only a tenuous link to the course outcomes. Sometimes it is because the instructor has been put into an awkward circumstance; they were handed a course and they are just trying to make a go of it (And might not get that course ever again!). Previous instructors made the text their curriculum and adopted it as “the source of truth”. The result is many “legacy” assessments that merely ask students to recall information. But, “List the four different kinds of organizational structure that the author identifies in the text” is not a great question. Questions like that reward “rightness” rather than comprehension. I’ll give you an example. Years ago, I was visiting a Social Studies class where students had been asked to work with terms associated with the Industrial Revolution. They were given a list of terms on one side of a page and a flowchart on the other, asking them to place the term where it might seem best. I sidled up to one student and asked him how it was going. He said “fine”, as he copied out definitions from the back of the textbook onto the paper he was given. I decided to find out “just how fine it was” going by giving him a quick quiz. I pointed to his paper and asked him to explain what capitalism was. “It is an economic system, based upon supply and demand whereby an entrepreneur controls the means of production.” He shared. “Aha”, said I, “So what is an economic system and what is an entrepreneur?” “Well, it’s got something to do with money. Not sure who this entrepreneur guy is either, but he controls the means of production!” He shrugged. I won’t bother you with the rest of the conversation. Suffice it to say, that student had a very superficial understanding of many of the terms on his sheet. However, if his instructor gave the student a short answer test asking him to define capitalism and a few other terms, he might attain full marks, if he had managed to memorize these definitions. And the instructor might be happy with this; a short answer/recall type of test would be easy to mark, provide full set of class marks and reward those students who spent time in studying (memorizing). But did real learning actually happen? I’m not so sure. That’s why our assessments should align with course GLOs and SLOs (general and specific learning outcomes). These outcomes have been crafted with an eye to essential understandings and skill development that we need to focus on in our teaching and assessment. Readings and resources, video clips and teacher lectures are used to reinforce these outcomes and not to replace them. So, our challenge is in designing assessments that truly assess the knowledge, skills, and attitudes that our students need, and that can be hard work. Related article: A Table of Contents is not a Curriculum Several times I’ve mentioned the Think-Pair-Share (TPS) strategy as one way to allow students time to process their thoughts and consolidate learning. This week I’d like to dive a little deeper into this strategy. The TPS is a very well-known strategy and one that is used extensively across the college, but some newer instructors might not have heard of it and some of the more experienced instructors may have forgotten just how powerful it can be and use it sparingly or ineffectively. The Think-Pair-Share teaching strategy is a very simple collaborative learning technique used to encourage active engagement and critical thinking. It consists of three distinct phases: Think, Pair, and Share. 1. Think: In this phase, students are given a specific question, prompt, or problem to consider individually. They are encouraged to reflect on the topic and generate their own thoughts or ideas without discussion with others. This step allows students to process information independently and develop their own understanding of the topic. Example: In a psychology class discussing theories of motivation, the instructor might ask students to think silently for a few minutes about what factors they believe motivate people to achieve their goals. What is important in this step is that students are given enough think time (Remember last week’s article?). Hurrying this phase will result in truncated thinking where students will simply take quick and superficial stabs at the problem and fail to unpack assumptions or discover complexities. 2. Pair: After the thinking phase, students are then paired up with a classmate to discuss their thoughts and ideas. This phase provides an opportunity for students to articulate their own understanding of the topic and to hear perspectives from their peers. By engaging in dialogue with a partner, students can clarify their own thoughts, gain new insights, and practice communication skills. Example: Continuing with the psychology class example, students might pair up and discuss their thoughts on motivation theories with a partner, comparing and contrasting different perspectives they came up with during the thinking phase. In this phase the quieter, more introverted students are given a safe place to share their ideas. Sharing thoughts will a class of seventy can be daunting, but talking to your elbow partner is a little less intimidating. This step is important because every student is expected to participate, not just the five or six that continually give feedback to the instructor. And it is helpful to remind students that listening is just as important as sharing! 3. Share: Finally, students are given the opportunity to share their ideas with the entire class. This phase allows for the consolidation of learning by synthesizing individual ideas into a collective understanding. It also provides an opportunity for the instructor to clarify any misconceptions and facilitate further discussion or exploration of the topic. Example: In the psychology class, each pair of students might share a summary of their discussion with the rest of the class, highlighting key points and insights. The instructor could then lead a class-wide discussion, drawing connections between different perspectives and providing additional context or examples. Often, I will ask students to share an insight that they heard from their elbow partner. This practice can tighten up the discussion (Students must remember and summarize, not pontificate!). Applying the Think-Pair-Share strategy in an online context requires some adjustments but can still be effective:
The Think-Pair-Share strategy encourages active learning, collaboration, and critical thinking, making it a valuable tool for both traditional and online instruction. NB: This article was a collaborative effort between Jeff and Chat GPT! The Central Purpose Question or Comprehension Purpose Question strategy (CPQ) challenges students to respond to an important and often complex question, to connect this question to their own background knowledge and beliefs, and then to read a relevant text or watch a short video with the CPQ in mind. This strategy presses students to read, listen, or watch with purpose and helps them to find, select and remember important details. Instructors may choose to deal with the CPQ in a variety of ways. One approach is to share the question on the whiteboard or a PPT slide and spend some time as a class talking about it before tackling the task (possibly as a think/pair/share activity). Other approaches might involve incorporating the CPQ into an anticipation guide, a quick write, a discussion web, a role play, a visualization exercise, or a case study. Regardless of how it is advanced, the CPQ is very important for the teacher as well as the student. It lends focus to the lesson and helps to provide relevancy. Some teachers have formalised the CPQ by developing an organizer and leading their students through the three stages in an active learning process (anticipation, consolidation, and extension). In this approach:
N.B. If the CPQ is one that can evoke a yes/no response that asks for reasons for and against (e.g., “Should 16-year-olds be given the opportunity to vote in municipal, provincial and federal elections?”), teachers should consider using a Discussion Web. Providing students with a central purpose question prior to assigning a short reading or video presentation serves several important functions:
One of the questions I frequently get asked is, “So just what is the difference between Applied Research and SoTL?” Well, this question is not that easy! Never-the-less I’ll give it my best shot in this latest installment of the SoTL Corner. And I had my friend ChatGPT to give me a little help with this. Applied Research I asked ChatGPT to tell me what conducting Applied Research is like at the college level: Engaging in applied research at a college level typically involves conducting research that aims to solve practical problems or address real-world issues within a specific field of study. This type of research often involves the application of theoretical knowledge to practical situations, with the goal of producing actionable solutions or innovations. Applied research projects at the college level may involve collaborating with external partners such as industry organizations, government agencies, or community groups to address relevant challenges or opportunities. The focus is on generating new knowledge or insights that can be directly applied to improve practices, policies, or technologies in a particular domain. At NorQuest College, we encourage our instructors and students to engage in a variety of Applied Research projects. Some examples of this are:
An important characteristic of Applied Research work is that it focuses on real world problems that often take place outside the confines of the college. It implies a partnership with an outside agency (like Capital Health, the Mustard Seed, or the City of Edmonton) and it often seeks to provide practical insights and possible solutions or efficiencies. For instance, in Early Learning and Child Care, faculty and student researchers might collaborate with daycare centers to evaluate the effectiveness of a new play-based learning curriculum in promoting cognitive development and school readiness among preschoolers. Or in the field of Nursing, nursing students might collaborate with a local hospital to study the effectiveness of a new patient care protocol in reducing hospital-acquired infections. Applied Research is attractive to college instructors because:
The Scholarship of Teaching and Learning
Here’s what ChatGPT told me about conducting SoTL research at the college level: Completing a Scholarship of Teaching and Learning (SoTL) research project at the college level involves conducting research that specifically focuses on enhancing teaching and learning practices within higher education settings. SoTL research seeks to investigate and improve the effectiveness of instructional strategies, curriculum design, assessment methods, and other aspects of teaching and learning. This type of research often involves studying the impact of innovative teaching approaches, interventions, or educational technologies on student learning outcomes, engagement, and retention. SoTL projects typically aim to contribute to the scholarship of teaching and learning by generating evidence-based insights and recommendations that can inform teaching practice and educational policy at the college level and beyond. Again, NorQuest faculty have also been very much engaged in this kind of research. Examples of this might include Sheryl Boisvert’s inquiry into “flipping the classroom”, Karen Taylor’s work on using Indigenous story telling as an important pedagogical practice, the research that Viola Manokore and I did on the transition to Emergency Remote Teaching Practice, and Dustin Grue’s ongoing inquiry into the use of a generative AI tool as a writing tutor. In each of these cases the focus is on understanding and hopefully improving the student learning experience. At the moment, we are conducting three rather ambitious college wide SoTL projects for Reimagine Higher Education… I’ll write about these in later installment of SoTL Corner. In contrast to Applied Research, the focus of SoTL is on what happens within the college, often right in the classroom. So, while an Applied Research project might study the impact of a certain care model for dementia in several local long-term care centers, a SoTL project would examine the best approach for teaching Practical Nursing and Health Care Aide students about dementia. Would the optimal approach involve readings, simulations, role plays, videos and recorded interviews, literature reviews, or guest speakers? And would the aim of the project be on cultivating empathy, learning boundaries and self-care, changing perceptions and orientations, or understanding the medical conditions and considerations? However, SoTL inquiries are not limited only to studying classroom interventions. When I asked Chat GPT to provide a few examples of this, it said that SoTL researchers might also look at:
SoTL doesn’t always seek solutions, according to Hutchings (2000) it can describe a current state of teaching and learning, it can compare teaching approaches, conditions, or interventions, it can show relationships between various factors in teaching (e.g., workload and student performance) or it can seek to explain (e.g., why are students dropping out?) or advance a model of what could be. SoTL is attractive to college instructors because:
So, could an Applied Research Project also be a SoTL Inquiry? Well, yes, sometimes, if… but that is a topic for another installment of SoTL Corner! Reference: Hutchings, Pat. (2000). Introduction: Approaching the scholarship of teaching and learning. In Pat Hutchings (Ed.), Opening lines: Approaches to the scholarship of teaching and learning (pp. 1-10). Menlo Park, CA: The Carnegie Foundation for the Advancement of Teaching. If you are anything like me, you can become very focused on making sure that, within allocated class time, you get through the central concepts in a timely fashion. Because of this pressure, you might hurry through classroom questions, waiting only for a quick response that confirms the answer so you can get on with the lesson. And, since you already know the answer, you are ready to jump in just as soon as you feel the students have had time enough. Trouble is, it usually isn’t time enough. What can seem like an eternity of waiting can sometimes be only 10-15 seconds. This is barely enough time for an online student to hear and understand the question and perhaps raise an online hand! (If you think I am overstating this, re-watch some of your recent recordings! I did… I am very impatient!) Giving think time when asking questions is important. Students need the opportunity to process questions thoroughly and formulate thoughtful responses. If we fail to give enough think time, we will end up with very little interaction–only the very quick, auditory processors will be volunteering answers. The rest of the class may disengage or simply let others do the thinking for them. A class of seventy will quickly devolve into a back and forth from one teacher with 5 or 6 keener beaner students. Ultimately, we want to encourage deeper engagement with the material and critical thinking. We would also like it if all students had an equal opportunity to participate and contribute meaningfully to the discussion! So how do we go about doing this? Well, one way is by setting time limits and conditions. When you present your question, let the students know that they are not to answer immediately, but that you are giving them a certain amount of time to think about their response. Depending on the question, this could range from a minute to four or five minutes. During that set time, students might jot down their thoughts, make an illustration, or formulate an argument. The hardest part in giving think time is keeping yourself from interrupting! That’s why you need a timer! When the time elapses, you might engage the students in a think/pair/share activity (in-person) so that every student gets a chance to share their thoughts with someone before you pull back to a general classroom discussion (More on TPS in another musing!). Or, if you are working in an online environment like Class Collaborate (BBCU) or Kaltura, you might ask the students to type their thoughts into the chat feed and then “flood the chat” when the time limit elapses. Having students press submit at the same time means that everyone can contribute and prevents students from piggy backing on responses from those quick processors. Flood the chat also provides a rich trove to use as you further discuss the topic. Often students provide perspectives and insights that we never thought of. However, you need to be careful with this strategy because the chat is not anonymous. If you are discussing sensitive topics and want to safeguard students, you might choose to get their ideas through an online polling or posting tool (Padlet, Jamboard) that allows for anonymity. Giving think time fosters a supportive and inclusive learning environment by reducing pressure and anxiety associated with rapid-fire questioning. We’d like our students to be confident and willing to participate! We also want them to engage in more than just passive listening, we want them to do some of the work! “Just who is doing the work?” Some of you know that this is one of Jeff’s favorite catch phrases. I try to ask myself this question before every lesson I teach or workshop I lead.
I need to. Like many other teachers I know, I am often guilty of doing the “heavy lifting” of sorting through content, identifying critical concepts and skills, and thinking how best to present key learning outcomes. And, when I’m done, I endeavor to enthrall (or overwhelm or perhaps bore...) my students (or instructors) with my knowledge, insight, and wisdom. The trouble is, when I do all the work, I can cheat my students out of the often exciting, sometimes frustrating, and always mind-expanding struggle that we call learning. They also need to engage with the outcomes in much the same way as I had, otherwise they are simply parroting what they’ve heard me spout forth. Wisdom indeed! Asking the right kind of questions. We can often inadvertently shut down thinking by defaulting to closed questions in our lessons. I call these “guess what’s in my head?” questions. We often use closed questions to check for listening, confirm understanding, test recall of facts, or gather specific information. Examples might include "What year did World War II end?" or "Who wrote 'Romeo and Juliet'?". These questions have their place in instruction, but they limit critical thinking and stifle discussion and exploration by focusing on recall and regurgitation rather than analysis or interpretation. One way to stimulate thinking, provoke discussion and sustain focused inquiry in our lessons is through the strategic use of open questions. Open questions invite students to think critically, consider multiple perspectives, and engage in problem-solving. As educators we would like our students to develop higher-order thinking skills (analysis, synthesis, and evaluation) and to develop professional judgement. So, just pop in a few good questions, right? Not so fast! Simply committing to use more open questions in our lessons isn’t enough. Good questions–questions that push our students to reflect and respond–don’t simply pop into our heads while we are in the high flight of teaching. So, while we prepare our lecture notes, plan out the necessary breaks, and think about our slides, we also need to rough out questions for our students (and ourselves) to chew on while the lesson is being taught. How can we get the students to connect to the learning, and think their way through? One of the best ways to produce a few juicy questions is to review the learning outcomes and turn them from outcomes into inquiry questions:
Questions play an important role in teaching and learning, but when used ineffectively, they can often shut down thinking instead of opening it up. An integral part of any teacher professional growth process is in participating in teaching triangles or teaching squares. These professional triangles are squares formed with your colleagues are used as a foundation to do some collective reflection and facilitate the visiting of each other's classes, either online or in person. This kind of collaboration allows for informal exploration of teaching practices and provides a non-threatening way of securing feedback so we can improve our practices. If you have never participated in this kind of process before, here is how it is explained at Washington University in St. Louis:
Teaching Triangles is a semester-long, reciprocal classroom observation and reflection program geared towards helping faculty think through their teaching practices in a supportive, structured environment. In Teaching Triangles, faculty observe and are observed by two colleagues. Observations are meant to stimulate personal self-reflection on teaching practice (not meant to provide direct commentary on colleagues’ performances). Participants engage in non-judgmental, formative feedback and discussion throughout the semester with other members of their Triangle following the observations. The CTL forms triangles and provides a structure for observation and reflection. Each Teaching Triangle consists of three faculty members committed to working together on their teaching for the semester. Triangle participants will agree to visit each other’s courses at least once during the semester to observe their colleagues teach. This observation will not be evaluative; direct commentary on colleagues’ performance is not part of this process. Instead, colleagues will write down their observations of the class session, and, following the class, reflect on their own practice having experienced what it’s like to be in another faculty member’s classroom. Following the observation and the initial reflections, triangle colleagues will convene to discuss their teaching, using some guiding questions provided by the CTL. From: https://ctl.wustl.edu/programs/teaching-triangles/#:~:text=In%20Teaching%20Triangles%2C%20faculty%20observe,commentary%20on%20colleagues'%20performances). Bowdoin College also uses teaching triangles and suggests these questions as a conversation/reflection guide: What happens in a teaching triangle? Each teaching triangle consists of three faculty members. After an initial meeting early in the semester to discuss logistics and establish expectations, each triangle member commits to visiting the other members’ classes at least once. Following all the class visits, the triangle reconvenes to reflect on teaching utilizing some of the questions below.
From: https://www.bowdoin.edu/baldwin-center/pdf/teaching-triangles-overview.pdf As you can see, the emphasis is on reflection and exploration not evaluation or judgment. In our NorQuest professional growth process resources there are several documents that might help you to facilitate such discussions and help you gather your thoughts. If you are from a small department or would rather get feedback from someone other than your close colleagues, the Educational Developers are always available to come, visit and chat! Several years ago, we published a blog on how this is done at NorQuest. Here is the link: Arranging a visit with an Educational Developer. Since today is the start of our two-day Authentic Assessment Institute and my involvement in this offering has challenged my capacity to crank out today’s MMM, I thought I’d ask my assistant ChatGPT to help write this week’s spotlight on Authentic Assessment. Here’s what we quickly produced. It’s a very serviceable (but somewhat superficial) summary of AA! At our institute we’ll be digging deeper, into not only what it is, but how you might build an authentic assessment and how you might assess it!
Authentic Assessment Authentic assessment is an approach to evaluating students' knowledge, skills, and abilities by providing them with real-world tasks or projects that resemble the challenges they would encounter in their desired professions or fields of study. It goes beyond traditional tests and quizzes by emphasizing the application of knowledge and the development of critical thinking, problem-solving, and decision-making skills. Authentic assessment aims to assess students' abilities to transfer their learning to real-world contexts, fostering deeper understanding and meaningful learning experiences. Grant Wiggins, an influential educational researcher, defined authentic assessment as "engaging and worthy problems or questions of importance, in which students must use knowledge to fashion performances effectively and creatively." This definition highlights several key components or characteristics of authentic assessment:
Examples of authentic assessments across different disciplines or vocations:
In each of these examples, the assessments align with the professional expectations and tasks students would encounter in their respective fields. They require students to apply their knowledge and skills in authentic, real-world contexts, promoting deeper learning and preparing them for future challenges. Citation: Wiggins, G. (1998). Educative Assessment: Designing Assessments to Inform and Improve Student Performance. Jossey-Bass. This book by Grant Wiggins delves into the concept of educative assessment, which emphasizes the use of assessment as a tool for enhancing student learning and promoting deeper understanding. While it does not solely focus on authentic assessment, it provides valuable perspectives on how assessments can be designed to have a positive impact on student performance. NorQuest College students, might use recipe cards as a creative and effective tool to study and memorize important concepts. Recipe cards provide a compact and organized format for summarizing key information, making them ideal for self-assessment and quick review. Here's how our students can use recipe cards for studying, using nursing studies as an example:
In this Frayer model for Pain Assessment, the term "Pain Assessment" is defined and its key characteristics and examples are outlined. The visual representation section can include diagrams, flowcharts, or illustrations related to pain assessment, which aid in visualizing the concept.
Remember, the Frayer model is a versatile tool, and you can adapt its sections and format to best suit the specific nursing concept you are studying.
By employing recipe cards in their study routine, college students can leverage the power of concise summarization, self-assessment, and active recall. These techniques aid in consolidating knowledge, reinforcing important concepts, and promoting effective learning. |
AuthorJeff Kuntz Ph.D. ImagesExcept where indicated, images used in the blog posts are personal photos, images from NorQuest College or images from Pixabay. Pixabay is a vibrant community of creatives, sharing copyright free images, videos and music. https://pixabay.com/ Archives
March 2024
Categories
All
|