Alternatives for exams and assessments

Description of the video:

>> So welcome everyone to today's webinar, Keep Teaching: Strategies for Remote Exams and Assessments. My name is Madeleine Gonin and I work here at the Teaching Center in IU Bloomington. I'm joined by my colleagues Lisa Kurz and Cordah Robinson as [NOISE] well. I know we know many of you, so welcome. Those of you who don't know you're also welcome and we look forward to talking to you and trying to help you with any of your teaching related questions. So our theme for today is to rethink our assessments that we have and think of ways to encourage our students to maybe interact with the content more, get more practice in. So we by designing assessments that encourage that, as opposed to having assessments where students might just quickly go and look up the answer, ask a friend and demonstrate that they can produce an answer in the moment as opposed to one where we are seeing that they have actually had some practice and have really understood the course contents. So with that, I'm going to hand it over to Lisa. >> Thanks, Madeleine. What we have on the agenda today, you can see in the slide, we're first going to as Madeleine said, we're going to talk about how to re-envision your assessment, how to do that, and what options you might consider as you're trying to re-imagine your assessment for an online environment. We'll be talking about some digital tools and strategies and spending a fair amount of time talking about some options in Canvas quizzes if you're interested in doing an exam in that way. Then at the end, we will be addressing some strategies to promote integrity in your test-taking and ethical test-taking. So that will come up at the end. Before we start, I need to explain this particular slide. Madeleine, Cordah and I all have cats. Although their cats are well-behaved, mine, which are the two in the upper left-hand corner, Poppy and Chip or not. So they may very well jump up on the table here and try to help with the webinar. So if they do that, then I apologize in advance. Anyway, they are co-workers. As Cordah said they are our co-workers, or at least trying to be. So the first step in this webinar is to rethink your assessments. There's a couple of things, a major things to keep in the front of your mind as you're doing that. One of the most important is to just think at a very basic level. You want to make sure that your students are on the right track. That they are learning the content. That's the most important thing that we are dealing with right now. What do you need to know in order to be sure that your students are on the right track. That's really where we want to focus our thinking as we think about re-imagining assessments. Another factor to keep in mind is what you've already assessed about of all of the learning outcomes you might have in your course. You may have felt like you already dealt with some of them before spring break, before we move to this online environment. In that case, you may feel like to simplify both your life and your students' lives, you don't need to reassess those again. So rethinking, especially cumulative exams, for example, may be a strategy you'd want to adopt. Finally, or not finally but another point is re-prioritizing. So of all the content you've got left of the learning outcomes you've got which are most important, which are really the critical things that you hope students will take away? Focus on those and maybe let some of the other things go. That's connected with the next point about the scope of the course. So when you designed your course and thought about it being, say 1,000 points spread out over the semester, you didn't imagine that you were going to be teaching the second half of it online. You don't have to, you are not bound to that 1,000 points given these circumstances. So if you, especially as we start thinking about specific assessments and ways of checking on your student's understanding, your temptation may be to assign a bunch, a lot of little things to assess your students learning. In order to get your students up to that 1,000 points, I hope you will consider not doing that so that you don't overwhelm your students. Instead, you can reorganize your points or your grading to incentivize things like the regular homework that students do if you're in a quantitative discipline, or say the reading checkpoints that you would have them do regularly along the way. So those little assessments that mark progress toward mastery, those are the things that you can think about as you reconsider your final assessment. So your original assessment type, you might have been thinking about, say, a multiple choice test, if especially if it's a larger class. You may have been thinking about something written, an essay, short answer exams, some term paper with or without books, notes, or research on library research. On the other hand, you might have thought about something that has more of a media components, say video or audio, maybe you wanted your students to create posters or infographics or give presentations. Or you might have had some combination of the above, a presentation plus something written or a multiple choice test that also had essays in it or something like that. So the first step in re-imagining this is to consider what assessment you planned on originally, and then to look at that assessment and analyze it. So first of all, there's a question of formative versus summative assessment. Most final assessments, like final exams, are summative in the sense that they are designed to evaluate how well your students have learned the content in the course. Formative assessments, on the other hand, are assessments that are designed to provide you and your students with information as they learn that can improve their learning. So considering what may be moving some of your assessment tool or formative direction to give feedback to students as they're learning content, rather than something that's evaluative or summative. If you have a multiple choice test, you can think about sorting your questions by topic. This is particularly relevant if you're thinking about breaking your exam along final exam down into shorter, smaller Canvas quizzes. You can sort them by topic. You could also sort by Bloom's taxonomy level. That is, think about the lower level questions require more straightforward work on your students part like memorization or defining terms, as opposed to the higher-order questions that can be more complicated and more challenging, like application and analysis extrapolation. So sorting your or you can analyze other assessments that you do in the same way. What tasks were you asking your students to do? Then as you re-imagine your assessment, you can choose a strategy based on what kinds of questions you were asking. Just for those of you who are not as familiar with Bloom's taxonomy, it is a way of categorizing the cognitive work that students do in college. There are lower level tasks that students might do, remembering and understanding. They are lower level in the sense that they're just more straightforward mostly. Then there are more complex abilities or cognitive work that students do like applying, analyzing or evaluating. Its unlikely that in a multiple choice exam that you would actually get two questions that [NOISE] would create. But other assessments that you do might be more in the creative end. So when you're considering options, a few things that we hope you will keep in mind that are very important on accessibility and equity. So for example, for your students who have needed accommodations in the past, for your students with disabilities to make sure that they have access to your content so that they can continue to work on and continue to learn it. That may mean multiple ways of providing content to your students. Also a number of your students it's very likely, especially in larger [NOISE] courses, that some of your students will not have reliable access to the Internet. They may not even have good access to a device on which they can do their work. So being aware of those things and being empathetic to them is a good idea in these days. Finally, it's just the case that all of us are feeling a little more stressed out these days, not only you, but also your students and being aware of that, and this is going to come up once or twice again in this webinar. Being aware of those personal challenges and reaching out to your students to see if there's any way you can help them with those challenges will go a long way toward helping them learn by decreasing their stress. >> So once you've analyzed your final assessment, the way you originally envisioned it. Then there are some things you can think about , as you re-imagine it. So if you chose a multiple choice exam, for example, as I said before, one of the things you might think about is doing shorter, what you might call quizzes, as opposed to an exam, and doing those more frequently. That's particularly useful in a remote learning context, because you can't see your students and make sure that they're not looking really [LAUGHTER] confused as you teach material. Because it's challenging to check in with your students on their understanding of your content, then shorter more frequent assessments is a good idea. You can don't have to throw away the exams that you used to use. However, you can draw questions from exams that you've used in the past or [NOISE] that you were planning on using for your final exam. Finally, asking a variety of different kinds of questions, is going to help more of your students, show more of what they can do, what they've learned. So the tip at the bottom, about using questions that reach different Bloom's levels. We put that there because it's something that many faculty have challenges with as they are designing exams, especially multiple choice exams, is writing questions that call for something more than memorization. Just fair warning, when you devise those questions that are more demanding, you'll find that they take longer to create. So allowing yourself some time to create those questions is a good idea. So written assessment strategies, again, as with exams with multiple choice, you may find that shorter, more focused and more frequent writing, serves you better and helps your students learn better than longer term papers. Again, the writing can focus on more recent materials. You have a variety of ways of getting your students to submit their material. They can submit it to you via a Canvas Assignment, which would be a conventional way you probably thought of in the past. Another thing to keep in mind though, is that as you create your course online, you may have students post points in a Canvas Discussion. That's also a writing task, and you can point that out to students, make it clear that they need to consider their audience, that they need to consider what point they want to make, how they want to support their assertion with evidence, that this is a very short focused kind of writing. But it is a writing task, and it's something that you could assess just as well as a longer piece of writing. However, make sure you consider your grading load. That relates both to your own well-being, and not being overloaded with grading, and your students ability to just get through all of the assignments and respond to all of the assignments. So make sure that you don't overwhelm yourself with too much writing, to grade. So the media assessment strategies. Fortunately, Canvas gives us a lot of options here. So students can upload images to assignments, they can also post media things in Discussion and then comment on other students posts. So for example, if you were planning to have your students create a poster, and then you're going to put up the posters, and have a gallery walk or poster session in your class. You can do that in a Discussion instead. Have students submit their posters in a discussion, and then students can comment on each others posts. They can also, of course, demonstrate their learning by submitting some audio or video. That is a particularly interesting possibility, in courses, for example, foreign language courses. Finally, we have what may seem like an old fashioned option, an oral exam. Or for example, if you had students writing a paper and you wanted to learn a little bit more about why they said certain things or what their thinking was as they were doing certain things. You could have an oral exam or allow them to elaborate on something they've submitted in a one-on-one Zoom. So a few additional strategies, this first one is, particularly for those of you who are thinking about multiple choice exams, and who have a lot of questions on your former multiple choice exams, that involve memorization or lower level cognitive strategies. One thing you can do is, instead of asking all those memorization questions in an exam is, assign your students to create their own study guide or reference sheet that mentions all of these things that they need to memorize. All of the terms with their definitions, vocabulary, the formulas, the equations, and so forth. To create their own study guide on paper or digitally. Then as the first question in the quiz, to have them upload that study guide, so that you can verify that in fact they've memorized or they've learned all that. Because the act of creating that study guide, is in itself a learning opportunity. If you have students for whom you want them to show their work, partly because you might want to give them partial credit. You can have them take a photo of their hand-written work, and submitted along with whatever other work they're going to submit. Finally, being in an online environment is a good opportunity to get students to reflect on what they've learned, and this is a double word score thing because not only, you get students to actually learn the material, but then they reflect on what they've learned, which will reinforce the original learning. Finally, Zoom office hours or one on one meetings. Especially, if you're going to give an exam over a certain time period, to make yourself available during, at least during some portion of that time period. Because if students have questions normally in a face to face class setting, you'd be there to answer their questions, or an AI would be there. Someone would be there to answer their questions. In this online environment, you may want to give them an opportunity, to ask you questions as they're responding to your assessments. So we have a couple of images here of media submissions, someone's notes that might be a study guide or reference sheet that I mentioned. Also on the right hand side, an example of a quantitative image, that could be submitted in order to get partial credit for an answer. For example. [inaudible] do you want to jump in here? >> Yes. Lisa, I sure can. We talked a little bit about providing feedback to students as they go along in this new learning experience, and we have several options in Canvas that help us do that. One is SpeedGrader for grading papers and providing comments and annotations. We can also add media and by media comments remain vocal recording, audio recording, or a video recording of your comments, and also text's comments and several tools in Canvas. Shall we look at the SpeedGrader slide? So this is just a screen capture of, well, you might guess it's a fake paper, but a paper uploaded by one of our tests students in Canvas. You can see with the markup tools here, an instructor can provide quite a lot of feedback to the student. Both inline feedback, point by point in the text, either through highlighting, drawing, circling, adding texts, or by call-out comments, such as you might make in a word paper if you're doing some editing there. Then in the right-hand side pane, we also see we can make comments on the assignment as a whole. Maybe that's more of a summary. Below that, there's a little icon for making a recorded comment if you like, if you have a lot of grading to do, or if the nature of what you're grading requires a lot of commenting, doing an audio recording of the comments is a much faster way than typing out the text. So that might be something that's of interest to you. It provides a personal sense to students that you're really paying attention. They can hear the tone of your voice and so forth. I think they really need that personal touch right now, that helps them feel connected and part of a learning community. So that's one of the digital tools that we have available to us and one of the strategies for using it in order to provide feedback. Let's take a look at some more digital tools and strategies that we have to help you get through the course and keep your students engaged. So as we talk about these, you should keep in mind that a lot of times, the strategies suggests the tool that we want to use. Sometimes, the tool suggests a strategy that we hadn't thought about. So try to keep an open and flexible mind as you're going through this new experience of online teaching. And consider, for example, that if you want your students to do some performance, say a speech, music, that certainly suggests a media submission because we can't do that face to face now. But the tools that we have also suggest ways that we might do assessments now. So Zoom suggests oral exams, as Lisa mentioned a few minutes ago. Our ability to record our screens and record video and audio through Kaltura suggests video responses to many assignments. So if you have students showing their work, they can narrate that, talk over it, and explain how they came to their answer result, recording that in Kaltura and upload that for example. So just try to be flexible as you learn new tools, think about new ways to use them. Our discussion today will be mostly about Canvas and the tools we can use within Canvas and also Kaltura and Zoom. Those are the big three tools that are helping us right now as we're getting accustomed to the second half of our semester online, even though we didn't really plan it that way. Canvas has many tools that can keep your students engaged. Discussions tool is one example, Quizzes, and we'll look at some different ways to use the Canvas Quizzes tool in a minute. Assignments tool, of course, most of you are probably familiar with that, but may not be familiar with all the ways that we can use it. The SpeedGrader, as we just looked at a couple of slides ago and talked about using Kaltura for recording, both on the instructor-side and for students to record and upload, and the same with Zoom. So as instructors, you may be using Zoom to do a meeting, you might be using Zoom to record your narration over your slide presentation, or you might be using Zoom to hold office hours or meet with students, and students can use Zoom also. For example, they could record a dialogue in Zoom and upload that to an assignment. Students often have devices such as a phone or a tablet that takes photos or on which they can record video, and they can use that to submit to a Canvas assignment also. Canvas Discussions are something that instructors often use for participation. It can be used for a number of different ways to engage students in your course. I think Lisa mentioned using discussions to have students do writing tasks and certainly multimedia. Because all these tools in Canvas are very media friendly, it's really easy to add media to a discussion. Canvas Discussions are assignments, they can be graded or ungraded, and there are ways for students to write and also to share their media with the rest of the class, especially for the purpose of the class commenting on that medium or commenting on their posts. An example of that we have here was actually used as an early assignment in a course to get students used to using the discussion tool and the instructor had ask students to either create video or find videos, such as a YouTube video that described their hometown to them and then to add their own explanation of why that described their hometown, and then to comment on each others posts. So that was quite a rich writing assignment for students, as well is a nice icebreaker to the course. Here's another way that we could use the discussions tool. Of course we could do this in some other tools in Canvas as well. Did you plan some poster session or gallery walk for your students this semester, for one of their big project assignments? Not so easy to do that now, and yet we have ways that we can make that experience available to students, allow them to comment on each other's work, and for you to be able to assess that work. So students might have posters or projects they've done, they might have infographics or some other graphic work like that. Maybe its photographs, maybe you have a photography class and students can upload those, say, to a discussion as we looked at earlier, where they can share their media and they can comment on each other's and you as an instructor, of course, would want to provide guidelines for those comments. That could be a graded discussion and an assignment. Students might also present their posters in Zoom, and explain their work and their poster just as they would in an in-person poster walk or gallery walk. Many of us are using Canvas Quizzes right now for several reasons. Canvas Quizzes are an easy way to do assessments with students. Part of what makes it a good choice under those circumstances, is most of the question types in Canvas Quizzes are automatically graded. So especially if you have a large class, that takes the burden of grading off the instructor. So if you're used to handing out quizzes, or handing out multiple choice exams, for example, in class, Canvas Quizzes could be a good alternative for you. You can manage access to each quiz with the window of availability that you set in the settings and of course, the due date. You can manage the presentation of questions on screen, whether students get those questions all at once, or if they get one at a time, or if they're allowed to go back and relook at a question that they've already answered. Canvas Quizzes also has ways to present a unique set of questions to each student who's taking a quiz. That includes randomizing questions that are drawn from a group so that each student gets a different order of questions or perhaps a different subset of questions. It also allows us to shuffle answers. So students get a different order of the answer options. There's a good variety of question types that are available in Canvas Quizzes, and that can help you assess a student's learning, not just their ability to recall or memorize, but get a sense of how well they're comprehending by asking essentially similar question through various types of these questions. Canvas Quizzes also has some analytics to help you as the instructor, know what your students are doing as a whole in the quizzes and whether or not you might need to rethink how you're presenting some material, or how you've ordered a question, or something along that order. >> Question types in Canvas Quizzes are a variety of multiple choice question types, and then an essay question type. So the variety of multiple choice questions are automatically graded, and they provide many opportunities for retrieval practice for students, and it's great to create a number of quizzes that students can use for practice in this way because it really promotes their long-term learning. The way that you compose the questions can help address different levels of learning in Bloom's Taxonomy, from recognition, through comparison, and analysis all the way up, and you're certainly not limited in the complexity of questions that you might compose, and that will help you get to higher orders of thinking with your students rather than them just recognizing the correct answer A, B, C, or D, and Canvas allows you to use these questions in combinations with each other as well. Sometimes you need to ask an essay question that will need to be graded individually by each instructor. In the essay question, of course, there's a text field students enter their responses, can be either long or short answers, although we would recommend not requiring students to spend a lot of time in a text field or compose a long answer for a text field, you might want to use assignments for that. The text field in an essay question gives you the opportunity to ask a student to explain or elaborate on a previous multiple-choice question response they may have given to a question that you've answered, and we'll see an example of that in a minute, and that input field also links to Kaltura and enables media response, a video response, or dropping in an image for students, which also can provide more enriched texting environment for your students. When we begin a Canvas quiz, if you have not done that recently, I just want to point out before we talk about any details of the Quizzes tool. You'll be asked to choose classic quizzes or new quizzes, right now, for what we're doing, we want to choose classic quizzes, and we'll learn a little more about new quizzes later in the semester, maybe getting ready for the summer or sometime during the summer, but right now we're going to be using classic quizzes. So Canvas gives us many opportunities and choices with which we can customize a quiz to help create different learning experiences for students. Of course, you can set a due date, but we can also set what I always referred to as a window of availability, so the quizzes only available during a specific time that you set or between specific dates that you set in the Canvas Quiz Details, and you can set the number of attempts that you allow students. Now normally, we only have one attempt to take an exam, certainly that would be the case in class in a face-to-face exam. But we can't quite simulate the face-to-face experience, and we know that students have many different digital environments from which they are working. So it doesn't hurt to give students more than one attempt at a quiz if you think that's appropriate. We can allow students to view correct responses at a particular time, maybe after all the quizzes are graded. We can set how students view the questions during a quiz or exam then they see them all at once or one at a time. I mentioned we have settings that shuffle the answer options, and also there are ways to extend both the number of attempts that you might want to give, the time available for taking the quiz, or the deadlines for the quiz when that may be needed for individual students. For those unfamiliar, this is what the Settings screen actually in Quizzes, it's called Quiz Details. Looks like in Canvas quizzes, we won't go through each one of these, but just to point out shuffling answers, showing the correct answers at a specific time, locking questions after students have answered the question, and the availability of the quiz between a certain date. If you want to know how to set these up exactly for what you intend with your quiz or exam, do check in with one of your teaching center consultants to go over what each one of these settings means, so that you get exactly what you intended. We want to remember when we set the times that are quizzes available, our students may be dispersed all over the world, and certainly they are dispersed throughout the United States in different time zones. So we want to keep that in mind, the time zones, so that things are available to the right students in the right time frame, unless you know that everybody's going to be taking it in the same time zone, and of course, that means you're seeking feedback from your students. So we went to keep you doing that throughout the course. Question presentation, a couple of times I've mentioned randomizing questions, shuffling answer options. This is something that is certainly helpful if students are all in the same room and they're taking the quiz together, seated side-by-side, obviously, that's not the case. Right now maintaining good academic conduct is very much on people's minds right now. But I also might suggest that randomizing the order of questions, particularly in practice quizzes that are leading to a summative exam is a really good way to help students fix that knowledge in their minds, because they are not depending on just memorizing the order of questions in a quiz, which is pretty easy to do if you just have a 10-question practice exam. But if you're shuffling, if you're randomizing those you're getting a different set of similar questions every time you take a practice quiz, that really helps, and shuffling those answer options also can be a help so that students don't fall into just a fixed state when they're responding to the questions, and it's easy to create the practice quizzes on the fly using question banks. So in order to randomize your questions, in other words, to get a random draw from a set of questions or to get a subset from a given set of questions, you want to use question groups, and you can see what that looks like. Here in this screen grab of a Canvas Quiz, we see a table that is the group of questions, and a couple of questions that are within that group, we can pick so many out of the total number, and then if you have question banks, and the more you use quizzes, the more you will build up question banks, you can pull from those question banks to create a random set of questions, and this is an easy way to make lots of practice quizzes on the fly. It literally takes one or two minutes to create a quiz for students to use for practice. So this is really good for students staying engaged with the content and doing self-assessment. >> So we talked about the multiple choice question types. To be specific, we have true/false questions, multiple choice, which everybody is familiar with, multiple correct answers, so students have to select all the answers that are correct out of the ones that are given, and we have what's called a drop-down, which is a bit of a mystery, and I'll show you what that looks like in just a minute, we have a matching question option, and we have a formula question option which allows you essentially to present a unique question to each student, and also allows them to respond within a range that you permit. Numeric questions allow both problem questions and word questions for which there is a numeric response from students, and again, that can be within a range that you specify. This is what the drop-down option looks like. You can set these up in different ways, but basically, it allows you to present a drop-down menu of choices for each blank that you leave, and you can have one or you can have several blanks in a question. This particular question has to do with understanding verb tenses and when they would be used, and we have the same answer options available for each blank, and a student has to understand how to apply those verb tenses in order to know which would be the correct answer. The responses are the same for all of these. We suggest that if you are used to doing a fill in the blank question in which students produce the answer, and of course we understand the value to that, we suggest maybe you would want to use this as an option as there are many parameters that you have to set in a fill in the blank question, and it's too easy to miss what you might accept as a correct answer. So this is a good alternative to that. This is a matching question. It's fairly self-explanatory also. But they look different in different programs and applications you might have used matching questions and quizlet or something else before. This is what it looks like in Canvas. It also provides something like a drop-down menu. In this case, the drop-down is the same for each blank, whereas in the drop-down question that we looked at previously, you can have a different set of responses for each blank. Here's the formula question from the instructor side of it. You can see there's quite a lot to specify, but this is what enables you to present a unique question to each student, and this is especially valuable again in quizzes for practice and mastery. Students can get a similar formula question with different numeric values in it each time they draw the question, so that can be very helpful. You can, if you wish, provide an option of acceptable responses or a range. Here's an example of a numeric question. This happens to literally be a numeric question [LAUGHTER] that also requires a numeric answer. But you can have a word question that requires a numeric answer as well, and again, you can specify a range that's acceptable. So the essay question. That's a question that's graded by the instructor. It has the input fields so students can type into it. There are a number of formatting options for students if they want to use those in the Rich Content Editor, which you see in virtually all the tools in Canvas. There is a link to Kaltura media, and so students can include media in their response, or they could upload an image if they wanted to. Kaltura media is generally video media or audio media. The instructor can grade and comment on these essay questions in SpeedGrader. I do want to say for longer texts in that input field, you might want to have students compose in Word and copy and paste that into the input field just in case they might get kicked out of there and loose something that they typed. But for any lengthy essay, we want to do that in the assignments tool. We mentioned combinations of these questions. So here's an example of a combination multiple choice question followed by an essay question in which students explain their work. Lisa, I see you have popped up on the screen. Would you like to comment on this a little bit more or go directly to the next slide? >> No, I just wanted to say that this is the classic brain teaser about the full glass of water with an ice cube. As the ice melts, what happens to the level of water? But it's the question, and this won't work for all multiple choice questions, but this is the question where you want to know not only whether the student knows the right answer, but also why they chose that answer. So this is a great opportunity to ask why is that one right or why are the other options wrong. So that's why I chose that particular question. >> Okay. >> Okay, so we want to talk a little bit about exam integrity and promoting ethical exam taking. There are a number of options we want. Of course, when you would like your students exam responses to reflect the fact that they studied, practiced and can apply this concept. However, i mean, so a, in this case is the correct answer to this particular question. But there are some other options like I crammed and remember this for now or I totally guessed or I Googled it. I asked my dad yeah. So there are other ways that students can answer questions on an exam other than actually knowing the question. But we hope that students performance on an exam will reflect what they actually know. So basically, there are some basic categories of options you have to promote exam integrity. One is using different question tags to get at different levels of understanding of particular concepts. So some questions that are more straightforward, that maybe memorization or understanding, and other questions that ask more complicated tasks. [inaudible] also talked about ways of randomizing question presentation using question banks and question groups and also by shuffling answer options and that's another way to promote integrity. A third way is to use an honor pledge. This can be set up as a regular question in your exam or quiz. The first question can say something that is in honor pledge. Like I promise that I will neither give nor receive help from others on this quiz, and then have students click yes or typing their names into a text field as a sign that they've read and agreed to that particular pledge. Finally, there is the option to do online proctoring, which we're going to talk about in just a minute. But something like an honor pledge, like what's shown here could be either the first question on your test to remind students that they are on their honor as they do this or it could be the last question or it could it could appear in both places so that you actually ask, you remind them at the beginning and then you ask them to affirm at the end that indeed they didn't receive any help in responding. Online proctoring and you may have heard about options for online proctoring. There are several different tools that are being considered. There are important constraints to keep in mind as you think about using online proctoring. By the way, the way this works is typically that students sit down at a computer or a laptop or tablet and take the quiz and the camera on the computer. If you have a webcam or the laptop video records, what they're doing as they're doing it. There are issues of downloading software. In some cases there is a cost and if you plan on using one of these online proctoring services, your department or academic unit may have to pay for it for you. The technical support is not fully established yet, I guess I would say. But the bottom line is that it's important to contact your teaching center to get some more information about this. There are other concerns with online proctoring about what to do with students for example, who don't have a computer with a webcam, who don't have a laptop that includes a webcam or and of microphone and accommodations for students who need accommodations and whether the online proctoring systems would allow for that sort of thing. So there are a lot of concerns with online proctoring and finally, just may be the case that it doesn't catch all the instances of cheating. It may not catch them all. It may deter some of them, but it may not catch them all. So final thoughts, and I hope [inaudible] will jump in here too. Most importantly, communicate with your students, tell them ahead of time what you're planning to do and as part of that communication, find out how they're doing, whether they have the technology, the resources they need, what's their stress level as they proceed through the semester. Because that's something that you can use to gauge the kinds of assessments you want to deliver. Give opportunities for practice, especially if you're trying something new with your students, it's important to give both yourself and your students a chance to practice it to find out how it works before you get to something that's high stakes. Finally, it's really critical to have a plan B, or C or D for the students who may need it. >> I would just jump in to underscore that Canvas tools offer so many ways to communicate with your students. Of course, there's the messaging tool to communicate directly and announcements and that sort of thing. But through commenting and so forth. Communication is two ways. So make sure you ask them how they're doing and collect feedback throughout the weeks ahead so that you know how to adjust things based on how your students are really getting along in the class and what their stress level is. We don't know what's going on at home with them. So we have to be sensitive to that. There's so many opportunities not only to just practice learning how to use some of these tools, but to practice with the course content using these tools. Quizzes is an obvious example, setting up all these quick practice quizzes, but also having students interact with each other in their own private Zoom meeting. I mentioned foreign language for dialog and that sort of thing. The plan B can be as simple as suggesting some settings so that students who need additional time or need a do over can have it. Or it can mean a completely different plan B, using nothing but pen and paper and snail mail. But just be ready, be flexible and be ready to help your students out. >> We wanted to mention several other. >> Lisa I think we're having trouble with your audio. Shall I go through these webinars or [inaudible] , why don't you? >> Yes. We just wanted to let you know that we have a few more webinars coming up. We had one this morning. I saw some questions about exam cheating. It was recorded. So will be available sometime next week on the teaching site. Then we have other webinars that are about writing assignments. For those of you who know that you'll be using Canvas quizzes for final exams. We have two of those webinars coming up, so I highly recommend you also check those out. Overall, I would say please come and see us. We are happy to talk through your particular assessment needs and try and figure out the best way to meet those goals that you have. At this point, yes, we also wanted to highlight a few resources. Certainly come and see us at the Teaching Center. Remember, you don't have to be technical support for your students. So 2:00 AM, the UTS support centers there and if you have questions at 2:00 AM about technical issues, certainly contact the support center. I think we're probably all familiar with a keep teaching site by now, but definitely also send your students to the keep learning site there some useful strategies for them there as well. I know there's been quite a few questions in the chat, so at this point, I'm going to stop the recording and take those questions out loud. Thank you.