Conduct exams via Canvas

Description of the video:

>> Thank you for joining us this morning. This is the final exams via Canvas Quizzes webinar as part of the keep teaching series. I'm Andi Strackeljahn, and I'm a Principal Digital Learning Consultant at the IUPUI Center for Teaching and Learning, and I'm joined by Zach Carnegey, who is also a Principal Digital Learning Consultant, but at IU Digital Education Programs and Initiatives. We also have a couple of consultants, Jenny Yang and Lynn Ward, who are joining us to answer questions in that chat. So if you have questions as we go, please feel free to ask them there. Also at the end we have left time for your questions. So once we are done getting through our fairly brief presentation and then we'll give time for questions as well. So today we're going to talk about what makes a Canvas quiz a Canvas quiz and when maybe you should sometimes use an assignment instead. We'll also cover how to modify your exam to help encourage academic integrity, as well as how to make accommodations for your students and communicating your exam expectations. Because we all know, especially in our current situation, how important communication and being clear about what's going on is so important for your students. If you have questions about that technical click-throughs of how to create an exam or different types of exam questions, there's a link here to take you to the Canvas guides. The Canvas guides are wonderful. They have clear screenshots and steps for pretty much everything you could ever want to do in Canvas, and Lynn has put that link there in the chat. So just to take a moment to think about what is our current situation. You, we know that you didn't sign up to teach this course online and you might be feeling some combination or a little bit of all of the above of these feelings of frustration, surprise, overwhelm, being uncertain, under prepared, stressed. That's totally understandable. You know what, your students, they might be feeling the same way because they didn't register to take this course online, and they might be feeling frustrated, surprised, overwhelmed, uncertain, under prepared, or stressed. So to get an idea of where we all are, I have actually created a little poll in Zoom for us to try out of what is your current office situation. So I'm going to hopefully launch this poll, and a little box should show up on your screen and you can click as many of those options as apply to you. So if you are battling for bandwidth while also sharing your office with children, co-workers, you can click on both of those things, or if you're battling for bandwidth with your cat that could be a problem too [LAUGHTER]. >> A lot of space. >> [LAUGHTER] I'll give just a couple more seconds for you all to respond to this poll. Once people are done responding, I will share out the results with you all so you can see what's going on. So I'm going to go ahead and close that poll and share these results with you. So looks like 50 percent of people are sharing an office with another adults, which is our highest hit there many people also have to share an office with fuzzy pet co-workers, which again they can take up space or be very particular, it found mine to be very particular. I'm going to go ahead and close that and get that out of our space there. But this is just to take a moment to say, we're all in a different situation than we ever would have expected. But we're all in this together, your students, they have more competing responsibilities now then maybe ever before. If we don't adjust expectations for our courses, then we risk grading students on privilege. That's hopefully something that you don't want to do. I think we can assume that something no one really wants to do. We want to know if our students understand sociology, if they get the basics of psychology, not what their situation is. So today Zach is going to talk a little bit about when you might want to use an exam in your course and how that fits in with your goals. >> So with that idea that Andi just brought up of, we really are looking at trying to measure students on what their true understanding is. Then the first way that you might consider doing this, since this is such a new situation for all of us, then is the way that I have been treating a final, if we want to think of final in the traditional sense of final exam final test. Then is the way that I've been treating that accurate for what or my students being asked to do what they really need to do within the final. So then the first question really in dealing with that is if we're talking about delivering this now online, is this really a quiz or is this an assignment? I use the word Quiz and this form to talk about the Canvas Quizzes tool and assignment in this form as Canvas assignments. So those are two potential methods that you could use for looking at a final examination. So what you see on the screen right now are the questions that if you want any of the things that are on the screen, and we'll go into all of these in greater detail throughout this webinar. But if you want any of these things, then what you're going to want as a Canvas Quiz as your tool for delivering your exam. So therefore, if you want students to work for a very specified period of time on the exam, if you want them to see different questions or potentially different sets of questions. So then tests are individualized. If you want students to only see one question at a time, thinking especially about procedural questions where the, where one question that follows gives a hint as to what the answer was to the previous. Or awarding points based on specific questions rather than a lump sum. Finally, if you want to give students automated feedback about their responses rather. So if any of these things are what you want, the tool you want for delivery is Canvas Quizzes, if you don't need any of these things, if you've instead are taking a slightly different route for your final exam, then Canvas assignments will be a much easier tool for you to manage. Canvas Quizzes has a lot of amazing features, but with features comes complexity. So that's why this webinar we're going to be going into some of what those features are and therefore why pedagogically you might choose one over another. So with that in mind, let's say you decide, yes, I definitely do want, for example, each student to have difficult questions or different sets of questions. So you've decided you want to do Canvas Quizzes as your delivery tool. Then if you go to Canvas Quizzes, you put in your questions, you maybe set a time limit and published it. Of course, publish, that's going to be something you'll probably hear a say a lot because it's easy to forget about that. Then your students click on the quiz and then what they see as this. Andi, could you go to the next slide? Thanks. So this is the student view of as soon as a student clicks on the quiz link, then this is what they're going to see. >> So I'm going to ask you to take just a few seconds to, again, let's imagine your students haven't ever seen this before, and that's highly possible. Then try to decipher, what am I supposed to do next if you're in your student's shoes. So it's a little bit intimidating, I think to see this and not have any more information. We can see that there's no due date which is peculiar, but there's an availability and there's a time limit. For clarifications sake for you all in case you've never done this before. Once they click that take the quiz button, that's when their timer starts. So that 75 minute timer, it starts as soon as it's take the quiz. So if you decide that you do need to use Canvas quizzes for what you need in your final exam then probably academic integrity is one of the foremost thoughts in your mind. So then Andi is going to in the next section here, take you through some of the features of Canvas quizzes that will help support academic integrity and most importantly what some of those impacts are to the students. >> So a lot of times when we think of final exams especially, and we think of things that need to be graded quickly and might have a lot of multiple choice or close ended items like true, false, fill in the blank. But actually one of the key ways that you can encourage academic integrity on your final exam, is to use some open-ended items. From this perspective, in our current situation, it's easier if you just assume that students are going to take your exam as an open book, open note exam. In this way, then you might want to add open-ended exam items so that you can actually again, coming back to that idea of actually have some insight into whether or not they actually have learned what you hoped they have learned as opposed to maybe how good they are at navigating your text book. So having some open ended items will decrease crowd source answers and will encourage your students to actually engage with the course concepts that you're needing them to learn. So if you already have a multiple choice item, which maybe would look something like this. Which of the following differentiate a eukaryotic and prokaryotic cell, A, plasma membrane B, membrane bounded cell structures C, ribosomes D, cytoplasm. So that's a perfectly fine close-ended question. You can modify it to be an open-ended question by saying, describe a eukaryotic cell. Cool. Now you've created an open ended question, but this one is kind of difficult for students to interpret what exactly you're looking for in their definition. If it's difficult for your students to interpret, it's going to be difficult for you to grade. So you could modify this and say, compare and contrast the key features of eukaryotic and prokaryotic cells. This question, the scope of it is very explicit. It communicates a clear expectation, as to what students need to be writing about. It requires higher-order thinking. So they can't just flip through the pages of a textbook or Google something to find the answer to this question. They're going to actually have to put intellectual work into constructing this answer. So this open ended item, it communicates the clear expectation to your students. It also eliminates the possibility of them guessing a correct answer when you have a multiple choice question. You know, they can use process of elimination or just random guessing and there's a 25 percent chance they would get that right. Also, this requires higher-order thinking. So you know that students have to be able to define eukaryotic and prokaryotic in order to actually answer this question. So I'm thinking about actually building this out in your Canvas course. This is a screenshot of actually building a question inside of a quiz and again, if you have questions about the exact click through and the steps that you need to take inside of Canvas, there are wonderful Canvas guides that show you all of those specific clicks. But when you're editing a question, you'll see the same rich content editor that you see all over Canvas. It's the same as if you were creating an announcement or editing the page. You have this wonderful information right here. So it might be tempting to embed media or to embed files in your exam questions. Because technically you can, however, depending on your students device, depending on their internet connection, trying to download a whole word document may not be possible for them in the midst of taking your exam. So think about a student taking your exam on a smartphone, which we know happens and is definitely a likely situation for your students. So they're trying to take your exam on their phone and they need to open a Word doc or a PDF. They may not have an appropriate app on their phone to open that. So now they're scrambling and very stressed and maybe not even able to access a necessary material to answer the question you've given them. So kind of the same idea comes with media. If you have any media in your course, hopefully it's encounter, whether it's audio or video. But it may be difficult for your students to watch that video during the exam because their child is sleeping in the same room or their internet is really rough at the time and so, putting files are embedding media in your final exam is really not the time to try that out. Here's a little. Are you going to add something to that? >> I was thinking and regarding that also, is in thinking about the bigger situation with how your students could be interacting with this is if you decide, I know Andi is going to get to talk about the timing in just a little bit. But, if you do decide to have a timer for your exam, so it's going to be specifically 60 minutes long or 75 minutes long, then them watching that video will count against that time. So anytime as soon as they click that button, the timer is running. So that means that every time they maybe need to watch that video again, or if there's any lag and watching that video, that's going to be counting against them. Going back to the point made of let's try to be really careful we're not grading on privilege. >> Yeah. So as soon as your students have started that quiz, yeah, if they need to watch something and that is problematic or that you need to download something. Yeah, that's eating away at their precious time. One thing I wanted to point out to you all is when you are editing questions in a quiz, make sure you click this very important update question button. Because if you just skip to save or if you're done save and publish, and haven't first updated a question, then it doesn't save anything you did inside of that question and you will be sorry. So learn from our mistakes. Always click the update question button. >> So to wrap up our conversation on media in your exam questions, really, whenever possible, avoid using media in your final exam. If it's absolutely necessary, because you teach film studies or something like that, then makes sure that the video you're using is, if possible, available and advance, and 100 percent the captions need to be edited. Kaltura does create mechanical captions of all files that are added to it. However, those mechanical captions, depending on the accent, depending on the technical language involved, there fidelity is not great. So even in looking at our Canvas webinars. Webinar is transcribed as weapons, weapon in webs. All sorts of totally heinous, silly things will be generated in your captions. So please please edit these. If you're using images, also make sure that they have alt text. Again, coming back to the idea of if you're taking an exam on a smart phone, images can be very difficult to interpret and see important pieces of when you have a very small screen and resizing can be difficult. So just think very deeply about their role in exam questions. So there are quite a few features of or options that you can set in a quiz that you might be drawn to that seem like a good idea. So you can set an availability window. You actually saw that in that screenshot of a quiz that Zach showed us, that there was an availability windows set, meaning the exam is published, but it's not actually available to students to start. There's also a duration timer that's separate. So the duration timer is the timer that starts once you click Start exam. So an availability window could be a couple of days. You should definitely ensure that the availability window is greater than this officially scheduled final exam time and overlaps with that officially scheduled final exam time. But your duration timer can be in minutes. So do make sure though that this timer is still set to be a rational amount of time to complete the questions that you have given your students, especially considering any access to media or other things that they might need to refer to, such as textbooks or other materials. There's also an option to show one question at a time, and while this might make it more difficult for students to share out your exam questions with each other. We'll also talk about another way that you can do that, that's better. But it's not the greatest idea to show one question at a time, because it makes it very difficult for your students to prioritize their time, and we know that good test takers, they go through and they answer the questions that make the most sense for them first, and then know how to make them most of their testing time. So if you only show one at a time, it's very difficult for students to have the autonomy to take your exam in the way that's best for them. That's the same with the option that comes up alongside of that, which is a lot question after answering. That means, as soon as your student answers question one, they're given question two, and they're never able to go back and change their answer to question one, and that's probably the most drastically different, as you could get to a classroom testing situation. In a classroom, you handout the final exam and students have the full time to work on it. You're not walking around and handing out a note card or a piece of paper with question one, and then as soon as they're done with question one, collecting that from them and issuing them question two. So doing this in Canvas is the furthest thing from what the exam would be like in your classroom. If you're concerned about cheating or students collaborating, really a better option is to use test question banks, and you do this in Canvas by creating question groups that then select randomly questions from a question bank to create a unique set of questions in a random order for each of your students. Now this is a great idea. However, do know that Canvas is only as smart as you tell it to be. So if you put questions of all different types or all different difficulties in one question bank, you could end up with a student having, a bunch of essay questions and no multiple choice items, while another student has all multiple choice items that are much easier. So make sure that you create your question banks according to difficulty or according to type, whatever makes sense for your course. Do not just put the entire exam into a single question bank. So if you have your quiz or if you have your exams set up in quizzes and you've taken some of the advice that we've talked about, then your exam might look like this, for your students. So this is from the student view in a course. So here I have clear instructions on what materials I can and should have available during my exam, I can use my book, I can use my notes. I shouldn't be consulting my classmates and peers, and I shouldn't be consulting other Internet resources. So it's very clear to me before I click that take quiz button, what I should have out in front of me and what I should have closed as the student. Also, I have an idea that I need to taking this in an actual regular Internet browser and not be using the Canvas app so that I can access my etext, and that I'm going to need a consistent Internet connection. Now you might think my students know they need to have a consistent Internet connection. But it's worth calling specific attention to this because if they fall out, if they're kicked out of the exam partway through because they lose Internet, then they will be locked out of the exam unless you have set multiple attempts or directly unlock it for them. So it's worth calling your students attention to that specifically. Then also I have a link here. So as a student, if I'm going through and I run into technical problems, I know hey, these are the people that can help me, because you're an instructor, you are a master of your subject area. You are not meant to be your student's technical support, there are people that are here for that, thankfully at IU. >> So now Zach do you want to talk a little bit about some of the features once we've published our exam? >> Yes, please. So once your exam is published, there is a number of options that you have in order to address accommodations that your students might need. So these could be officially documented disabilities, so a documented accommodations, so therefore your student has been working with DSS, or with your disability services office on your Canvas, since every Canvas name is slightly different for that. It could also be that you decide on a student by student basis, someone does have an extraordinary situation, a life situation that needs to be accounted for. So as Andi said, we're not going to trouble you with the click-through on this, so that you would be able to do it at your own time, this is all in the Canvas guides, but just wanted to make sure that you know that these options are available, for example giving students extra attempts, because of maybe an extraordinary life situation, and then of course also you can have extended time. As I said before on a student by student basis, you can go in and make these changes. So it doesn't have to be a global, you give everyone an extended attempt, or an extra attempt, or extended time, although of course you could. But this is so that a single situation or a few situations can be accommodated for. As well, when you get students have submitted their answers to the exam for any of those closed-ended items, multiple choice, multiple selection, numeric answers, then those are auto graded, and you set that up as soon as you write your question, then you specify, for example on numeric answers you have the possibility of setting a range of possible numbers, that as long as they enter a number within that range then they get the answer correct. For fill in the blank, you can specify a certain level of correct spelling for the word. So all of those are auto graded, for anything that's an open-ended question, then when you grade that, then you're going to be using SpeedGrader. SpeedGrader is still within Canvas, then this is the same SpeedGrader that if you've used it for students submitting papers, or any other document, then the functionality ends up being the same. You can as you see on this screen, down at the bottom it says additional comments. So if you want to make comments on a question by question basis, then you can. Over on the right, you will see there is assignment comments, I know it's saying assignment, but in this case assignment has something which has a grade, then you can make basically global comments about their quiz to the students. Now something important to bring up here is, unless you have otherwise specified, once you click that Submit button after you've given this particular question or questions a grade, then that is going to post those grades and that comment to the student. Now you might want to do that, but you might want to wait until all of the exams have been graded, which is more common. That way, if you've decided to grade someone's exam, and it's still during the exam window, and then you submit their grade, then that student could share all of your comments with their peers, and therefore give their peers an advantage on the exam. So you can make this change, it's in the grades tab and navigation, and its grade posting policy as what all of this is under. So if you want to look up grade posting policy and the Canvas exams, and that will take you through some of the options that you have there. All of that brings us to ultimately the communication with students. What is critical and what I think you can hear Andi and I talking about or bringing up repeatedly is, just how important it is for you to explain what your expectations are for your students, be able to provide any necessary media if there is anything beforehand, and basically just give them as much information ahead of time so that they can make this work. By giving this information early, then presumably they'll be able to navigate this difficult space, and be able to meet those expectations, or at the very least, communicate with you why they can't meet it, and therefore you can make a reasonable accommodation for them. So a couple of the ways to do this just as a quick prototype/commercial break is, you can use announcements, these go to all students, but this is really about one-way communication, you are sharing that information out with your students. Another way of course that you have that's built-in Canvas and is therefore secure is using the global inbox. So this is for especially two-way communications and private communications, so you would be able to have basically an email thread with just one student, but it's all held within Canvas, and it's all associated with just that course. So rather than potentially setting up complicated rules within your normal work email box, then this is one way that you could make that work a little bit better. Now, what if you need a little bit more help? Andi. >> Well, the most important place probably for you right now is the KeepTeaching websites. So KeepTeaching.iu.edu has the most up-to-date information on the university's status, as well as resources and strategies, to keep you teaching and your students learning during this time. So they were actually be a recording of this webinar available under the resources in the next week or so, along with that are the recordings of our other KeepTeaching webinar, is about Canvas, is about using Zoom and Kaltura. So you heard me mention Kaltura a few times in reference to media in your course, if that's a brand new tool to you, and you want to know more about it, definitely check out the webinar that [inaudible] has done, that's available on the KeepTeaching website. Also the Knowledge Base has wonderful guides information about all of the IU technology, and how it's set up specifically at IU, and important KB article to be aware of is, how to contact your local center for teaching and learning or teaching and learning center. So each Canvas at IU has its own teaching center, and the consultants there have specialized information on how to best support you. So if you have questions that we didn't answer today, or that you want some further conversation on, reach out to the folks that are at teaching center, and they can help you. Also there's this link to the Canvas Quizzes guide, which again is all the technical clicks that you need to do to create your exam in Canvas. So with that, is there anything that was discussed in the chat that I should show? >> Just in case, maybe it's worth talking about for anyone who wasn't looking at chat, there's some questions about LockDown Browser, and whether or not we should use that, and what value there is. >> So LockDown Browser is a tool that is available at IU, its original use is for testing centers where there's someone to validate student's identity, and then to check that the resources they have out or don't have out are appropriate, it is not recommended for use outside of a testing center. So there are significant hardware limitations, as in it is very problematic on a Mac, students have to be using a full computer, so they wouldn't be able to do this on a smartphone. So for all of those reasons, it doesn't give you anything as far as test security really goes, because students could still have another device alongside them as they're taking your exam in a LockDown Browser. So it doesn't really give you much as far as test security goes, and it's just going to create a lot of problems for your students. >> I think even though again, I feel this was pretty well cleared up in chat, but just in case there's any other misunderstandings about, the idea about using a multiple question banks versus one single question bank. If you just have an exam that is only multiple choice questions, could you just use a single question bank? >> Yeah, I mean, you could as long as you aren't concerned about students being asked an equal number of questions from each unit, or it's totally okay if they end up not getting asked questions from an entire unit. If you're okay with that level of randomness, then okay, you could do that, but I think in most situations people want to make sure that you're finding out if your students understand content in each of your units or each of the chapters, and so that's why we would recommend not putting it in one giant quiz bank or question bank. So I'm going to actually end our recording.