We had some requests to post the PowerPoint we used in our February 13 workshop. In it we give some examples on how one works across the matrix row from outcome to assessment type to plans for the future. Let us know if you have any questions. I will also post the grid that we based on what is available on the QEP section of the teaching.ua.edu website.
Formative assessments are those that help to informally measure a student’s understanding of a concept or concepts and thus have very low stakes or are not graded at all. Regularly, formative assessments are used in classes to gage the effectiveness of the teaching being provided and/or as “check in” (often “on the spot”) activities to ascertain whether or not students are effectively learning the concepts being taught, and as such provide immediate qualitative feedback. The results of formative assessments more often offer information about the environment for learning as opposed to evaluating precisely what is being learned.
Examples of formative assessments:
1. Brainstorming activities
2. Think-pair-share activities
3. Muddiest point
4. Likert Scale clicker questions
Summative assessments are those that formally measure student learning in a quantifiable way at a particular time. In other words, these types of assessment aim to clearly and logically “summarize” learning at a specific point in the semester. As opposed to formative assessments, which assess the environment for learning, summative assessments measure the rate, quality, etc. of learning itself.
Examples of summative assessments:
1. Tests
2. Quizzes
3. Exams
4. Graded papers
Direct Assessment Measures are those types of measure or activity that involve looking at or analyzing actual samples of student work produced to demonstrate that specific learning has taken place. They are, in other words, a quantitative measure used to chart the progression, quality, rate, etc. of student learning.
Direct assessment measures include the analysis of:
1. The results of final exams
2. The results of capstone projects
3. The results of senior thesis projects
4. The results on exhibitions or performances
Indirect Assessment Measures are those types of measure that involve looking at or analyzing student-derived responses or opinions of their own learning and the learning experience. Results such as these often imply that learning has taken place rather than document it specifically; they are a qualitative measure used to understand, for example, student satisfaction of the learning process, a student’s sense of his/or her own best learning practices or learning environment, student opinions of instruction, etc.
Indirect assessment measures include the analysis of:
1. The results of a student exit survey (from a course or from a program/degree)
2. Self-assessments
3. Student opinions of instruction
4. Career path or placement after graduation
Helpful tip: Formative and summative indicate TYPES of assessment. Direct and Indirect measures are METHODS for measuring student learning.
Related Terms:
Assessment Measure: Any assignment or task that is directed toward student learning, which may be summative or formative and whose results are evaluated directly or indirectly.
Quality Enhancement Plan (QEP):Programs put into place by accrediting bodies (e.g. SACS) via institutions (or by institutions on their own) that reflect and affirm the commitment to learning in higher education. QEPs are usually premised on the assumption that student learning (and the caliber of that learning) is foundational to the mission of all institutions of higher learning. QEPs usually begin by identifying an area for improvement, followed by the implementation of the means (or measures) by which to make the improvement, and then the assessment of those measures to indicate that improvement has taken place.
Examples of QEPs
UA’s QEP: http://www.ua.edu/qep/mission/content01.html
UAB’s QEP: http://main.uab.edu/Sites/DOE/QEP/50582/
University of Georgia’s QEP: http://www.qep.uga.edu/index.html
University of North Alabama’s QEP: http://www.una.edu/qep/
Matrix: The chart used by the University of Alabama to organize and record data related to the QEP.
Student Learning Outcome (SLO): An idea, concept, method, etc. that the student is expected to learn upon successful completion of a class.
Rubric: A measurement or grading device used by individual instructors to assess the achievement of a student learning outcome (through a test, essay, project, etc.).
I’ve become a big fan of grading within Turnitin, so much so that I sometimes forget its additional purpose as a plagiarism prevention tool. The grading is convenient since I don’t have to lug around piles of papers and fast because rubrics can be built into the grading tool. These features are also useful from an assessment standpoint.
Because I’ve been using Turnitin for a few semesters now, I have access to all these graded student papers from past classes. At a glance, I can see my grade distribution for a particular assignment, and I can look through the rubrics to quickly determine which skills my students mastered for that particular paper.
In essence, I have a gold mine of information for self-assessment. By skimming through old papers, I can see what skills students needed more instruction in and which skills they mastered during that unit. I can track the progress of individual students throughout the semester, and I can look at how the class as a whole improved. An analysis of rubric scores can provide numeric data if I’m so inclined.
If you haven’t used Turnitin or the grading features in Turnitin, this three minute video gives a good introduction to the program: https://turnitin.com/static/training/instructor_grademark_overview.php
On September 7, The Chronicle of Higher Education published the article “Could Professors’ Dependence on Turnitin Lead to More Plagiarism?” On September 9, Inside Higher Ed published “Plagiarism Betrayal?” a more in-depth look at the same subject: Turnitin’s role in the fight against plagiarism. At first glance, these are troubling articles. “Plagiarism Betrayal?” is full of bellicose diction; The Chronicle’s piece uses a shoplifting metaphor. They both grapple with whether or not Turnitin’s parent company is betraying teachers by providing a confidential service that allows students to check their papers for plagiarism.
What’s missing from these articles? An appropriate emphasis on the collaborative process that teaching should be.
Turnitin can help students and teachers check for correct source usage, but it is only one part of the equation. I teach my students what plagiarism is and techniques to help them research and compose using correctly attributed sources. We practice summary, paraphrase, and quotation skills. We talk about the difference between original ideas and cited material. Turnitin becomes part of the class process. Students upload a draft; we talk about their problems with correct citation, quotation, etc. Students upload a final paper; I look at the Turnitin originality report, but I also know that I have to do due diligence if I suspect a problem not being picked up by Turnitin. I don’t use Turnitin solely for the plagiarism prevention, and I don’t expect the program to be my only check on whether or not students are ethically using sources.
Both articles mention the controversy surrounding Writecheck, the confidential student service mentioned above, but I hope that my students wouldn’t feel the need to pay for such a service. I always provide a draft Turnitin assignment for students to use. It is a space without penalty; it is a space for conversation if the student discovers a problem with source usage. I know that not all cases of plagiarism can be prevented, but a process-based approach to classroom writing takes away some of the incentive to engage in unethical writing practices.
The other thing these articles neglect is the functionality of Turnitin outside of plagiarism prevention. My students do online peer review through the program, and I grade online through Turnitin. Instead of focusing on how the program can “catch” undesirable writing behaviors, I focus on how it allows for peer collaboration, student-teacher dialogue, and transparent grading.
Because of a multi-pronged integration of Turnitin into my classroom, the concerns voiced in The Chronicle and Inside Higher Ed aren’t pressing to me. Instead of criticizing Turnitin, both articles seem to be calling for an active approach to teaching. No software will do the work for you. Turnitin is one tool; it won’t build the house by itself.
Sep 192011
Collaborative learning is usually interpreted as edspeak for working in small groups outside of class to accomplish a project of some significance. These types of exercises require that instructors assign groups, determine how to grade the group if members contribute unevenly, and commit significant time to a single project. However, collaborative learning exercises can be efficiently employed as in-class exercises, even in large lecture sections, with the help of clickers. One easy way to incorporate collaborative learning is to use the Think-Pair-Share technique (Barkley, E.F. et al. Collaborative Learning Techniques. San Francisco, CA: Wiley, 2005). With Think-Pair-Share, the instructor poses a conceptual question and asks students to think about possible answers. After a minute or so of thinking, students are asked to discuss with nearby classmates why they selected the answer they did. This technique acts as a catalyst for in-class discussions. Discussing a new concept with others gives the student a chance to try out his/her understanding of the new concept and receive feedback. It also provides an opportunity for a struggling student to hear an explanation of the concept from a peer who may be better able to communicate with the student (a foundation of peer-led instruction). As with any formative assessment technique, the instructor should set up the exercise by explaining what will happen, the purpose of the exercise, and what students are expected to do.
Adding clickers can extend the advantages of a Think-Pair-Share approach. I’ll take you through the process I use in class:
1) Ask a fairly difficult conceptual clicker question, possibly one that exploits a known misconception of the topic. Ideally you would like to have no more than 30-40 percent of students get the right answer. This technique also works with qualitative questions by forcing the students to defend their position. Clickers provide an added benefit because they require that students commit to an answer or position.
2) After students respond to the question, you can choose to either show or hide the initial results. I prefer to hide the results so that students won’t assume that the answer with the highest total must be correct.
3) Ask the students to discuss with the four or five people nearest them which answers they chose and why, in an attempt to convince others that their answer is correct. I usually allow 2-3 minutes for discussion and will travel around the room to check in on various clusters of students. In my experience, the first time you do this the students will need a bit of coaxing to actually talk with each other.
4) Re-poll the question. After polling has finished, the clicker system will let you display a comparison between the answers from both attempts. In theory, students will coalesce around the correct answer. This also gives them feedback on how the class did as a whole. Re-polling also serves to calm the class back down as they await the results of the new poll.
5) You can now ask a similar concept question on a quiz or exam later in the course.
Assessment Analysis: By building this technique into your teaching, you will have completed a formative (in-class, ungraded) assessment and a summative (quiz/exam) assessment, and you’ll have statistics that document initial understanding, peer-led instruction improvement (always very high and probably not very useful), and actual retention of the concept. Coupled with a pre-assessment of the students’ understanding prior to instruction you will have a better understanding of the effectiveness of your instructional methods as well as plenty of data to report for your T&P assessment matrix.
If anyone has implemented this technique in the classroom or has any questions, post a comment about your experience.
Sep 152011
One of my jobs here in the College of Arts and Sciences is to serve as the iTunes U liaison, which means that I tell people about iTunes U and help coordinate the recording and upload of lectures and performances. We realized early on that it wasn’t possible to get a videographer for every event, especially when we sometimes learn about the event hours before it is to take place. Tegrity is already in the classroom and many faculty already know how to use it. Multiple users can also record simultaneously in rooms across campus with no difficulties. We have created a Tegrity account just for iTunes U recordings. If you’d like the user name and password, contact me. Please know, however, that you’ll need to get the iTunes U release form signed by the presenter, as well as provide additional information on the iTunes U website before the recording can be uploaded to iTunes U. If you would like to learn how to use Tegrity, contact the Faculty Resource Center. If you have questions about implementing Tegrity in your class, contact one of our eTech Tegrity Mentors, John Giggie in History, David Brommer in Geography, or Lucy Curzon in Art and Art History.
Sep 122011
This is my second year working as a mentor for The College of Arts and Sciences’ Office of Educational Technology (eTech) and I am thrilled to be here again for the 2011-12 academic year! My specialty is mentoring A&S faculty in the use of Tegrity, which is a lecture capture system currently being used across the UA campus as a teaching and learning device. For those of you who are unfamiliar with this tool, Tegrity — in a nutshell — records my voice and all of of my visuals (usually embedded into a Powerpoint presentation, but it also captures streaming video, websites, and the odd Sympodium sketch) running through the classroom computer. This is to say, while I’m lecturing, showing images, holding classroom discussion, etc. everything is being recorded and, at the end of class, is uploaded to the Tegrity server, which students can then access through the class eLearning shell.
The favorable response that Tegrity garnered in my own classes when I first started using it four years ago was enough to make me reconsider my own teaching and the ways in which technology might improve the way I conduct classes AND the way that my students absorb and/or interact with the subject matter at hand. All of a sudden, I realized that scores on student tests were going up, student confidence was improving, and — importantly — students weren’t focusing all of their attention on trying to write down each and every word I said…without really grasping what the words meant or the context in which they were being used. Class time became much more productive! Importantly, I was becoming much more cognizant about HOW I was teaching rather than merely focusing on WHAT I was teaching.
As a Tegrity mentor, I would love to mentor faculty in how to use the system at all levels of interaction. From beginner users who would like to learn the “dos and don’ts” to those faculty who would like to introduce student recordings and other collaborative activities or integrate Tegrity data into an assessment plan.
Aug 292011
For years I’ve known that technology can facilitate my research, giving me access to out-of-print archival texts and up-to-the-minute journal articles, streamlining source management and citation, and offering invaluable search tools. Recently I’ve learned that technology can also support my teaching. In the grand scheme of things, I’m more Luddite than technophile, and readily eschew new electronic gewgaws and software suites that seem like they’re going to cost me more effort than they’ll save. But Turnitin has shown itself, time and again, to make me more efficient and effective. Writing assignments figure centrally in my courses, and so my students are often engaged in producing paper drafts, participating in peer review, and reworking their prose. Peer work can present particular challenges. What can be done for students who miss in-class peer exercises? How can I oversee the paper exchange process for out-of-class reviews? Turnitin allows students to complete the collaborative work of the writing workshop–activities like peer review exercises–online. For students, this makes Turnitin not software that aims to “catch” them but rather a tool to help them improve their writing and insure that sources are being used correctly. For instructors, online course activities and assessments assist in producing the evaluations of student learning that are a key component of the learner-centered college.
Aug 262011
Summer is officially over, and it’s time to get enthusiastic about the fall semester. Personally, my back-to-school catalyst is shopping for school supplies. In late summer, I ardently buy a bountiful bouquet of #2 pencils for my desk. It might be old school, but for me, there’s inspiration in those freshly sharpened, bright yellow pencils. However, to rocket me out of the Stone Age (or the 1980′s), I try to incorporate technology in my classes. This is especially important to help students relate to the course content. For current college students, technology is their #2 pencil. Did you know that the class of 2015 has never known a world without the Internet? (See Beloit College’s Mindset List for more doozies like that.) In their eyes, the more technology, the better. Demonstration videos – yes, please! Poll the audience – super! Lecture playback – excellent! 24-7 access to course assignments, handouts, and grades – of course! For professors, the easier the technology, the better. With all that we have to do, from preparing lectures, to conducting research, mentoring students, serving on committees, and writing publications, we do not have hours to wrangle technology. However, we do care that students can relate to the material that we passionately study. Luckily, the Faculty Resource Center and eTech in the College of Arts and Sciences have our backs. They’ve got all of the new tools and can help us deploy them easily and quickly. As an eTech mentor, I have committed my time to helping colleagues in this quest. I can give you tips and tricks for using technology in the classroom in a way that will make your classes more relatable, educational, and enjoyable for students, and in a time-sensitive way for you. Please email if I can help.
The first time I worked as a TA in a Biochemistry class, the professor was covering signal transduction. She was an expert in the field so I expected great things. This video is a pretty close approximation to my experience (Caution-do not operate heavy machinery while watching this video). Although you wouldn’t know it from the video, signal transduction is a field of biochemistry that deals with understanding how living systems respond to external stimuli. By themselves, these pathways demonstrate the incredible complexity of living systems and are responsible for many biochemical processes that you are familiar with, including vision, smell, the flight-or-fight response, blood sugar regulation, the appetite stimulatory effect of THC, and Viagra. Ok, so you may not be familiar with all of these effects, but clearly signal transduction plays a central role in our lives and could make for a lively discussion. The icing on this boredom cake is that the professor had probably spent a good portion of her life researching this very topic and actually has a passion for it. I have found that transferring the passion we feel for our favorite subjects requires an interactive dialog with the students. We have to get them to think about the questions that originally sparked our interest in the subject. One of the most effective ways I have found to accomplish this is through the use of clickers. Over the 2011-2012 academic year, I will be blogging on some of the best and not-so-best practices for the use of clickers in the classroom, and I look forward to hearing back from you about your clicker experiences.
Recent Comments