“Where do I even start?”: Mapping Introductory Student Engagement with Course Activities to Successful Content Mastery
The fall semester course BIO143 “The Cell” is a foundational course intended for first-year students in multiple majors—Human Biology, Biochemistry, Cell and Molecular Biology and Exercise Science. The large course enrollment and variable student preparation for the course presents specific challenges for instruction: how do we engage and challenge highly prepared students while supporting and encouraging learners who have neither substantial content background or robust study skills from their pre-university experiences?
In response to the uncertainty of the pandemic, week-to-week (or even daily) changing of instruction modes, the health of students and faculty, BIO143 was moved to a flipped classroom format in the fall of 2020. Dr. Pierette Appasamy and I added topic-specific high-production value short video lecture segments to our content modules, in addition to assigned readings, power point slides, learning objectives, active learning assignments, additional support activities, extension activities, short quizzes…suddenly, students began asking “There is so much here… where do I even start?!” In our goal to support all students at all levels, we’d inadvertently swamped some of our introductory students with a flood of information.
So how do I answer “Where should I even start?” Or identify students that are confused but aren’t brave enough or self-aware enough to identify as such? BIO143 is already challenging for both students and instructor, but add in a pandemic and intermittent virtual instruction— and BIO143 became a course in which all of us were learning to use new technology and struggling to identify where student learning was breaking down. In normal times, face-to-face meetings with students during office hours or supplemental instruction sessions are a critical tool for assessment of student learning and the creation of student support networks. Zoom meetings were a less-than-ideal substitute. Even just gauging the level of confusion on a student’s face became challenging with in-person masking mandates or virtual sessions where Zoom fatigue led students to keep their cameras off.
My solution: fully engage with our learning management system and video recorder/server. Brightspace has the capabilities to determine and report if students have interacted with posted content and for how long, Panopto will allow you to determine which students watched any given video, for how long, and the exact timestamp.
My project idea was simple: identify students who demonstrated learning gains from the beginning of the semester to the end and who interacted with the content in a differential manner (so not students who began as ‘A’ students and stayed there by watching/reading/completing every activity to the fullest; nor students who remained ‘C’ students) and then determine the content that they interacted with and for how long.
Moreover, by tracking student interactions with specific content, I hoped to determine if there are items that could be either highlighted for student review or moved to the “optional” portion of a content module; thus streamlining the content organization and facilitating student navigation of the material.
In the Fall of 2020, our goal was to move as much of our content online as possible in order to support any quarantined students or — in case we fell ill—any faculty member who might have to step in and lead the course. By Fall 2021, I realized that I could leverage this same technology to try to answer “Where should I start?” in a quantifiable way. To generate the data I needed I did the following:
- Panopto: Many of our short videos were hosted on other platforms, so we used a capture program to move all videos to Panopto— thus enabling tracking of student viewing. This was approximately 25 videos.
- Brightspace data: I began working with Lauren and Becky to generate and parse the big data sets that we could get out of Brightspace relating to the course. Here we hit a snag: despite the fact that Brightspace was tracking student clicks on content and time spent with content; upon data retrieval, all of that was lumped into single categories called “content completed” and “total time spent in content”.
This is not fine-grained enough for me to answer “where do I start?” as all I can determine is how many of my 392 content items students interacted with (but not which ones); and the total time per students for all items (for example, 1592 minutes). This is not tremendously helpful: A student could have been reviewing the same set of PowerPoint slides for 1600 minutes during the first week of class from all that I could tell from this dataset. Instead, I needed to determine:
- Which specific content items did students interact with
- When (the actual date timestamp)
- For how long
This type of data should allow me to determine, for example, that students that reviewed slides and watched a particular video prior to the quiz performed better than students that only watched a video.
Thanks to the tireless efforts of Lauren and Becky, I believe that soon we (and by ‘we’ I mean ‘Lauren and Becky’) will be able to use a more advanced data retrieval tool to mine that fine-grained data.
Without the fine-grained dataset, assessment becomes impossible; however, once in hand, I should be able to directly compare timestamp and length of the engagement period of student interactions with specific content modules to student performance on related course assessments (quizzes and exams).
Reflections and Next Steps
This summer, I will reorganize the BIO143 content and course pages to facilitate fine-grained data retrieval and to enhance ease of student navigation of the course page (not always congruent goals). Then, using the dataset from this upcoming fall 2022 student cohort in conjunction with those from the past two years, I should be able to identify which activities successful students engaged with, when, and for how long. This should finally allow me to definitively answer future students when asked “Where do I start”?.