This project utilizes the Feedback Survey feature in Moodle for students to assess their levels of confidence in various aspects of the Evidence-Based Practice/Process (EBP) (including objectives contained within the Accreditation Council for Occupational Therapy Education Standards). An accompanying One Sheet assignment challenges students to address the results of the self-assessment. Through the use of a pre-/post-test design, students level of confidence is assessed before and after completion of the One Sheet assignment.
I selected this project because of the timeframe in which the Tech Fellows presentations were scheduled to occur. Though I have other courses that would also benefit from additional use of instructional technology methods, a project that was focused on the EBP courses was more feasible, most notably as it related to implementation.
Because a large portion of the EBP series process involves student group projects, assessment of individual learning, particularly in the application of EBP processes, is challenging. Although the group processes address clinically relevant content that requires an exploration of available evidence, the skills that are needed to complete the EBP process are varied in level of complexity. Identifying individual strengths within the group at times results in student performance of tasks that are aligned with areas of strength. This may facilitate a more successful group process/product, but this approach is not optimal for individual skill building/individual learning. This project utilizes each of the course objectives from EBP I and applicable objectives from EBP II which are based upon the ACOTE Standards. For the purposes of this project, these objectives serve as the primary items for each student to individually self-assess their level of confidence. Some of the objectives are knowledge-based, while other objectives require active demonstration of skill (for example, the ability to articulate an EBP concept).
Because the Master of Occupational Therapy program curriculum sequence is based upon the concepts of skill acquisition, development, and active learning, this project is also the foundational underpinning, both for the assessment components and for the process of creating the One Sheet.
This project was developed and implemented as follows:
- Consultation with Lauren Panton regarding project concept/idea occurred.
- Confirmation of concept from Lauren Panton was received during this consultation.
- An individual exploration of the Feedback feature in Moodle was undertaken to determine applicability to this project.
- OTH612/EBP I, applicable OTH628/EBP II, and ACOTE Standards were collected.
- Each objective was formulated into the question beginning with “How confident are you in …….” (Actual examples will be provided).
- A total of 26 questions were loaded into the Feedback feature in my Sandbox.
- Consultation with Becky Borello occurred in order to trial the administration of the Feedback survey and to determine the reporting features associated with Feedback.
- A workaround was identified in order to achieve numerical data in Excel for purposes of analysis (utilization of the Find and Replace function).
- Students completed the Feedback survey, having been instructed to complete this survey outside of class, individually, and to base their responses in consideration of their own reflections and based upon any feedback they’ve received from any professor since their admission into the MOT program.
- All data was converted to a numerical score in an Excel file.
- Student averages were calculated.
- Individual scores were examined for the associated content area.
- Once all students had completed the Feedback survey, the students were randomly paired with a peer for the One Sheet assignment.
- Students received instruction on the One Sheet assignment while in class.
- A rubric was provided (Actual example will be provided) to facilitate the content level of the One Sheet.
- Students were asked to identify their lowest scored (based upon their self-assessment in the Feedback survey) items/content areas.
- Students received their responses via individualized email from the instructor.
- Students prepared the One Sheet with their peer which contained two items/content areas.
- Students had one week to create and submit their One Sheet.
- Submission deadline was set for an hour prior to the scheduled class.
- Students completed the Feedback post-survey while in class.
- All data was converted to a numerical score in an Excel file.
- Pre/post data was analyzed for the presence of change in scores and for changes in the individual average score (pre to post).
This project contained formal assessment. Because of the pre and post design and through utilization of a Likert scale to assess self-confidence, scores could be analyzed by individual item or by content area and could further be assessed in terms of change over time.
This project provides both formative and summative assessments specifically related to students’ level of confidence in specific evidence-based practice/process skills. Students are encouraged to reflect upon their knowledge-base and EBP skills and are also able to compare their change in levels of confidence.
This project also gives the instructor valuable information that may impact the information that is presented within the first course within the EBP series, how learning is assessed and what additional assignments might be used to facilitate active, self-directed learning.
Students also benefited from this project, in that 35 out of 39 students demonstrated an overall increase in their level of confidence upon completion of the pre/post Feedback Survey and the One Sheet assignment.
Reflections and Next Steps
The concept of using a survey to assess students’ level of confidence worked well. The ability to compare pre/post scores by individual item, by content area, and overall average contributes to the usefulness of this project. Utilization of the One Sheet, as well as the pairing of students, also contributed to the usefulness of this project.
Moving forward, it would likely be beneficial to utilize this survey at the very beginning of EBP I to get a baseline on students’ level of confidence. This might also result in changing the emphasis on various content areas/objectives.
Pairing the students on an assignment that requires selection and discussion of content areas, creativity, examination of additional resources, and development of individual, measurable goals all encourage self-reflection, self-directed learning, and accountability.
I would likely utilize a different avenue within Moodle to administer the survey (such as Questionnaire) because of the limitations of the Feedback option on students’ ability to retrieve their submitted ratings. Otherwise, I believe that the overall project was a success and would definitely utilize the various components of this project again.
The additional points I’ve learned…..again that there are many ways to utilize technology to enhance teaching/learning opportunities. Personally, I believe that my own growth (and level of confidence!) related to the use of technology has been noteworthy. The anxiety that initially presented itself in contemplating the use of the various forms of technology has been remediated with a growing knowledge base, an expanding skill set, and a knowing that persistence and patience (primarily with myself) will lead to great opportunities as an instructor and as a life-long learner.
Several additional notes: I want to express my thanks to Chatham University for providing such a wonderfully generous program. Without the time and the actual technology ‘equipment’ provided, it would be quite challenging to make the gains that I’ve experienced. The ability to interact with colleagues, my fellow Tech Fellows, and my Tech Fellow buddy regarding specific application of technology to the classroom has been very helpful as well.
Beyond that, a point that I have learned numerous times over since I joined the Chatham Community….that Lauren and Becky are amazing teachers, guides, facilitators, ….completely willing to problem-solve alongside, provide direction, provide numerous tips, strategies, presentations, all to facilitate successful use of technology for instructional design and implementation. Their calm, kind approach, combined with their sense of humor really have encouraged me to keep taking my next steps in exploring technology. This all has been a great triumph for me professionally, but has also been a wonderful personal achievement as well. My deep gratitude to Lauren and Becky.