I wanted to find technology that would support students’ writing and perhaps improve two assignments in the Psychometrics course that I teach in the first semester of our PsyD program. The assignments are an accurate paraphrasing activity that did not work very well in the previous semester and a final paper.
The technology I found was a program called NoodleTools. NoodleTools is a program intended to help students take notes, create outlines, and create correct bibliographies in several accepted formats. NoodleTools allows students to share their materials with instructors throughout the note taking and outlining process.
I wanted to find technology that would support students’ writing. Specifically, the Psychometrics (PsyD) course includes a project where students are asked to identify an assessment (interview, test, survey) that they are interested in, research the measurement properties (reliability, validity, sensitivity, norms, etc.) of that assessment, and write a final paper summarizing those measurement properties and expressing a professional opinion about where that assessment can and should ethically be used. In past semesters, I have noticed that students struggle with various aspects of this project including accurate paraphrasing and avoiding unintentional plagiarism, organizing information logically, thinking critically about the research they read, and writing in general. I’ve responded to these difficulties by creating an accurate paraphrasing activity, providing a general outline for the paper, and occasionally requiring a draft of the paper. However, this is still a challenging project in the first semester of doctoral study and the accurate paraphrasing assignment was frustrating for students in the previous year, so it needed to be improved. I hoped that technology would provide a novel way to modify the assignment and also provide another avenue to support student research and writing.
The final paper in the course is supported by the accurate paraphrasing activity I designed and is intended to demonstrate several student learning outcomes from the course:
- find and describe the psychometric properties of test and measures
- apply the concept of reliability to the evaluation test and measures
- apply the concept of validity to the evaluation of test and measures
- articulate the ethical dilemmas faced when selecting tests and measures
- demonstrate ethical decision-making by identifying choices consistent with the ethical guidelines related to assessment
I created a ‘Project’ for the accurate paraphrasing assignment, did an example for students to look at (Millon Behavioral Medicine Diagnostic), and a Dropbox folder (Psychometrics) connected to NoodleTools for students to use to share their projects with me.
NoodleTools focuses on how students take notes (using notecards) and on linking the notes to sources. Students create notecards which can be sorted, stacked, tagged, etc. and are displayed visually.
With the accurate paraphrasing assignment, students were asked to start with a direct quote from a source and then paraphrase it. There is also space to add questions or critical ideas.
When they shared their project with me, I commented on the paraphrasing and answered questions directly on their note card.
I was definitely learning NoodleTools as I went along.
My plan B was to simply modify the Turnitin assignment I had created the previous year.
I assessed my project using a brief anonymous survey of students at the beginning of the semester, before I had completed a presentation on unintentional plagiarism.* Students were asked to rate their agreement or disagreement with a handful of statements (that I made up) about aspects writing papers. I repeated this survey at the end of the semester.
At the beginning of the semester, most students used direct quotes in their notes, felt that they could paraphrase accurately, but did not consider unintentional plagiarism.
At the end of the semester, there was more variability in how much they started with direct quotes, they were more confident in their accurate paraphrasing skills, and they were all considering unintentional plagiarism (a win!).
They also generally thought that NoodleTools was easy to use (yay!) and helped them paraphrase accurately (yay!), which is most likely a function of the assignment they had to complete in NoodleTools.
But they did not find that NoodleTools helped them think critically about the research they were reading, organize their ideas, or write their paper.
Overall, they didn’t think NoodleTools was worth the effort.
Reflections and Next Steps
The focal assignment for this NoodleTools project was modifying the accurate paraphrasing activity I had created previously. NoodleTools worked for that. What it didn’t do – help students organize ideas, think more critically about the research they were reading, create an outline, write their papers – I hadn’t supported. I would like to learn more about NoodleTools, particularly the use of tags, sorting and organizing notecards, and creating outlines so that NoodleTools begins to support those aspects of the final paper. For next year, I will not provide students with a general outline for the final paper and will instead ask them to create their own outlines in NoodleTools. I also plan to add more of a focus on the “My Ideas” section of the notecard so that I can increase the focus on critical thinking about the research students read.