To help ensure students’ career success after graduation, Colorado State University’s School of Social Work, along with development partners on campus, is breaking ground with a new data reporting system and sharing implementation strategies with other institutions.
For accreditation, the school reports data which includes specific social work student ‘competencies,’ such as ethics, critical thinking, and research, which are directly tied to readiness for professional social work practice.
By allowing students, as well as administrators and instructors, to access this data in real-time, the school aims to encourage students to focus on improving those competencies—not just grades—for their professional careers as social workers.
The new system is based on using the Outcomes tool in Canvas, CSU’s online learning management system for the administration of courses.
Collaboration key to system development
This fall, the school will use the new system to track performance data for all graduate-level courses in both on-campus and distance programs.
The development team includes Melody Brake, the Canvas central administrator and manager of learning systems at CSU, Aaron Darnell, the Canvas administrator and assistant director of IT for the College of Health and Human Sciences, and Amy Martonis, the director of master’s and distance programs in the School of Social Work.
Together they worked purposefully to set up Outcomes so the app works for CSU as a whole, yet is run by college subaccount for specific programs. As a result, it is program-driven to fit the needs and goals of department administrators, instructors, and students.
The original idea came from a professional connection between Audrey Shillington, CSU’s director of the School of Social Work, and Dale Fitch, the School of Social Work director at the University of Missouri, Columbia.
“Dr. Fitch gave us a demonstration of how he was using Canvas Outcomes to grade electronic portfolios of students’ capstone projects and shared materials that helped us conceptualize this tool for our setting,” Martonis said.
“We still considered several different options for collecting and analyzing our data and ultimately decided to use this tool because it is integrated with our learning platform and accessible to all master’s program administrators, faculty, and students in real-time,” Martonis said.
Customizing for unique program requirements
For CSU’s School of Social Work, the Outcomes tool in Canvas now provides unique reports depending on the needs of the viewer. Department administrators can use data across student progression through programs. Faculty can see data from an instructional perspective within a specific course. Students can assess their own competency and see areas for continued growth.
Yet tracking those social work competencies presented a unique challenge because of the combination of classroom and field education [internships] essential to the programs. “We have nine competencies and under each one are a series of practice behaviors that we wanted to measure at a granular level,” Martonis said.
During the development process, a separate system was used to test various models for display, naming, and structure in Canvas. “Any progress we made in testing would frequently need to be reapplied,” said Darnell. “To facilitate this need, I developed a tool to import the practice behaviors in bulk.”
Once practice behaviors were successfully linked to competencies in Outcomes, the system reported data for both classroom and field instructors to assess students in terms of performance within the context of a practicing social worker.
Dovetailing data with social work curriculum design
Within the School of Social Work, developing and implementing the Canvas Outcomes tool also aligned with a complete curriculum update. “It shifted our focus to the bigger picture of how our students develop their professional competencies across the curriculum,” Martonis said.
“We mapped the system intentionally to show when each competency is first introduced for students, and when we expect them to demonstrate mastery,” Martonis said.
As a result, Outcomes is positioned to help identify gaps in the curriculum or gaps in any specific competencies that students in the same cohort group may encounter.
“If we see patterns in the data, it will point us to areas for very specific curricular enhancements,” said Martonis. “It also has potential to provide significantly more information than aggregate competency achievement.”
The team plans to monitor the Canvas Outcomes tool closely and apply further development over time to ensure the graduate program will continue creating a workforce of social workers who are well-prepared for their careers.
Sharing result with other schools and other departments at CSU
In addition to creating the data reporting system for CSU, the development team has been fielding ‘how’d you do it?’ inquiries from other institutions.
So far this year, the team of CSU collaborators have met with fellow administrators from Carnegie Mellon and University of Minnesota, with more inquiries pending, to offer advice and answer questions so other institutions may follow CSU’s lead.
The team shares detailed insights gained through the development process in the School of Social Work, such as setting up the system so reporting is managed at the college level, while administration is managed by each individual department or program.
“Organization and naming were important, so faculty would be able to distinguish their desired competencies from those belonging to another department,” Darnell said of the eight academic units in the College of Health and Human Sciences.
Furthermore, Outcomes can be set up by individual instructors. “Instructors have their own way of grading and love their own rubrics,” Brake said. “They can add Outcomes and set it to not track scoring, so instructors can grade as they wish, plus mark student proficiency while grading, with little extra effort.”
As a result of the long process of working through the details to create the system, all social work graduate program faculty will use Canvas Outcomes in both on-campus and distance courses this fall, and it is available for other departments at CSU to customize for their programs.
“Now once a program agrees on their standard set of competencies, metrics, and measurements,” said Brake, “the Outcomes tool in Canvas can be used in program courses, on select assignments, by their instructors. The resulting layers of data will benefit program administrators, instructors, and students.”
The School of Social Work is part of CSU’s College of Health and Human Sciences.
