College 1 Assessment
The College 1 Assessment Project exemplifies how consistent faculty engagement with student learning outcomes assessment, coupled with comprehensive support from assessment specialists, can inform, energize, and inspire faculty to make meaningful improvements to pedagogy and curriculum at an impressive scale.
Project background • Methodology • Assessment Support • Impact
10
Colleges
Each college conducts an assessment study that aligns with the college theme
~80
Faculty Participants
College Provosts and faculty collaborate with ALEE Center specialists
~4500
First-Year Students
All incoming freshman students are assessed, creating a comprehensive view of UCSC students’ first quarter experiences
Project Background
First launched in 2018 when the Academic Literacy Curriculum was established, the College 1 Assessment Project emerged out of collaborative efforts between the Council of College Provosts and the ALEE Center. The course learning outcomes include college-level critical reading, thinking, and discussion skills. College 1 (also known as the Core Course) is taken by all first-year students in their first quarter at UCSC. Learning assessment in this course has been an ambitious, large-scale undertaking– the first of its scope at UCSC– involving nearly 80 faculty members and 3,700 first-year students annually.
Methodology
The College 1 Assessment methodology includes two forms of assessment: (1) faculty assessment of students’ skills using detailed criterion-based rubrics; and (2) student self-assessment in an end-of-course survey.
Criterion-based rubrics
Faculty assessment of students’ skills
While the College 1 course has one set of course learning outcomes (CLOs), each of the ten colleges offers a version of the course built around a unique theme reflected in the readings and syllabus. For the assessment, each college has designed an assignment and criterion-based rubric fully integrated within each college’s syllabus. Instructors assess students during the second half of November.
Example of rubric-based results:
“Based on the assessment, the majority (87%) of students demonstrated general/full proficiency in using a range of strategies to understand and analyze texts (CLO 1), but only 64% demonstrated general/full proficiency in drawing explicitly from knowledge gained in class and apply it in novel situations (CLO 8).”
Student Surveys
Students’ self-assessment of skills
In collaboration with the ALEE Center, each college administers a survey at the end of the course to collect students’ reflections on their learning, academic and social experiences in the course, and college-specific assignments.
This course-specific survey provides additional in-depth information about student learning related to each course learning outcome and effective teaching practices.
Example of survey results:
“Based on the survey, the majority (79%) of students reported very good/excellent skills in using a range of strategies to understand and analyze texts (CLO 1), and 72% reported very good/excellent skills in drawing explicitly from knowledge gained in class and apply it in novel situations (CLO 8).”
Assessment Support
Spring Quarter
During Spring quarter, ALEE Center specialists meet with every College Provost (CP) and their Core Course Coordinator (CCC) to review assessment results from the prior year, discuss implications, and make plans for the upcoming fall quarter.
Summer Quarter
During the summer, the CP and CCC complete a Debrief and Planning Report to summarize their results and describe goals and revisions for the upcoming fall. If the CP and CCC would like to revise the assessment (e.g., to align with some changes in the syllabus), ALEE Center specialists offer consulting opportunities throughout the Summer to revise tools.
Fall Quarter
As the course begins in the fall, ALEE Center specialists provide extensive support to College 1 instructors by:
- Setting up assessment materials in Canvas courses,
- Introducing the assessment project at faculty orientation meetings,
- Holding training meetings for each college, where faculty evaluate and discuss student samples),
- Creating and sharing relevant resources and materials (see sample rubric tip sheet),
- Offering one-on-one technical assistance and accommodations as needed,
- Sending out a student survey at the end of the course.
Winter Quarter
After the rubric and survey data are collected at the end of the fall, ALEE Center specialists:
- Extract the data from Canvas
- Merge the assessment data and survey data with institutional data on students’ academic and demographic backgrounds; and
- Analyze the results through an equity-minded lens
Assessment specialists summarize the findings in a rubric study report and a survey report for each of the ten colleges (see sample rubric study report).
Impact
Over the years, the positive impacts of the College 1 Assessment Project on student learning have been diverse and far-reaching. Click below to learn more about the impacts of the College 1 Assessment Project:
Support for students beyond the classroom
Beyond the classroom, the unusually rich and nuanced perspective of the first-year student experience provided by the college-specific survey study reports has energized and inspired faculty to respond to their students’ needs. Based on survey findings, many colleges have made focused efforts to boost student awareness and engagement with campus resources, enhanced specific resources such as Peer Navigators to foster community and strengthen student sense of belonging, and used survey insights to inform special programs such as college plenaries, events, workshops, and hikes, which may play pivotal roles in students’ first-year experiences.
Targeted support for student learning
College 1 assessment has identified specific areas for improvement in students’ skills, allowing College Provosts and Core Course Coordinators to provide targeted support in key areas. Some examples include the following:
- Across all ten colleges, understanding genres (CLO 2) emerged as a recurring area of challenge for students. Several colleges implemented additional learning activities and structured opportunities for students to develop and practice this skill throughout the quarter.
- Colleges have drawn on students’ survey responses to ensure that course materials reflect the diverse backgrounds and experiences of their students and offer meaningful ways for them to connect to academic discourse.
- Some colleges have successfully narrowed equity gaps by implementing new scaffolding activities and carefully revising the wording on assignments for accessibility.
- Still others have invested heavily in collaborating with ALEEC specialists to develop new criterion-based rubrics for other major assignments in the course in order to align faculty expectations and make the hidden curriculum visible.
Increased support for faculty
The College 1 Assessment Project has been essential in advancing College 1 faculty’s awareness and understanding of the course learning outcomes. Every year, a substantial proportion of College 1 instructors are newly hired and have never taught the course. Because of this project, many of these new instructors learn about the course learning outcomes in the pre-Fall faculty orientation, receive individual technical support with their Canvas course, and engage in thoughtful discussions with their more experienced colleagues about reasonable faculty standards for student work. Moreover, the process of reflecting on the prior year’s assessment findings creates an important space for new and continuing faculty to collectively define goals for the upcoming year.
Building a culture of assessment
One of the College 1 assessment project’s most impressive and striking impacts is the emergence of a culture of assessment within the ten colleges. College Provosts and course instructors view this assessment as a useful barometer of student learning and experience, not a fruitless administrative exercise—or worse yet, a threatening intrusion in their classrooms. Every spring, College Provosts and course coordinators meet to review results with ALEEC specialists and brainstorm together what improvements might be made the following year (see sample annual debrief and planning report). Achieving equity in student learning outcomes has become a shared and explicit goal. In recent years, a number of colleges have broadly implemented criterion-based rubrics modeled closely on the rubrics from the College 1 assessment project, with the intention of advancing equity by improving student understanding of assignment expectations and reducing instructor bias.
As the College 1 Assessment Project has expanded and matured over time, it is increasingly evident that this effort advances first-year students’ abilities to engage in university-level discourse, supports faculty to close equity gaps in learning, and offers a valuable snapshot of students’ academic and affective experiences during this important transitional period.