Evaluation of Research Experience

Evaluation: NSF Project

A Longitudinal Student Outcomes Evaluation of the SUNY Buffalo State Undergraduate Summer Research Program

At Buffalo State we have undertaken a multi-year effort aimed at developing and field-testing a methodology for measuring student learning and related student outcomes for the Undergraduate Summer Research Program. This effort is being coordinated by Dr. Jill Singer, Director of Buffalo State’s Office of Undergraduate Research, in collaboration with Daniel Weiler Associates, a professional evaluation firm with many years of experience evaluating NSF-supported educational programs. 

The purposes of the evaluation are to:

  1. Obtain a reliable assessment of the program’s impact on participating students that goes beyond the student survey data that has so far been the principle source of impact findings reported in the literature.
  2. Obtain information from faculty mentors on how the program has influenced their teaching and their view of student capabilities.
  3. Provide information to participating students that helps them assess their academic strengths and weaknesses.
  4. Begin the creation of a longitudinal database that can provide impact data on a range of outcomes for different types of students and a variety of academic disciplines.

Working with faculty from a wide range of disciplines (including arts, humanities, and social sciences, as well as STEM faculty), the evaluation has selected 11 student outcomes to be measured. A detailed rubric describes the specific components of interest for each outcome, and faculty mentors assess students on each component, using a five-point scale. Students evaluate their own progress using the same instrument, and meet with the faculty mentor to compare assessments as a way to sharpen their self-knowledge. A range of complementary instruments and procedures rounds out the evaluation. A preliminary version of the methodology was field-tested with a small number of faculty mentors and students during the summer of 2007 and a refined evaluation has been implemented during the summer of 2008 with the participation of all of the program’s faculty and students (17 mentor-student teams).

A spring 2009 CUR Quarterly article describes the approach we are taking to assess our summer research program and how we are measuring outcomes that are related to the experiential goals of the program. A spring 2012 CUR Quarterly article provides data for each outcome and a summary of our findings for the period 2008–2011.

Thank you to the CUR Quarterly and the CUR National Office for giving us permission to access the two articles.

Undergraduate Research

Buffalo State •  1300 Elmwood Ave  •  Twin Rise South Wing Room 510
Buffalo, NY 14222  •  Phone: (716) 878-3093