Project Description
Substantial reform in science, mathematics, and engineering courses has occurred over the past several decades on both in- and out-of-class experiences and some innovators have made substantial modifications to their courses. Astonishingly, however, in the presence of such a broad investment of time and money, no reliable instruments exist to measure the frequency and quality of students’ cognitive and behavioral engagement or their social engagement both inside and outside of the classroom in relation to the activities associated with a specific course. This project developed instruments within an evaluation framework, termed the Course Social and Cognitive Engagement (CSCE) Framework, that supports engineering faculty development and improvement of their courses by providing empirical data on students’ in- and outside-classroom activities, active participation in learning, and networked social capital as a measure of social and emotional engagement. We examined the decision process of faculty as they implemented evaluation plans in their courses so as to build the CSCE to maximally support future classroom evaluation efforts; this was achieved in-part through front-end evaluation efforts and developing scale measures for several constructs of interest for engineering faculty. The outcomes of this project were: (1) valid and reliable CSCE instruments with modular subscales (that are also valid and reliable); (2) an exploratory framework for how engineering faculty make choices and engage in the measurement process; and (3) a deeper understanding of processes that underlie cognitive and social engagement in an engineering classroom.
Intellectual Merit
This project advances knowledge and understanding about the education of undergraduate engineering students within engineering as well as across other fields by contributing to: 1) adaption and application of methods used in other fields (i.e., sociology, learning sciences, and psychometrics) to STEM education and learning settings; 2) instruments that can be used by evaluators to assess the impacts of engineering education reform efforts (e.g., faculty as evaluators of their own classrooms to understand cognitive and social engagement in their classrooms, or for university leadership to evaluate short-, mid-, and long-term outcomes/impacts of educational reform efforts); 3) empirical evidence to inform the instructional decisions of faculty; and 4) an understanding of how faculty choose to utilize the instruments and corresponding lessons for future adopters and for revisions of the instruments. This study is creative and original in the combined development of instruments and the examination of faculty assessment and measurement approaches and practices using theoretical frameworks of change and adoption.
Broader Impacts
Intellectual infrastructure was enhanced by developing instruments and a framework that can have substantial impact on educational improvement efforts in STEM disciplines. Future researchers could develop a modified instrument that could be used in other college programs and courses. 24 participating faculty across 6 institutions have utilized all of some of the instruments in their courses at least two times, and worked with our team to utilize aspects of the instruments that fit their needs and beliefs about education. Twelve of the participating faculty provided feedback on the development of the instruments, facilitating development of instruments that fit the needs and expectations of stakeholders in engineering education. Hundreds of other engineering faculty will have exposure to the instruments and results through participating faculty presenting formally at conferences in their disciplines and discussing informally the instrument and what they learned about their teaching.