Valencia College and Motivate Lab
Boosting student performance in math
collaboration: Motivate lab and Valencia college
Advancing learning mindsets in entry level math courses
Motivate Lab conducted one of its largest randomized-controlled trials of learning mindset interventions in partnership with Valencia College, a community college system in Florida. The project was aimed at addressing three primary questions:
Do the growth mindset and utility value interventions boost student performance and persistence in entry level math courses?
What student, instructor, and classroom factors influence the effects of the growth mindset and utility value interventions on performance and persistence in entry level math courses?
Do the growth mindset and utility-value interventions boost students’ longer-term college-level persistence (e.g., enrollment in a four-year program)?
background
Although Valencia had been committed to boosting outcomes for their most under prepared students in remedial and front-door mathematics for over a decade, with involvement in local and national projects such as Achieving the Dream (2005-09) and Statway™ (2012-13), dropout rates (40%) have remained only slightly better than the national average (48%). In Fall 2013, Motivate Lab and leaders at Valencia partnered to begin a series of pilot studies focused on collecting data on student attitudes toward developmental math. We transitioned to initial pilot testing of the growth mindset and utility-value interventions in Spring 2014. This initial pilot study revealed that the utility-intervention boosted pass rates in front-door math courses for low-achieving males (Kosovich et al., 2018). After the awarding of a three-year, $1.5 million NSF EHR Core grant in 2015, we expanded the project into a large-scale randomized controlled trial.
the study
Students in entry level math courses were randomly assigned to a growth mindset, utility-value, or control condition. Students in all conditions were asked to complete four activities over the course of a semester: a pre-survey, two intervention activities, and a post-survey. In the surveys, students answered questions measuring various aspects of their motivation, beliefs about learning, and other psychological indicators. During the intervention activities, students in the growth mindset condition were presented with information about neuroplasticity and how our brains continue to develop through practice/effort (e.g., Blackwell et al., 2007). Students in the utility-value condition read quotes from other students about how math is useful in their lives, and were then prompted to write a short essay on how math is useful to their own everyday lives or future goals. Students in the control condition were asked to summarize a math concept they were currently learning.
Over the course of five years, the growth mindset and utility value interventions were tested in three entry level math courses: Intermediate Algebra, Introduction to Statistical Reasoning, and College Math. As of Fall 2018, after 11 semesters of data collection, 13,031 students had participated in the intervention activities across 434 CRNs, with 57 instructors taking part in the project. In addition to intervention activities, we collected data in 39 student focus groups and 28 faculty focus groups. The final semester of data collection ended in December 2018. To date, over 10,500 student essays have been thematically coded by researchers.
Our Findings
Current findings are promising, both in terms of intervention effectiveness and for the potential to inform instructional practices in front-door mathematics.
Intervention Effects*
Analyses of the Spring 2017 semester (671 students) indicate that:
Underrepresented minority students who were randomly assigned to the utility-value condition were 17% more likely to pass their class than underrepresented minority students assigned to the control condition
Students who were randomly assigned to the control condition experienced a decrease in their sense of relevance for the course from pre-intervention to post-intervention, whereas there was no change in relevance across these time points for students in the utility-value condition.
Analyses across three semesters (Spring 2017, Fall 2017, Spring 2018; 2,611 students) indicate that:
Students who were randomly assigned to the utility-value condition were 41% more likely to pass their course compared to students assigned to the control condition.
Students who were randomly assigned to the utility-value condition were 30% less likely to withdraw from the course than students randomly assigned to the control condition.
Continuing generation students who were randomly assigned to the utility-value condition displayed 6% lower course withdraw rates than those randomly assigned to the control condition
*All intervention impact analyses account for student demographics (e.g., prior math achievement, racial/ethnic group membership) and course characteristics (e.g., instructor, modality).
Qualitative Findings
In both the growth mindset and utility value conditions we coded students’ written responses for indicators that they internalized the intervention message. In the growth mindset condition, we coded student essays for endorsement of growth mindset beliefs and strategy use. In the utility-value condition, we coded student essays for how personal and specific students’ connections were between course content and their lives. We also coded for the type of connections students made.
Our coding analysis of the growth mindset condition reveal that:
Students who demonstrated belief in both growth mindset and the importance of strategy use received course grades that were almost one-third of a letter grade higher than those randomly assigned to the control condition (Figure 4).
Our coding analysis of the utility value condition reveal that:
Students who wrote about how their current math course content connected to their lives were more than twice as likely to pass the course than students who made connections to more general and basic math, such as addition and subtraction.
Students who clearly explained how they would use a certain math topic in their daily lives performed more than a third of a GPA point higher than students randomly assigned to the control condition.
Students asked to connect math to their daily lives most often wrote about situations involving money, such as budgeting to pay bills (i.e., 475 students out of 1,318 students, or 36%).
All of these effects remained significant after controlling for important variables such as prior academic achievement and the word count of student essays.
In addition to the analyses presented above, Motivate Lab is continuing to examine students’ longitudinal outcomes, such as overall Valencia GPA, major path, 4-year college matriculation, and career trajectories.