How to evaluate whether we change intrinsic motivation
We look at various fascinating topics to measure and understand the impact of Oxfam’s projects. These topics range from easily quantifiable topics like people’s “economic well-being” to the more elusive topics of “identity” or “self-confidence”, or the multi-faceted “quality” of a humanitarian response. It is imperative to define all these topics carefully when using them in our – quantitative and qualitative – studies. For the ongoing qualitative evaluation of the PAPAB project, we looked at “intrinsic motivation”. How we’ve defined and studied this, is what you will read in this post. We would love to hear your thoughts too.
Intrinsic motivation is a central topic in the PAPAB project, which aims to increase farmers’ motivation, so that each family on the basis of a shared vision can create their own integrated business plan for a sustainable future. This in itself can be considered remarkable. None of the projects we had evaluated so far pay such specific attention to this crucial driver for behavior change. It is hard to find development projects or evaluations of development projects that focus explicitly on intrinsic motivation. For instance, to date, no systematic reviews have been done by 3ie. Exploring new territory, we therefore consulted literature outside the development realm to understand intrinsic motivation and to learn how we could evaluate whether PAPAB’s intervention was having an impact on farmers’ intrinsic motivation to change their reality.
From the outset, it was clear that a qualitative evaluation method was appropriate. Indeed, the topic had hardly been evaluated before in development projects, and it had only been studied superficially in an earlier quantitative evaluation of the PAPAB project. The PAPAB management team thus saw the need to explore very fundamentally the project’s impact on the intrinsic motivation of the project participants. It is indeed qualitative evaluations that rely on open-ended data collection techniques that can allow for such fundamental exploration.
Then, for the first step of the actual evaluation design, i.e. defining intrinsic motivation, we borrowed conceptual elements from, among others, the fundamental psychological Self-Determination Theory, an applied article on the popular topic of changing employee’s behavior by professor Ben-Hur, and from this qualitative study of the intrinsic motivation of physicians and other health professionals to teach. For our evaluation, we retained the following definition of intrinsic motivation: “The set of factors, coming from inside a person, that determine the choices of that person. Intrinsic motivation itself is influenced by three elements: autonomy (can that person freely make his or her choice to do something?), mastery (does he or she have the adequate capacities?) and connection (to a purpose & other people).” (mostly inspired by Ben-Hur).
A second step was to develop our data collection method. Inspired by the above-mentioned researchers, we conducted in-depth, semi-structured interviews to understand how the PAPAB project impacts people’s intrinsic motivation to change their reality. A first open question asked about the changes people had experienced through the project. These changes could be anything. If the interviewee named a change that related to intrinsic motivation, then questions about how this change had happened, followed. If people mentioned other types of changes (e.g. using a plan to set out and reach their life goals), then interviewers asked about people’s motivation to undertake this change, about how capable they felt to e.g. develop the plan, how other people appreciated this change, etc.
A team of five dedicated interviewers conducted the 30 interviews earlier this month. Coding and analysis of their interview notes is ongoing. While it is too early to draw final conclusions on our methodology or on the content, first analyses are promising and we hope to share more with you on this in a later blog.
In the meanwhile, thanks for reading this post, and please reach out to the author via Ruben.DeWinne@oxfamnovib.nl. Since evaluations of development projects on intrinsic motivation are still quite rare, we appreciate suggestions and/or to discuss evaluation methods.
