1. A typology of uses
EvalC3 can be of use at all stages of a project cycle:
- During project selection:
- To identify what attributes of project proposals are the best predictors of whether a project will be chosen for funding, or not
- To identify how well a project proposal appraisal and screening process is as a predictor of the subsequent success of projects in achieving their objectives
- During project implementation
- When the effectiveness of specific activities are being measured using survey instruments which include both specific (facet) and general (global) measures of satisfaction with service or product delivered. EvalC3 can identify what combination(s) of facets best predict global satisfaction. For example:
- Participants experiences with workshops and training events
- Donors and grantees experiences of their working relationships with each other
- When the effectiveness of specific activities are being measured using survey instruments which include both specific (facet) and general (global) measures of satisfaction with service or product delivered. EvalC3 can identify what combination(s) of facets best predict global satisfaction. For example:
- During a project evaluation
- “Causes of effects” analysis: To identify what combination(s) of project activities (and their contexts) were associated with a significant improvement in beneficiaries lives.
- “Effects of causes” analysis: To identify what combinations of improvements in beneficiaries lives were associated with a specific project activity (or combination of)
- To identify “positive deviants” – cases where success is being achieved despite the fact that failure is the most common outcome. See Postscript note below for details.
- During a review of existing evaluations
- Re-analysing data that was collected, to verify the results. This is often possible with QCA based evaluations because QCA data sets are usually published as annexes to evaluations
- Synthesising the results of multiple evaluations, into prediction rules concerning different types of outcomes
“Loose” Theories of Change
More generally, EvalC3 is suitable for use where a project, or part thereof, has a “loose” Theory of Change. Loose in the sense that while the outcomes have been identified, the activities needed to achieve these may not yet be clear and even less so the specific causal pathways that will be involved.
Loose Theories of Change are more likely to be found in participatory development projects, or projects involving a substantial degree of decentralization, as is often the case with projects covering large geographic area and/or many sectors.
For an extended discussion of loose ToC, see my 2016 paper “Evaluating the impact of flexible development interventions using a ‘loose’ theory of change Reflections on the Australia-Mekong NGO Engagement Platform. ODI Methods Lab Working Paper, March 2016
2. Published papers on the use of EvalC3
So far, these are few in number
RESILIENCE IN IRAQ Impact evaluation of the “Safe access to resilient livelihoods opportunities for vulnerable conflict-affected women in Kirkuk” project. Alexia Pretari and Filippo Artuso, Farah Abdulrazzaq Salih, Kayghan Muhamed Saeed Taher, Mahran Alhaeyk, Sarah Nijholt for Optimum Analysis: data gathering on behalf of Oxfam. Effectiveness Review Series2019/20
SUSTAINABLE WATER AND SANITATION IN SIERRA LEONE Impact evaluation of the ‘Improved WASH Services in WAU and WAR Districts’ project. Jaynie Vonk, Oxfam GB. Effectiveness Review Series 2019/20
Prediction Modelling with Qualitative Comparative Analysis. Hur Hassnain , 2019 YouTube video
WOMEN’S EMPOWERMENT IN LEBANON Impact evaluation of the project ‘Women’s Access to Justice’ in Lebanon. Lombardini, Simone, Hassnain, Hur, Garwood, Rosa, 2019. Oxfam Effectiveness Review Series 2017/18
Learning From The Civil Society Challenge Fund: Predictive Modelling. TripleLine Briefing Paper, Rick Davies September 2015
A pdf version of this page is available here: Example uses _ EvalC3
EvalC3 by Rick Davies is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Based on a work at https://evalc3.net/.