Example uses

EvalC3 can be of use at all stages of a project cycle:

  1. During project selection:
    • To identify what attributes of project proposals are the best predictors of whether a project will be chosen for funding, or not
    • To identify how well a project proposal appraisal and screening process is as a predictor of the subsequent success of projects in achieving their objectives
  2. During project implementation
    • When the effectiveness of specific activities are being measured using survey instruments which include both specific (facet) and general (global) measures of satisfaction with service or product delivered. EvalC3 can identify what combination(s) of facets best predict global satisfaction. For example:
      • Participants experiences with workshops and training events
      • Donors and grantees experiences of their working relationships with each other
  3. During a project evaluation
    • “Causes of effects” analysis: To identify what combination(s) of project activities (and their contexts) were associated with a significant improvement in beneficiaries lives.
    • “Effects of causes” analysis: To identify what combinations of improvements in beneficiaries lives were associated with a specific project activity (or combination of)
    • To identify “positive deviants” – cases where success is being achieved despite the fact that failure is the most common outcome. See Postscript note below for details.
  4. During a review of existing evaluations
    • Re-analysing data that was collected, to verify the results. This is often possible with QCA based evaluations because QCA data sets are usually published as annexes to evaluations
    • Synthesising the results of multiple evaluations, into prediction rules concerning different types of outcomes

More generally, EvalC3 is suitable for use where a project, or part thereof, has a “loose” Theory of Change. Loose in the sense that while the outcomes have been identified, the activities needed to achieve these may not yet be clear and even less so the specific causal pathways that will be involved.

Loose Theories of Change are more likely to be found in participatory development projects, or projects involving a substantial degree of decentralization, as is often the case with projects covering large geographic area and/or many sectors.

Postscript on how to find “positive deviants”: Today I have been reading excerpts from “The Power of Positive Deviance: How Unlikely Innovators Solve the World’s Toughest Problems” and have been wondering how “positive deviants” could be found using EvalC3. The solution, when using EvalC3, is relatively simple.

First, develop a predictive model that is good at predicting the absence of the outcome. Usually we are trying to predict its presence.  It may be easiest to do this by testing out combinations of attributes that according to prior knowledge and theory are not conducive to the outcome occurring – especially attributes of this kind that are quite common.

Then focus on the False Positives i.e. those cases where the model attributes predicted absence of the outcome but in practice the outcome was present. These cases qualify, on first glance, as Positive Deviants. They are the cases where it would be well worthwhile doing a within-case investigation in order to find out how they managed to succeed against the odds.

This approach can be tested out using the Krook data set. The absence of quotas for women in parliament is sufficient for low levels of women’s participation in parliament. It predicts 13 of the 14 countries with such low levels. The one exception is Lesotho, where there are no quotas but there are high levels of participation of women in parliament. This is an example of a “positive deviance” case that would be worth investigating

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s