ga(‘create’, ‘UA-91159576-1’, ‘auto’);
EvalC3 has similarities and differences with two other methods of analysis:
- Qualitative Comparative Analysis (QCA)
- Decision Tree learning, as available in software packages like Rapid Miner
- All three can work with binary data, which is more widely available than numerical data
- All three analyse relationships between cases in terms of how their sets of attributes overlap, or not
- The results of all three analyses can be compared using some common performance measures
- Their findings can all be interpreted within the same view of causality, known as multiple conjunctural causation
- Missing data: EvalC3 is tolerant of missing data points, more so than QCA. Decision Tree software in packages like Rapid Miner also seem to be more tolerant than QCA, and it is currently more transparent than EvalC3 in the way it deals with missing data.
- Minimisation / Search: (i.e. finding a configuration that fits many cases)
- Exhaustive search is used by QCA software and EvalC3. When used to find important single attributes this is both quick and effective. When used to find combinations of multiple attributes this can be very slow, though still very effective.
- The Quine-McCluskey algorithm is used by QCA to reduce many configurations to the smallest possible number that have the same core elements. While it can be used on large data sets with many attributes it does not seem to be good at minimising configurations that are significantly different from each other (i.e. more than one attribute different).
- Decision Tree models can be generated using EvalC3 and Rapid Miner software to produce very readable results. However Decision Tree algorithms can be prone to over-fitting i.e. prioritizing accuracy over generalisability.
- A genetic algorithm is also built into EvalC3. This is efficient when dealing with large data sets and many attributes, but it may require more than one application to find the best solution.
- In summary all algorithms have their strengths and weaknesses. Ideally we should test the results found by using one algorithm by also using an alternative algorithm.
- Performance measures: EvalC3 uses provides multiple measures of model performance, which will be suitable in different contexts. QCA focuses on two (consistency and coverage) and of these it then seems to privilege consistency. Rapid Miner has a similar range to EvalC3 but fails to use QCA measures like necessity and sufficiency, which EvalC3 does use
- Visualisation of results: Decision Trees are the most user friendly visualisation. Venn diagrams as sometimes used by QCA require more familiarisation/explanation. EvalC3 uses three methods: Decision Tree diagrams, plus a combination of a Design menu and a Confusion Matrix (truth table) to display the results.
- Manipulation of results: EvalC3 includes the capacity to manually configure prediction models, and to tweak models developed by its two algorithms. These options are not available in Rapid Miner and QCA software
- Sensitivity / contribution analysis: Prediction models developed by EvalC3 can be manually adjusted to identify which attribute in the model contributes most to its overall performance. QCA can do a similar form of analysis (known as INUS analysis) but not with the same degree of precision.
- Case selection: EvalC3 has a systematic process for identifying individual cases most suitable to follow up within-case analysis. This option are not available in Rapid Miner and QCA software
A future version will also…
- Be able to optimise the choice of specific attributes to include in analysis when there are few cases relative to the number of attributes that could be relevant. See more about the next version