This is the point in the work flow when the focus changes from across-case analysis to within-case analysis. This is where case collection strategies and tools become relevant. Before doing any within-case investigations choices need to be made about which case(s) to focus on.
Eval3 now has two sets of tools to use to aid case selection. Associated with these are two analysis strategies, which are discussed further below
Finding modal and extreme cases
When you click on View Cases this is a screen shot of what you will see. The cases are listed row by row. Their attributes are listed column by column, with the outcome column being on the far right (often initially out of sight).
On the left are two columns. The Status column tells you what Confusion matrix cell the case belongs to (True Positive, False Positive, False Negative, True Negative). The rows will be sorted so that all cases appear in the same color coded Status group.
Next to it is the Hamming Distance column. Contents for this column will be generated when you click on the “Calculate Hamming Distance” button above. The smallest % value means that row case is the most similar to all other cases with the same status, .e. it is a modal case. The highest % value means the row case is the least similar to all others in that group, i.e. it is an extreme or outlier case.
The next step is to select cases for subsequent within-case investigations, to identify casual mechanisms that may be at work underlying the associations represented in the predictive model. These types of cases may be useful:
- Modal case within the True Positive group. This is where a causal mechanism needs to be found that might apply to all other cases in this group
- Outlier cases within the True Positive group. Ideally the same mechanism will also be found here
- Modal cases within the False Positive Cases. These are cases with the same (predictive model) attributes but different (absent) outcomes. Here the same causal mechanism might be expected but along with other features that prevent them from working and delivering the outcome
- Modal cases within the False Negative Cases. These are cases with different (predictive model) attributes but the same (present) outcomes. Here the same causal mechanism should not be expected to be found.
See within-case analysis for more information on the kinds of cases selections and analyses that can be made at this stage.