1. The View Models perspective
When you click on “View Models” this View Models worksheet will appear.
Each row represents a specific model. The columns describe three types of features for each model: (a) model identifiers – name and date, (b) the performance of the model according to different measures, (b) the attributes of each model (not yet visible in the above screenshot). For more on how to make use of this data, see Reviewing Models
Any one of these models can be re-loaded by selecting the relevant row, then clicking on Evaluate Model button at the top left of the worksheet. It will be highlighted in orange as soon as any one model is selected.
2. The Compare Models perspective
This is a new feature that is now accessible via the View Models worksheet.
While in View Models, select two or more models which are of interest to you, by holding down Control as you click, as shown below. Then click on the now highlighted orange “Compare Models” button on the right.
This will take you to the Compare Models worksheet. See the example below.
In this worksheet, the selected models are listed in the columns and all the cases in the dataset are listed in rows. Cell values tell which cases is predicted by the column model to have the expected Outcome and found to be a True Positive.
The right-hand column counts the number of models that predict a given case. By using the sort function in Excel the cases most frequently predicted by the different models can be sorted and made available as above
The bottom two rows tell us the number of TPs predicted by each model, and what percentage of all Outcomes are uniquely predicted by that model alone. Ideally, we would find a model that accurately and uniquely predicted many cases with the expected Outcomes.
However, where more than one model predicts a case as a TP this has practical implications. These cases could be worth selecting for within-case analysis to see where there is most evidence for an underlying causal mechanism at work, supporting one model versus another.
Minimisation
If two prediction models predict the same set of cases, it is worth examining the two models to identify how different they are. If they differ in respect to one attribute only, a QCA type minimisation process may be appropriate. The simpler of the two models could be preferred.