5. Compare models

1. The View Models perspective

When you click on “View Models” this View Models worksheet will appear.

view models 2

 

Each row represents a specific model. The columns describe three types of features for each model: (a) model identifiers – name and date, (b) the performance of the model according to different measures, (b) the attributes of each model (not yet visible in the above screenshot). For more on how to make use of this data, see Reviewing Models

Any one of these models can be re-loaded by selecting the relevant row, then clicking on Load Model.

2. The Compare Models perspective

This is a new feature that is now accessible via the View Models worksheet.

While in View Models, select the models which are of interest to you, by holding down Control as you click, as shown below. Then click on the now highlighted orange “Compare  Models” button on the right.

highlighted

 

This will take you to the Compare Models worksheet. See the example below.

Compare models

 

In this worksheet, the selected models are listed in the columns and all the cases in the dataset are listed in rows. Cell values tell which cases is predicted by the column model to have the expected Outcome and found to be a True Positive.

The right-hand column counts the number of models that predict a given case. By using the sort function in Excel the cases most frequently predicted by the different models can be sorted and made available as above

The bottom two rows tell us the number of TPs predicted by each model, and what percentage of all Outcomes are uniquely predicted by that model alone. Ideally, we would find a model that accurately and uniquely predicted many cases with the expected Outcomes.

However, where more than one model predicts a case as a TP this has practical implications. These cases could be worth selecting for within-case analysis to see where there is most evidence for an underlying causal mechanism at work, supporting one model versus another.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.