5. Compare models

1. The View Models perspective

When you click on “View Models” this View Models worksheet will appear.

view models 2

 

Each row represents a specific model. The columns describe three types of features for each model: (a) model identifiers – name and date, (b) the performance of the model according to different measures, (b) the attributes of each model (not yet visible in the above screenshot). For more on how to make use of this data, see Reviewing Models

Any one of these models can be re-loaded by selecting the relevant row, then clicking on Load Model.

2. The Compare Models perspective

This is a new feature that is now accessible via the View Models worksheet.

While in View Models, select the models which are of interest to you, by holding down Control as you click, as shown below. Then click on the now highlighted orange “Compare  Models” button on the right.

highlighted

 

This will take you to the Compare Models worksheet. See the example below.

Compare models

 

In this worksheet, the selected models are listed in the columns and all the cases in the dataset are listed in rows. Cell values tell which cases is predicted by the column model to have the expected Outcome and found to be a True Positive.

The right-hand column counts the number of models that predict a given case. By using the sort function in Excel the cases most frequently predicted by the different models can be sorted and made available as above

The bottom two rows tell us the number of TPs predicted by each model, and what percentage of all Outcomes are uniquely predicted by that model alone. Ideally, we would find a model that accurately and uniquely predicted many cases with the expected Outcomes.

However, where more than one model predicts a case as a TP this has practical implications. These cases could be worth selecting for within-case analysis to see where there is most evidence for an underlying causal mechanism at work, supporting one model versus another.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s