5. Compare models

1. The View Models perspective

When you click on “Compare Models” this View Models worksheet will appear.

Compare models worksheet image

Each row represents a specific model. The columns describe two features of each model: (a) the performance of the model according to different measures, (b) the attributes of each model (not yet visible in the above screenshot). For more on how to make use of this data, see Reviewing Models

Any one of these models can be re-loaded by selecting the relevant row, then clicking on Load Model.

2. The Compare Models perspective

This is a new feature that is now accessible via the View Models worksheet.

While in View Models, select the models which are of interest to you, by using the normal Excel drop-down sort feature, as shown below. Then click on the orange “Compare  Models” button on the right.

Select casesThis will taken you to the View Models x Cases worksheet. See the example below.

View cases x models

In this worksheet, the selected models are listed in the columns and all the cases in the data set are listed by rows. Cell values tell which cases is predicted by the column model to have the expected Outcome and found to be a True Positive.

The right hand column counts the number of models that predict a given case. By using the sort function in Excel the cases most frequently predicted by the different models can be sorted and made available as above

The bottom two rows tell us the number of TPs predicted by each model, and what percentage of all Outcomes are uniquely predicted by that model alone. Ideally we would find a model that accurately and uniquely predicted many cases with the expected Outcomes.

However, where more than one model predicts a case as a TP this has practical implications. These cases could be worth selecting for within-case analysis to see where there is most evidence for an underling causal mechanism at work, supporting one model versus another.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s