Reviewing models

All models generated by exhaustive or evolutionary searches are automatically saved, with a name that specifies the search criteria that was used. The details of each saved model are listed in the View Models screen. Manually designed models can also be seen here, if they have been saved manually.

Sometimes a search result may generate more than one model, because more than one model performs equally well on the selected performance criteria, such as Accuracy. In this situation each saved model with the same level of performance is given a consecutive number at the end of its saved name.

Multiple models can be evaluated using secondary and tertiary evaluation criteria, such as

  • Lift – being how well the model predicts the outcome relative to chance. A higher lift value signifies a better performance relative to chance
  • Simplicity – being how few attributes are used in the model, relative to the number available in the design menu. Fewer is better for two reasons (a) Simple models will have wider applicability across cases that exhibit the range of all possible combinations of attributes, (b) Simpler models will be easier to implement in real life.

If there is still more than one competing model remaining then the Quine-McCluskeymethod of minimization can help us find the simplest formulation among these. If there are not too many (e.g. half a dozen cases) remaining then this can be done manually. Here is a simple example. The table lists a number of models, each of which represents a group of cases with the model features

QMC

  1.  There is only one difference between model 1 and 2 (Attribute B), which seems to make not difference to the presence of the outcome. So attribute B can be removed from these models
  2. There is only one difference between model 1 and 3 (attribute A), so by the same logic attribute A can be removed from these models model
  3. There is only one difference between model 1 and 4 (attribute C), so by the same logic, attribute C can be removed from these models

This leaves us with one common set of attributes that apply to all 5 models: D=1 plus E=0 plus F = 1 is associated with outcome = 1

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s