Reviewing models

All models generated by exhaustive or evolutionary searches are automatically saved, with a name that specifies the search criteria that was used. The details of each saved model are listed in the View Models screen. Manually designed models can also be seen here, if they have been saved manually.

Sometimes a search result may generate more than one model, because more than one model performs equally well on the selected performance criteria, such as Accuracy. In this situation each saved model with the same level of performance is given a consecutive number at the end of its saved name.

Multiple models can be evaluated using secondary and tertiary evaluation criteria, such as

  • Lift – being how well the model predicts the outcome relative to chance. A higher lift value signifies a better performance relative to chance
  • Simplicity – being how few attributes are used in the model, relative to the number available in the design menu. Fewer is better for two reasons (a) Simple models will have wider applicability across cases that exhibit the range of all possible combinations of attributes, (b) Simpler models will be easier to implement in real life.

If there is still more than one competing model remaining then the Quine-McCluskeymethod of minimization can help us find the simplest formulation among these. If there are not too many (e.g. half a dozen cases) remaining then this can be done manually. Here is a simple example. The table lists a number of models, each of which represents a group of cases with the model features


  1.  There is only one difference between model 1 and 2 (Attribute B), which seems to make not difference to the presence of the outcome. So attribute B can be removed from these models
  2. There is only one difference between model 1 and 3 (attribute A), so by the same logic attribute A can be removed from these models model
  3. There is only one difference between model 1 and 4 (attribute C), so by the same logic, attribute C can be removed from these models

This leaves us with one common set of attributes that apply to all 5 models: D=1 plus E=0 plus F = 1 is associated with outcome = 1

Caveat: It is possible that two models that predict the outcome and differing only in one attribute are covering slightly different sets of cases. Simplifying the model by the means shown above would come at a come at a cost of its coverage. So check the cases covered by any two such models (via the Compare Models view) before going down this road.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.