3. Design model


The default approach to building a predictive model is manual.

  1. Once the Design and Evaluate view is open look at “Design” on the left side. Here you choose what values to place next to each of the attributes that are automatically listed here. The drop-down menu in the Status column provides three options: N/A meaning ignore this attribute; 1 = this attribute is present, 0 = this attribute will be absent.
  2. The default status for each attribute when this view is first opened is N/A.
  3. You also need to choose whether the Outcome is expected to be present or absent when these attributes are as described above, using the same kind of drop down menu in the Status column
  4. This combination of attribute values and the selected outcome then constitute a predictive model
  5. The performance of this model can then be seen immediately in the Confusion Matrix under the heading “Evaluate”, which is explained more below
  6. Click on the Save button (above left) to save details of this model and its performance. You will need to save the model with a name you will recognize later.
  7. If you want to remove all the attributes of a model in one go i.e re-set them all to n/a click on the round “Stop”sign stop signto the right of attribute “Status”

Explore alternative approaches to building a better predictive model

  1. Choose the search strategy. Click on Find New Models, under the heading Explore. There are four choices here.See Search Options on this website for more detailed information about these choicesFind new models B
  2. Choose the performance indicator: the measure that should be maximised by the best models that can be found. There are three groups of these: Overall, Specific and Relative. For more information on these see Evaluate Model. Clue: Start by using the most widely used measure: Accuracy
  3. Set constraints. These can be of three types
    1. Particular attributes in the Design view whose values need to remain fixed. For example,as being present or absent
    2. Specific performance measures other than the one selected as the objective. For example that True Positive Rate =>50%
    3. Specific values for one or more cells in the Confusion Matrix
      1. Try setting False Positive = 0, to find Sufficient but Unnecessary attributes (or configurations of attributes)
      2. Try setting False Negative = 0, to find Necessary but Insufficient attributes (or configurations of attributes)
  4. Implement the search by clicking Okay
    1. If using exhaustive search, watch the process bar in order to assess if the results will be ready within the time available. If not, cancel.

View the results of the search, given the settings above. 

The attributes that have been selected as the best predictors of the outcome (known as “the model”) will appear in the Design area, replacing any previous selection. This model will automatically be saved and the saved name will be visible to the right of the “Save Model” button

The raw results of the prediction model will be shown in the Confusion Matrix in the Evaluate area. See Evaluate Model for more information on how to read the Confusion Matrix.

The performance measures derived from the Confusion Matrix can be seen listed further below the martix. These are used to summarise the performance of the current model in predicting the outcome of interest.

Revise the results

Within the Design & Evaluate worksheet you can tweak the values of the attributes in the new model in order to:

  1. Incrementally improve performance of the model
  2. Identify what attributes in the model contribute most/least to its overall performance. For more on this option see Sensitivity Analysis

Save the results

Save the results of each version of the model that you find to be of value. This will be done automatically, with a unique name, if exhaustive of evolutionary searches have been carried out. But if there has been any manual tweaking the resulting model will then need to be saved manually

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s