NeuralTools
Re: Variable Impact Analysis

Thread Starter: NathanA   Started: 10-12-2006 11:05 AM   Replies: 2
 Palisade Corporation Forums » Products » NeuralTools » Re: Variable Impact Analysis
 Printable Version    « Previous Thread
  12 Oct 2006, 11:05 AM
NathanA is not online. Last active: 10/12/2006 11:20:22 AM NathanA

Top 10 Posts
Joined on 10-12-2006
Posts 1
Variable Impact Analysis

I am in the process of selection the most accurate neural net based on 25 variables.  I am currently running ~10 variable per trial and switching variable in and out based on the Variable Impact Analysis to converge (hopefully) on a net that is the most accurate.  Doing an exhaustive search is not practical so I'll stop when I find a net that is good enough.

My question is, how is Variable Impact Analysis done?

My second question is, what is the best way to select the best variables to use without taking an eternity to find the answer?

Thanks,
Nathan


  
  13 Oct 2006, 4:04 AM
Mirek is not online. Last active: 4/20/2010 9:12:43 AM Mirek

Top 10 Posts
Joined on 08-30-2005
Posts 75
Re: Variable Impact Analysis

You may want to elaborate on why the process of variable selection using Variable Impact Analysis is becoming so complex.  A simpler approach that occurs to me would be to start with all 25 variables, train a net, throw away x least significant, train a net on the remaining ones, and see if the resulting net still makes good predictions.  Possibly repeat this a few times to reduce the number of variables further.  The objective of reducing the number of independent variables is to get better predictions, so if that doesn't seem to be happening, you may want to use all 25.

 

Regarding how Variable Impact Analysis works, there is some information in the manual.  Let me copy that here, and then append some additional info.

 

MANUAL: The purpose of Variable Impact analysis is to measure the sensitivity of net predictions to changes in independent variables.  This analysis is only done on training data.  As a result of the analysis, every independent variable is assigned a "Relative Variable Impact" value; these are percent values and add to 100%.  The lower the percent value for a given variable, the less that variable affects the predictions.  The results of the analysis can help in the selection of a new set of independent variables, one that will allow more accurate predictions.  For example, a variable with a low impact value can be eliminated in favor of some new variable.  However, one needs to keep in mind that the results of the Impact Analysis are relative to a given net.  The fact that one net "learned" to disregard a given variable makes it likely that another net will also "learn" to disregard it; but then again, another training session with a different type of net might "discover" how to the variable can make a significant contribution to accurate predictions.  In data sets with smaller numbers of cases and/or larger numbers of variables, the differences in the relative impact of the variables between trained nets may be more pronounced.  Also, it is important to remember that these values are "relative".  Suppose that with two independent variables one is assigned 99%, and the other 1%.  This means that the latter is much less important than the former, but does not mean that it is unimportant, particularly if high accuracy of predictions is desired.

 

ADDITIONAL INFO: Only the training data set is included in the analysis.  (If Auto-Testing or Auto-Prediction are used, those cases are not included.  The reason is that they might have numeric values outside the training range, which could make analysis results more unpredictable.)  For a given category independent variable, for every case the analysis steps through all the valid categories for that variable, and measures the change to the predicted value.  (With category prediction there is no numeric predicted value, but there are raw numeric net outputs on which the category prediction is based; those numeric outputs are used by the analysis.)  For a given numeric independent variable, for every case the analysis steps through the range from the minimum to the maximum training value for that variable, measuring the change to the predicted value (or, in the case of category prediction, change to the raw numeric outputs.)


  
  25 Oct 2009, 10:06 PM
Edward de Veres is not online. Last active: 1/16/2010 8:52:32 PM Edward de Veres

Not Ranked
Joined on 10-25-2009
New York
Posts 1
Re: Variable Impact Analysis

Nathan:

I use Variable Analysis Impacts in a different way.  I can't make much sense of a single VAI.  What I do is run a half-dozen Nets and use XL to aggregate the VAIs.  With XL this  is easy.

1 - First put all the Vars into a spreadsheet.  Then I sort all the Vars to be in the same order.  As I number Vars from 01 to 30 this is quick and easy.

2 - Then I average these across the board.

3 - I freeze (copy, paste) the Averages into Values, then sort by size, large on top.

4 - Now I have the 6-value averages of the 30 Vars and I gain a sense of which are the most valuable.

So far, I haven't cut any Vars, but I feel I have a more reliable technique to fine unreliable Vars.

Edward de Veres




Edward de Veres

  
 Page 1 of 1 (3 items)
Palisade Corporation Forums » Products » NeuralTools » Re: Variable Impact Analysis

You can add attachments
You can post new topics
You can reply to topics
You cannot delete your posts
You can edit your posts
You cannot create polls
You can vote in polls
Forum statistics are enabled
Forum is unmoderated

Powered by Community Server :: Forums