Vensim Help

 

:SENSITIVITY=OFF { OFF | PAYOFF_VALUE{=4.0} |
                 PAYOFF_PERCENT{=10.0} |
                 PARAMETER_PERCENT{=10.0} |
                 ALL_CONSTANTS{=10.0}}

This option allows you to find out how sensitive the payoff is to changes in parameter values.  The default setting is off, which does not invoke the sensitivity calculations. Otherwise, there are four modes of use: PAYOFF_VALUE, PAYOFF_PERCENT, PARAMETER_PERCENT, and ALL_CONSTANTS. A fifth mode, PAYOFF_MCMC, is available as a Markov Chain Monte Carlo option, but is not integrated into the optimization control interface.

The sensitivity analysis done here is not part of the sensitivity analysis discussed earlier in this chapter though they are closely related conceptually.  The terminology, unfortunately, overlaps somewhat and where clarity is required we will refer to optimizer sensitivity analysis.

PAYOFF_VALUE and PAYOFF_PERCENT make sense only at an optimum.  When invoked, they run through the parameter list, and determine how much each individual parameter would have to be changed to decrease the payoff by the desired value or percent.  Positive and negative value changes are calculated separately, since they are not normally symmetric.

PAYOFF_VALUE is useful when the payoff is a true log likelihood or properly weighted sum of squares.  In this case, different changes in the absolute value of the payoff can be used to derive confidence bounds on parameters.  For example, for a sum of properly weight squares using the default PAYOFF_VALUE of 4 (the default)  results in approximately 95% confidence intervals being reported for each parameter.   If you are using Kalman Filtering, which computes the actual log likelihood, you should use a PAYOFF_VALUE of 2 (or strictly, 1.92, which is half the Chi Squared 95% level of 3.84).  (If errors are normally distributed, the properly weighted sum of squares is 2 times the log likelihood.  It can be shown that the log likelihood tends to a Chi Squared distribution with one degree of freedom and this can be used to determine confidence bounds for a single parameter.)

PAYOFF_PERCENT is useful when the payoff cannot be interpreted as a log likelihood, as, for example, in the case of policy.  The resulting intervals serve to identify which policy levers need to be keenly controlled, and which can be largely ignored.

PARAMETER_PERCENT looks at a percentage change in parameters and reports the percentage change in payoff.  This is faster to execute than PAYOFF_PERCENT and gives similar information if the payoff surface is well behaved.  Also, it can be executed even if the system is not at an optimum, and still give meaningful results.  The numbers reported show the change in the magnitude of the payoff as a result of increasing and decreasing each search parameter by the specified percent.  If the model has been optimized, both of these numbers will be negative.

ALL_CONSTANTS is similar to PARAMETER_PERCENT but considers all model Constants, not just search parameters.  This is a comprehensive diagnostic tool that has little to do with optimization as such. Again, it shows the change in the payoff as a result of the given percentage increases and decreases in all Constants.  Since most Constants are not search parameters, there is no reason to expect these changes to be negative.  At completion, a file named sortsens.tab is created, which lists the information in sensitivity.tab in order of importance.

The optimizer sensitivity analysis can be invoked in conjunction with an optimization or MULTIPLE_START, in which case the sensitivity will be determined relative to the point with maximum payoff.  If the OPTIMIZER and MULTIPLE_START control parameters are turned off, the sensitivity will be performed relative to the input evaluation point.

Optimizer sensitivity analysis will not be performed if you have clicked on Cancel and no optimization has been completed.  If an optimization has been completed and a break requested, then the sensitivity analysis will be performed about the maximum.  If the sensitivity analysis is wanted after a break, but not performed because of a break, set the options OPTIMIZER=OFF and MULTIPLE_START=OFF in runname.out, copy it to a new file and use it as the optimization control file.  The sensitivity will be done about the best point the optimizer has reached.

The output from the sensitivity analysis will be placed in a file called sensitivity.tab.  For SENSITIVITY set to PAYOFF_VALUE or PAYOFF_PERCENT, the file will look like:

:COM The base payoff is 123.22

:SENSITIVITY = PAYOFF_PERCENT = 10

:COMMENT  * Means a boundary was reached. 

.21 <= FRAC_REQ_REW = .332 <= 1.0 * 

26.5 <= NORM_QUALITY = 27 <= 27.21

10* <= WORK_SPEED_BASE = 16 <= 29

The first line reports the payoff value at the starting point.  The second line just reiterates the sensitivity settings.  The third line is a comment reporting that * means that a boundary has been hit.  (That is, the difference in payoff can be less than specified at this point but the constraints in the optimization control file forbid moving this parameter any further).  The remaining values give the parameter names with their starting values, surrounded by their values at the sensitivity limits.  

This file is in a form to be used with the optimizer and the vector graphing option.  During the calculations of the sensitivity bounds, no intermediate calculations will be posted to the screen nor will any output go to trace, startpoint, or endpoint files. The fractional tolerance for sensitivity calculations is set at 1.0e-7 to ensure high accuracy of these results.

For SENSITIVITY set to PARAMETER_PERCENT and ALL_CONSTANTS, sensitivity.tabwill look like:

:COM The base payoff is 3.000000

:SENSITIVITY = PARAMETER_PERCENT = 10.000000

:COM Parameters are changed by +- 10% if 0 by += 0.1

Again, the first two lines are the payoff at the starting point and the type of sensitivity analysis performed.  The next two comments tell the definition of elasticity as reported and what happens in special cases. The sensitivity results are given in table form, where the parameter name and its base value are given on the left and the changes in payoff that result from changes in the parameter are given on the right.  

If SENSITIVITY is set to ALL_CONSTANTS, then another file named sortsens.tab is created.  This file is basically the same as sensitivity.tab, but the constants are sorted by decreasing impact so that it is easier to weed out unimportant constants.  If the optimization is interrupted with a Cancel command, the sorted file will not be created, but sensitivity.tab will.