A payoff is a single number that summarizes a simulation. You need to define a payoff before you can run an optimization or do Kalman filtering. You can also define a payoff for normal simulations. At the end of the simulation the value of the payoff will be reported and added to the error history window.
The payoff for a model can be defined in terms of a comparison of model variables with actual data, or as a combination of model variables. These two types of payoffs are known as calibration payoffs and policy payoffs. The payoff is defined inside of a Vensim Payoff Definition (.vpd) file. This file is referenced as second step the Optimization wizard takes.