Risk analysis tools are used to create a risk model, perform simulations using the model, and analyze the results.  An Excel spreadsheet can be a simple, yet powerful tool for creating your model -- especially when paired with Monte Carlo simulation software such as Analytic Solver Simulation.  If your application calls for a programming language, Monte Carlo simulation toolkits like the one in Frontline's Solver Platform SDK provide powerful aids.

A special-purpose simulation modeling language can be a productive tool, but we must consider its non-simulation-related capabilities:  For example, how easy is it to access external databases in the simulation language, or embed a model in this language in a larger application program?  Such a language is most useful if your application calls for discrete event simulation, as opposed to Monte Carlo simulation.  For example, a manufacturing process with many different "flows" of materials, and many steps taking place over time could be a candidate for a simulation modeling language.

Once we have a complete model in a form appropriate for our chosen risk analysis tool, we can execute or "run" it, performing one or more simulations, to get results.  We want the software to do the work, since it would take many hours to run the model manually, thousands of times.

The basic step in a simulation run, called a trial, is very simple:

  1. Choose sample values for the uncertain variables, and "plug them into" the model. Evaluate the model:  Run the program, or recalculate the spreadsheet.
  2. Observe and record the values of the uncertain functions.

For models with few uncertain variables, where the possible values of these variables cover a limited range, it may be possible to run a series of trials, where we systematically  "step through" the range of each variable, in all combinations.  For example, if we had two variables, X ranging from 1 to 100 and Y ranging from 10 to 50 in steps of 5, we'd perform a trial with X = 1 and Y = 10, then X = 1 and Y = 15, and so on for a total of 900 trials.  But for most models of any size, this approach is impractical:  We would need millions or even billions of trials, and running them all might not actually tell us very much.

Hence, simulation normally relies on random sampling of values for the uncertain variables:  In step 1 above, we draw one or more random numbers -- analogous to flipping a coin or rolling dice on the computer -- and we use these numbers to randomly select sample values from the range of possible values (the distribution) of each uncertain variable.  If we do this effectively (using high-quality random number generation and sampling methods), we can obtain excellent "coverage" of the possible values and model outcomes -- even if the model has many uncertain variables.

Trials, Sampling, and Accuracy

A simulation can be run for as many trials as you specify.  To obtain more accurate results, you must run more trials -- so there is a tradeoff between accuracy of the results, and the time taken to run the simulation.  But there are several ways you can improve this tradeoff, and obtain good accuracy in a limited amount of time:

  • Ensure that each trial runs as fast as possible.  If you create your model in a compiled programming language, it can execute trials at the fastest possible speed -- but this usually requires more development time.  Running a model in Excel is slower, since your Excel formulas must be interpreted on each simulation run.  But you can take advantage of PSI Technology in Frontline's Analytic Solver Simulation software to run trials hundreds of times faster than using Excel alone.
  • Use a sampling method that provides better coverage of the possible values of variables, and lower variance (higher accuracy) for the outcomes, than standard Monte Carlo sampling for the same number of trials.  Two advanced methods are Latin Hypercube sampling and Sobol numbers, which are an alternative to simple random numbers.  Frontline's Analytic Solver Simulation and Solver Platform SDK support both of these advanced sampling methods.
  • For demanding applications, such as those found in quantitative finance, use multiple streams of random numbers in combination with a high quality, long period random number generator to minimize any dependencies between samples for your uncertain variables.  Again these capabilities are available in Frontline's Analytic Solver Simulation and Solver Platform SDK.

Analyzing Model Results

Because a simulation yields many possible values for the outcomes we care about -- from Net Profit to environmental impact -- some work is needed to analyze the results.  It is very useful to create charts to help us visualize the results.  We can summarize the range of outcomes using various kinds of statistics, such as the mean or median, the standard deviation and variance, or the 5th and 95th percentile or Value at Risk.

Another powerful tool for assessing model results is sensitivity analysis, which can help us identify the uncertain inputs with the biggest impact on our key outcomes. Using software, we can also run multiple simulations, with an input we choose taking a different value on each simulation, and assess the results.  Analyzing the model can give us more information, but also insight about our real-world problem.