Dave Pankhurst - Using ‘MultiStream’ to Control Uncertainty to Model Changes

  • Presenting author: Dave Pankhurst (Department for Work and Pensions, UK)

  • Authors: Howard Redway & Steve Webster

  • Session: C04D - Uncertainty - Wednesday 16:00-17:00 - Erika-Weinzierl Hall

  • Slides: PDF

The use of one single stream of random numbers in a dynamic microsimulation model can result in an increase in uncertainty if a change in a model parameter results in a change in the number of random numbers required by later processes. The effect of the disruption to the assignment of random numbers to objects (e.g. persons or families) may change many outcomes at the micro level. One consequence can be that small model changes have a disproportionate effect on aggregate outputs, due to the additional randomness. UK Department for Work and Pensions (DWP) refers to this randomness as ‘Stochastic Ripple’. The DWP is responsible for over £230 billion of benefit and pension expenditure. It uses a range of dynamic microsimulation models to forecast expenditure and assess proposed policy changes. The additional uncertainty caused by Stochastic Ripple may make the estimate of a policy change unreliable. For small changes Stochastic Ripple may result in modelled effects that are contrary to expectation, causing confusion and a loss of confidence in the models. Stochastic Ripple was found to be a major problem during the development of Gaussian Emulators for DWP models (see separate abstract) as this methodology relies on model outputs varying smoothly with the model parameters. To reduce Stochastic Ripple the DWP has developed a method, called ‘MultiStream’, using multiple random number streams. A separate stream is allocated to each object in the model for which there are stochastic events. Each stream is seeded from a function of the unique object ID. The presentation will:

  • Illustrate the problem with a simple example.
  • Show how MultiStream reduces the additional uncertainty.
  • Present results of small parameter changes, with and without MultiStream.
  • Explain how MultiStream is being implemented in DWP models.
  • Describe the theoretical basis of MultiStream and the statistical tests to validate the method.
  • Place this in the context of the development of DWP microsimulation models.