5.2 The Monte Carlo Method
In Section 2.15, we considered quadrature rules for numerical integration. These suffer from the curse of dimensionality, so they are useless for evaluating high-dimensional integrals. The Monte Carlo method is a technique of numerical integration that overcomes this curse. It is as applicable to a 500-dimensional integral as it is to a one-dimensional integral.
5.2.1 Stanislaw Ulam
Credit for inventing the Monte Carlo method often goes to Stanislaw Ulam, a Polish-born mathematician who worked for John von Neumann on the United States’ Manhattan Project during World War II. Ulam is primarily known for designing the hydrogen bomb with Edward Teller in 1951.1 He conceived of the Monte Carlo method in 1946 while pondering the probability of winning a game of solitaire.2 After attempting to solve this problem with pure combinatorial calculations, he wondered if it might be simpler to play multiple hands of solitaire and observe the frequency of wins. This lead Ulam to consider how problems of neutron diffusion and other questions of mathematical physics might be represented in a form interpretable as a succession of random operations.
The Monte Carlo method, as it is understood today, encompasses any technique of statistical sampling employed to approximate solutions to quantitative problems. Ulam did not invent statistical sampling. This had been employed to solve quantitative problems before,3 with physical processes such as dice tosses or card draws being used to generate realizations of samples. Ulam’s contribution was to recognize the potential for the newly invented electronic computer to automate such sampling. Working with John von Neumann and Nicholas Metropolis, he developed algorithms for computer implementations, as well as exploring means of transforming nonrandom problems into random forms that would facilitate their solution via statistical sampling.4 5 This work transformed statistical sampling from a mathematical curiosity to a formal methodology applicable to a wide variety of problems. It was Metropolis who named the new methodology after the casinos of Monte Carlo. Ulam and Metropolis published the first paper on the Monte Carlo method in 1949.6