4.2 From Probability to Statistics
In the last chapter, we considered probability theory, which is the mathematics of probability distributions. Given a characterization of a distribution—usually a PF, PDF, or CDF—we may infer certain probabilities. This is probability.
Things become more complicated when we attempt to apply probability to practical problems. We soon realize that probability offers no guidance as to how we might construct distributions. It tells us what to do with them, but not where they come from!
Before we construct distributions, we must decide what we want them to represent. What should probabilities signify? This is another question that probability does not address. According to probability theory, probabilities are numbers satisfying certain axioms. Any interpretation of those numbers is our own.
Over the years, many interpretations have been proposed for probabilities. We may broadly describe these as falling into two categories:
- objective interpretations, and
- subjective interpretations.
According to objective interpretations, probabilities are real. They exist independently of us. We can deduce or approximate them through logic or careful observation. According to the competing subjective interpretations, there are no true probabilities for us to deduce or approximate. We construct probabilities to reflect our perceptions.1
Although philosophers may debate the merits of objective or subjective interpretations, we shall find it convenient to embrace either, depending upon our application. When we discuss the Monte Carlo method in Chapter 5, we shall perceive the underlying probabilities as largely objective. When we discuss the modeling of financial markets in Chapter 7, we shall perceive them as more subjective.
There are various ways probabilities are inferred or assigned:
- symmetry—we ascribe each of 52 cards an equal probability of being drawn;
- personal judgement—we look at the sky and contemplate the probability of rain;
- data analysis—we repeat an experiment a number of times and analyze the results.
Statistics is a body of techniques for inferring or assigning probabilities based on data. Objective and subjective interpretations of probability support competing statistical traditions. Both arose during the 20th century. The objectivist tradition, called classical statistics, developed from the works of Karl Pearson and Ronald A. Fisher. The subjectivist tradition, called Bayesian statistics, developed from the works of Bruno de Finetti and Leonard J. Savage.
In this book, we employ the methods and terminology of classical statistics, irrespective of whether we perceive specific probabilities as objective or subjective. We do so for expedience only: readers are likely to be more familiar with classical statistics.