The Law of Large Numbers
The Law of Large Numbers
In the last section 5.5 of Chapter 5, Mayo touches on the question of "applicability" of probabilistic models to actual experiments. Drawing on Neyman's considerations on this problem, she first points out the character of "real random experiments" (165): they are such that "even if carried out repeatedly with the utmost care to keep conditions constant, yield varying results" (quotation from Neyman). Then she continues:
Although we cannot predict the outcome of such experiments, a certain pattern of regularity emerges when they are applied in a long series of trials. ("Long" here does not mean infinitely long or even years, but that they are applied often enough to see a pattern emerge.) The pattern of regularity concerns the relative frequency with which specified results occur. The regularity being referred to is the long-run stability of relative frequencies. (165)
It is this empirical fact of long-run stability, Neyman explains, that gives the mathematical models of probability and statistics their applicability. "It is a surprising and very important empirical fact that whenever sufficient care is taken to carry out" a large number of trials as uniformly as possible, the observed relative frequency of a specified outcome, call it "success", is very close to the probability given by the Binomial probability model ... .
Our warrant for such a conceptual representation is captured by the law of large numbers (LLN). (165-6)
Now, her argument here is sometimes confusing, since she has actually two laws of large numbers in her mind, one mathematical and the other empirical; that is,
(1) Mathematical: if each trial is a Binomial trial with a constant probability of success equal to p, then the relative frequency of success will, with high probability, be close to p in a long series of trials.
(2) Empirical: long-run stability of a relative frequency.
What she is trying to say, in terms of quotations from Neyman and other contemporary writers, seems to be this: since we have the empirical law of large numbers in many kinds of phenomena, we can apply mathematical theory of probability (in which the mathematical law of large numbers holds) to model such phenomena. Thus she concludes, with Neyman, that:
if in repeatedly carrying out a series of random experiments of a given kind we find that they always conform to the empirical law of large numbers, then we can use the calculus of probability to make successful predictions of relative frequencies. (167)
Now, I do not wish to deny this conclusion, but do wish to point out her unfairness as regards the contrast between the frequentist and the Bayesian. Recall that she criticized (in chaptyer 3) the Bayesian as follows:
The possibility of eventual convergence of belief is irrelevant to the day-to-day problem of evaluating the evidential bearing of data in science. (84)
But what is the empirical law of large numbers? Mayo wishes to say that it is a fact (empirical fact), and other frequentists, including Neyman, are mostly in agreement. I do not want to quibble over words, but if it is a fact, it is a quite different kind of fact from, say, the fact that I am male, a Japanese, and short-tempered! The word "fact" here is almost a fake, vis-a-vis the Bayesian, since the empirical law of large numbers is nothing but a supposition of convergence of relative frequency, or "convergence of belief" she denounced! I am not saying that such suppositions are unwarranted, but saying that suppositons or hypotheses or conjectures (or whatever) should not be easily named a "fact"; we must notice the gap between what we observe and what we infer from such an observation, and the empirical law of large number definitely falls within the latter camp. If you begin your argument against the Bayesians, and for the frequentist, by saying that the empirical law of large numbers is a fact, you just beg the whole question, and there would be no need for Reichenbach and Wes Salmon worrying about the justification of induction and applications of probability to finite cases.
Last modified Jan. 26, 2003. (c) Soshichi Uchii