5.4 Canonical Inquiries into ErrorsLet us make clear the scenario of this subsection. At the end of Chapter 1, Mayo mentioned four types of canonical error. The first type was mistaking artifacts for real effects. Inquiries into this type of error can be modeled on the Binomial Experiment (like a coin tossing).
Experimental Model: Suppose you toss a coin 3 times, and you want to test the hypothesis that this coin is fair (null hypothesis). What is essential in this test is the set of possible outcomes and the probability distribution over the set. Given the hypothesis, this distribution can be easily calculated. The essential information is condensed in the proportion---the relative frequency---of heads in the 3 tosses, so that what we need is the probability of each possible value of the relative frequency.
Data Model: Given the actual outcome, what's important is the relation of the relative frequency in the outcome (sample) and the probability of head; the former gives some information (evidence) for learning the latter.
Checking the Data Model: This canonical model can be applied to an experiment for the first type of error. But notice that, in applying this model, you make the experimental assumption that this case is a binomial experiment. And we can check this assumption separately: do the actual data meet assumed conditions?
An example of such an application is "A Lady Tasting Tea". Is the judgment of this lady on tea a mere guess, or a more credible one? "Mere guess" corresponds to the "fair coin hypothesis", and is regarded as the null hypothesis.
Then mathematical details of this experiment are just the same as the binomial experiment; and in particular, significance level is set, and the null hypothesis is either rejected or accepted according as the experimental results exceed or not that level. Crucial judgments in this experiment are made (according to Neyman-Pearson) in terms of two types of Error Probabilities.
Type I Error: To reject the null hypothesis when it is in fact true.
Type II Error: To accept the null hypothesis when it is in fact false.
Added March 30, 2001:
Readers who need more explanation should consult Ishikawa (1997), ch. 3, pp.45-61. There, a similar problem (in terms of dice) is treated more in detail. Try to calculate probabilities yourself (on a computer)!
石川幹人『サイコロとExcelで体感する統計解析』共立出版、1997。
INDEX- CV- PUBLS.- PICT.ESSAYS- ABSTRACTS- INDEX・LAPLACE- OL.ESSAYS-CRS.MATERIALS
Last modified Sept. 20, 2007. (c) Soshichi Uchii
