January 20, 2004
Probability & Law of Large Numbers
We begin with an introduction to probability and a statment of the Law of Large Numbers.
The definition of a probability of one-half is a bit curcular, but we all agreed that the concept of a fair coin as "obvious" as the concept of "round." We are also able to construct reasonable physical counterparts to probabilities of 1/3, 1/4, and 1/5.
Probabilities have to satsify three postulates that do not unambiguously determine probabilities, but do constrain their properties.
A random variable X is a function of the elementary outcomes in the sample space. The population expected value of X is the limit (for large sample) of the sample mean. This convergence is a direct implication of the Law of Large Numbers, which states that the sample relative frequency of an outcome converges to its probability as the sample size goes to infinity.
A similar result follows the for sample variance converging to the population variance (bottom panel).
We concluded with a preview of chapters to come. The Central Limit Theorem extends the Law of Large Numbers to tell us the distribution of sample averages. If vote for Candidate A is "heads = 1" and vote for Candidate B is "tails = 0", then X equal to the sum of the ones and zeroes is how many people vote for Candidate A.
A classic statistical problem is considering the population percentage of people in favor of Candidate A given a sample of survey responses.
If the percentage is likely to be near 50%, a survey of 100 people is not very useful. A sample of 10,000 is more precise.
If the winning margin is lilely to be a half percentage point or less, then even a survey of 10,000 people is not large enough.
TV talking heads beware!