Summary of Probability Theory

What is probability? Experiments, elementary outcomes, and events

An experiment is a repeatable test that can lead to various outcomes or, as they are commonly called, elementary outcomes: either an outcome happened or it didn’t.

In the simplest situations these outcomes are equal and we cannot favor one over the other, meaning the probability of each is the same. Such outcomes are called equally probable. In a fair experiment (an experiment with equally probable outcomes) with n elementary outcomes, the probability of each outcome is the same and equals 1/n.

The set of all of an experiment's potential elementary outcomes is called the sample space. You can select a subset containing a number of elementary outcomes. This is called an event.

An impossible event is an event which can never happen, so the probability of it occurring is 0. A certain event is an event which will definitely happen, so its probability equals 1. The probability of other events happening is between 0 and 1.

As long as you maintain the condition that all elementary outcomes have equal probability, the event probability is the number of outcomes in the event divided by the overall number of events (i.e., the size of the sample space).

The law of large numbers

The law of large numbers says that the more times you repeat an experiment, the closer the frequency of a given event will be to its probability.

We can use this rule in reverse. If we don’t know the probability of an event, but we can repeat the experiment many times, we can estimate its probability from the frequency of the outcomes.

Mutually exclusive and independent events, multiplying probabilities

To illustrate the intersection between events, you can use a Venn diagram:

Untitled.jpg

If events A and B intersect, this means that there are elementary outcomes that occur in both A and B.

Events that cannot occur simultaneously in the same experiment are called mutually exclusive—they do not intersect on the Venn diagram:

Untitled/_2.jpg

The probability of mutually exclusive events both occurring is zero.

Events are called independent if the occurrence of one does not affect the probability of the other. If events are independent, then the probability of their intersection is the same as the product of their probabilities of occurrence. This rule works in reverse.