Powered by MathJax
We use MathJax

Probability Rules

As the number of variables in a frequency distribution grows, the enumeration of different events becomes more complicated. Rather than continuing with a representation of every situation as a multidimensional sample or sample space, some basic rules will be useful.

Basic Properties

We first make some basic observations about probabilities.

The Four Probability Rules

Whenever an event is the union of two other events, the Addition Rule will apply. Specifically, if $A$ and $B$ are events, then we have the following rule.

$P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$

In set notation, this can be written as   $P(A \cup B) = P(A) + P(B) - P(A \cap B)$.

Whenever an event is the complement of another event, the Complementary Rule will apply. Specifically, if $A$ is an event, then we have the following rule.

$P(\text{not } A) = 1- P(A)$

In set notation, this is written as   $P(\overline{A}) = 1 - P(A)$.

Whenever partial knowledge of an event is available, then the Conditional Rule will apply. Specifically, if event $A$ is already known to have occurred and probability of event $B$ is desired, then we have the following rule.

$P(B, \text{ given } A) = \dfrac{P(A \text{ and } B)}{P(A)}$

In set notation, this is written as   $P(B|A) = \dfrac{P(A \cap B)}{P(A)}$.

Lastly, whenever an event is the intersection of two other events, the Multiplication Rule will apply. That is, events $A$ and $B$ need to occur simultaneously. Therefore, if $A$ and $B$ are events, then we have the following rule.

$P(A \text{ and } B) = P(A) \cdot P(B,\text{ given } A)$

In set notation, this is written as   $P(A \cap B) = P(A) \cdot P(B|A)$.

An Empirical Example

As an example, consider the relative frequencies of blood types in a particular population, as given in the following table.

  Type A Type B Type AB Type O Totals
Rh+ 0.34 0.09 0.04 0.38 0.85
Rh- 0.06 0.02 0.01 0.06 0.15
Totals 0.40 0.11 0.05 0.44 1.00

Suppose one individual from the population is randomly selected. We have the following probabilities.

In the last part of the example above, use of the Multiplication Rule probably seemed rather silly. In fact, its use was a bit circular, since the Multiplication Rule and the Conditional Rule are so closely related. And in fact, whenever you have a completed contingency table, there would be no need to do such a computation. The Multiplication Rule is much more useful in other contexts.

Rolling Dice

Let us now examine the probability rules in the context of the classical example of rolling dice. Suppose we roll two dice.

Selections With and Without Replacement

Suppose we have a bag of ten jellybeans. Four of the jellybeans are red, three are green, two are yellow, and one is orange. Two jellybeans will be randomly selected. But before we can do the computations, we must know whether the first jellybean selected will be returned to the bag before the second jellybean is selected. If it is returned to the bag, the bag will be restored to its original condition, and the probabilities for the second jellybean will be identical to the first. If the first jellybean is not returned to the bag, then the probabilities for the second jellybean will be different than the first.

In looking at the examples above, you will probably notice that the "without replacement" case is much more interesting.

Statistical Independence

The Conditional Rule required taking into account some partial knowledge, and in so doing, recomputing the probability of an event. Sometimes, the value changed. In the first example, the probability of selecting an individual with Rh+ blood was 85%, but once it was known that the individual had Type AB blood, the probability changed to 80%. Similarly, the probability of selecting a green jellybean was $\dfrac{3}{10}$, but once a green jellybean was removed from the bag, the probability of another green jellybean changed to $\dfrac29$.

Sometimes, though, partial knowledge will not change the probabilities. This was certainly the case when rolling dice. The probability that the second die is a 5 is $\dfrac16$, whether or not we know the outcome of the first die. In cases where partial knowledge of one event has no effect on the probability of another event, we say the two events are statistically independent, or (as long as the context is statistics), just independent. Events on the two dice were independent. So were the two selections from the jellybean bag when the selections were made with replacement. In other words, if $A$ and $B$ are two events, then they are independent whenever:

$P(B, \text{ given } A) = P(B)$

This condition is equivalent to   $P(A, \text{ given } B) = P(A)$.   In other words, it does not matter which event is the source of the partial knowledge. Furthermore, it is equivalent to   $P(A \text{ and } B) = P(A) \cdot P(B)$,   which is a huge simplification of the Multiplication Rule.

The independence of two events is not the same as two mutually exclusive events. In fact, mutually exclusive events are never independent. If $A$ and $B$ are mutually exclusive, then partial knowledge that $A$ has occurred means that the probability of $B$ will become zero, that is,   $P(B, \text{ given } A) = 0$.