Independence

Simulation of the dice experiment

Simulation of Buffon's coin experiment

Independence is one of the most important concepts in probability and is frequently used as a modeling assumption.

Independence of Two Events

Two events A and B in a random experiment are independent if

If both of the events have positive probability, then independence is equivalent to the statement that the conditional probability of one event given the other is the same as the unconditional probability of the event:

This is how you should think of independence: knowledge that one event has occurred does not change the probability assigned to the other event.

1. Recall that the dice experiment consists of rolling a pair of (distinct) fair dice and recording (X, Y) where X is the score on die 1 and Y is the score on die 2. Show that the following pairs of events are independent. Think about the result.

1. {X = 3}, {X = 4}
2. {X = 3}, {X + Y = 7}

2. Run the simulation of the dice experiment 500 times. For each pair of events in Exercise 1, compute the product of the relative frequencies and the relative frequency of the intersection. Compare the results.

3. Recall that Buffon's coin experiment consists of tossing a coin with radius r < 1 randomly on a floor covered with square tiles of side length 1. The coordinates (X, Y) of the center of the coin are recorded relative to axes through the center of the square, parallel to the sides. Show that {X < 0} and {Y > 0} are independent. Think about the result.

4. Run the simulation of the dice experiment 500 times. For the pair of events in Exercise 3, compute the product of the relative frequencies and the relative frequency of the intersection. Compare the results.

The terms independent and disjoint sound vaguely similar but they are actually very different. First, note that disjointedness is purely a set-theory concept while independence is a measure-theory concept. Indeed, two events can be independent relative to one probability measure and dependent relative to another. But most importantly, two disjoint events can never be independent, except in a trivial case.

5. Suppose that A and B are disjoint events for an experiment, each with positive probability. Show that A and B are negatively correlated.

General Independence of Events

6. In the dice experiment, consider the three events {X = 3}, {Y = 4}, {X + Y = 7}.

1. Show any two of the events are independent.
2. Show that {X = 3, Y = 4} implies (is a subset of) {X + Y = 7}.

Exercise 6 shows that a collection of events can be pairwise independent, but two of the events together can be related to a third event in the strongest possible sense.

We will now extend the concept of independence to a collection of events A_i indexed by a countable set I. The events are said to be (mutually) independent if

The general definition of independence is equivalent to the following condition that involves only independence of pairs of events: If J and K are disjoint subsets of the index set I., and if B is an event constructed from the events A_j, j in J (using the set operations of union, intersection, and complement) and C is an event constructed from the events A_k, k in K, then B and C are independent.

7. Suppose that A and B are independent events in an experiment. Show that each of the following pairs of events is independent:

1. A^c, B
2. A, B^c
3. A^c, B^c

8. Suppose that A, B, C, and D are independent events in an experiment. Show that the following events are independent:

The following problem gives a formula for the probability of the union of a collection of independent events that is much nicer than the inclusion-exclusion formula.

9. Suppose that A₁, A₂, ..., A_n are independent events. Show that

10. Suppose that A, B, and C are independent events in an experiment with P(A) = 0.3, P(B) = 0.5, P(C) = 0.7. Express each of the following events in set notation and find its probability:

1. At least one of the three events occurs.
2. None of the three events occurs.
3. Exactly one of the three events occurs.
4. Exactly two of the three events occurs.

Independence of Random Variables

Intuitively, random vectors X and Y are independent if the observed value of one variable does not change the distribution of the other. Mathematically, independence of random vectors can be reduced to the independence of events. Suppose that X_i is a random vector taking values in a set S_i for each i in an index set I. The random vectors are independent if any collection of events of the following form is independent:

11. In the dice experiment, show that X and Y are independent. Think about the result.

12. In Buffon's coin experiment, show that X and Y are independent. Think about the result.

13. Consider a collection of independent random vectors as defined above, and suppose that for each i in I, g_i is a function from S_i into a set T_i. Show that the random vectors Y_i = g(X_i), i in I are independent.

14. Show that events A_i, i in I are independent if and only if the indicator variables 1_Ai, i in I are independent.

Probability Spaces