Conditional Distributions

Java Simulation of the bivariate uniform experiment

Suppose that X and Y are random vectors for the same experiment. Suppose that X takes values in a subset S of Rⁿ and Y takes values in a subset T of R^m. Then (X, Y) is a random vector taking values in the subset S × T of R^{n + m}.

The purpose of this section is to study the conditional distribution of Y given X = x.

The Discrete Case

Suppose that (X, Y) is discrete with density function f. Recall that X is also discrete and has density function g given by

1. Use the definition of conditional probability to show that

P(Y = y | X = x) = f(x, y) / g(x) for x in S, y in T

2. Let h(y | x) denote the function in Exercise 1. Show that as a function of y for fixed x, h( y | x) is a discrete density function on T, namely the conditional density of Y given X = x.

3. Show that

4. Use the law of total probability to show that

Of course, the conditional density function of X given Y = y in T can be defined analogously. We will denote the value of this density at x in S by g(x | y).

The Continuous Case

Suppose now that that (X, Y) is continuous with density function f. Recall that X is also continuous and has density function g given by.

We cannot use conditional probability to define the conditional distribution of Y given X = x, because the given event has probability 0. Nonetheless, the concept should make sense. If we actually run the experiment, X will take on some value x (even though a priori, this event occurs with probability 0), and surely the information that X = x should in general alter the probabilities that we assign to other events. A natural approach is to use the results obtained in the discrete case as definitions in the continuous case. Thus, we will define

h(y | x) = f(x, y) / g(x) for x in S, y in T

5. Show that as a function of y, for fixed x, h(y | x) is a density function for a continuous distribution on T.

The function in Exercise 5 is called the conditional density function of Y given X = x. Now we can define

The real justification for our definition is the fact that the law of total probability holds.

6. Show that

Just as in the discrete case, we define the conditional density function of X given Y = y analogously. The value of this function at x in S will be denoted g(x | y).

7. Show that in either the discrete or continuous cases, the following conditions are equivalent:

1. X and Y are independent
2. g(x | y) = g(x) for x in S, y in T
3. h(y | x) = h(y) for x in S, y in T

Intuitively, Exercise 7 is reasonable: X and Y are independent if and only if knowledge of the value of one of the variables does not change the distribution of the other.

Bivariate Uniform Distributions

Bivariate uniform distributions provide nice examples of the ideas in discussed above. Suppose that R is a subset of R² with finite, positive area a and that (X, Y) is uniformly distributed on R. Recall that the joint density is

f(x, y) = 1 / a for (x, y) in R

8. Show that the conditional distribution of Y given X = x is uniformly distributed on {y: (x, y) in R}.

9. Show that the conditional distribution of X given Y = y is uniformly distributed on {x: (x, y) in R}.

In the last section, we saw that even though (X, Y) is uniformly distributed, the marginal distributions of X and Y are not uniform in general. However, by Exercises 8 and 9, the conditional distributions are always uniform.

The Bivarate Uniform Simulation

Recall that in the bivariate uniform simulation, the random vector (X, Y) is uniformly distributed on one of three regions that can be selected from the list box:

1. the square R = {(x, y): -6 < x < 6, -6 < y < 6}
2. the triangle R = {(x, y): -6 < y < x < 6}
3. the circle R = {(x, y): x² + y² < 36}

When you run the simulation, the values of (X, Y) are ploted in the scatterplot on the left. The other two graphs show the marginal densities of X and Y in blue and the corresponding empirical densities in red. The table on the left gives the values of X and Y. The middle tables give the mean, standard deviation, sample mean and sample standard deviation of X and of Y. The table on the right gives the correlation and sample correlation between X and Y.

10. Suppose that (X, Y) is uniformly distributed on the square (1). Find explicitly the conditional density of Y given X = x in (-6, 6) and the conditional density of X given Y = y in (-6, 6).

11. In the bivariate uniform simulation, select square in the list box. Run the simulation 5000 times, updating every 10 runs. Watch the points in the scatter plot and the graphs of the marginal distributions. Interpret what you see in the context of Exercises 8 and 9.

12. Suppose that (X, Y) is uniformly distributed on the triangle (2). Find explicitly the conditional density of Y given X = x in (-6, 6) and the conditional density of X given Y = y in (-6, 6).

13. In the bivariate uniform simulation, select triangle in the list box. Run the simulation 5000 times, updating every 10 runs. Watch the points in the scatter plot and the graphs of the marginal distributions. Interpret what you see in the context of Exercises 8 and 9.

14. Suppose that (X, Y) is uniformly distributed on the circle (3). Find explicitly the conditional density of Y given X = x in (-6, 6) and the conditional density of X given Y = y in (-6, 6).

15. In the bivariate uniform simulation, select circle in the list box. Run the simulation 5000 times, updating every 10 runs. Watch the points in the scatter plot and the graphs of the marginal distributions. Interpret what you see in the context of Exercises 8 and 9.

Additional Exercises

16. Suppose that (X, Y) has probability density function f given by

f(x, y) = (x + y) / 4 for 0 < x < y < 2

Find the conditional density of X given Y = y and the conditional density of Y given X = x. Be sure to specify the range of values of the variables.

17. Suppose that (X, Y) has probability density function f given by

f(x, y) = (x + y) / 3 for 0 < x <1, 0 < y < 2

Find the conditional density of X given Y = y and the conditional density of Y given X = x. Be sure to specify the range of values of the variables.

18. Suppose that (X, Y) has probability density function f given by

f(x, y) = (15 / 32) x²y for 0 < x < y < 2

Find the conditional density of X given Y = y and the conditional density of Y given X = x. Be sure to specify the range of values of the variables.

19. Suppose that (X, Y) has probability density function f given by

f(x, y) = (3 / 2) x²y for 0 < x < 1, 0 < y < 2

Find the conditional density of X given Y = y and the conditional density of Y given X = x. Be sure to specify the range of values of the variables.

Bayes' Theorem

Bayes’ theorem is a formula for computing the conditional densities in one direction in terms of the conditional densities in the other direction. The formula is frequently used to update probability distributions in light of new information.

20. Bayes’ theorem. Show that

h(y | x) = g(x | y) h(y) / C(x) for x in S, y in T

where the normalizing constant C(x) is given by

Distributions