Moment Generating Functions

Definition

Let X be a random variable for an experiment taking values in a subset S of R. The moment generating function of X is the function M_X defined by

M_X(t) = E[exp(tX)] for t in R

Note that since exp(tX) is a nonnegative random variable, M_X(t) exists as a real number or positive infinity for any t.

1. Show that if X has a discrete distribution with density function f, then

2. Show that if X is continuous with density function f, then

Since the exponential function is positive, the moment generating function of X always exists, either as a real number or as positive infinity.

The most important fact is that if the moment generating function of X is finite in an open interval about 0, then this function completely determines the distribution of X. Under the same condition, we can differentiate through the sum in Exercise 1 and through the integral in Exercise 2. For proofs of these facts, see the book Probability and Measure.

Properties

In the following exercises, assume that the moment generating functions are finite in an interval about 0.

3. Show that for any nonnegative integer n,

M_X⁽ⁿ⁾(0) = E(Xⁿ)

Thus, the derivatives of the moment generating function at 0 determine the moments of the variable (hence the name).

4. If a and b are constants, show that

M_{aX + b}(t) = exp(bt) M_X(at)

5. Suppose that X and Y are independent. Show that

M_{X + Y}(t) = M_X(t) M_Y(t)

The result in Exercise 5 is one of the most important properties of moment generating functions, and is frequently used to determine the distribution of a sum of independent variables. By contrast, the density of a sum of independent variables is the convolution of the densities, a much more complicated operation.

Joint Moment Generating Functions

Suppose now that (X, Y) is a random vector for an experiment, taking values in a subset S of R². The (joint) moment generating function of (X, Y) is defined by

M_{X, Y}(s, t) = E[exp(sX + tY)] for s, t in R

Once again, the important fact is that if the moment generating function is finite in an open rectangle containing (0, 0) then this function completely determines the distribution of (X, Y).

6. Suppose that (X, Y) has a discrete distribution with density function f. Show that

7. Suppose that (X, Y) has a continuous distribution with density function f. Show that

Properties

8. Show that

1. M_{X, Y}(s, 0) = M_X(s)
2. M_{X, Y}(0, t) = M_Y( t)
3. M_{X, Y}(t, t) = M_X+Y(t)

9. Show that X and Y are independent if and only if

M_{X, Y}(s, t) = M_X(s) M_Y( t) for (s, t) in a rectangle about (0, 0).

Naturally, the results for bivariate moment generating functions have analogues in the general multivariate case. Only the notation is more complicated.

Expected Value