10. Hypothesis Testing Methods and Confidence Regions 609
2.5 Marginal Probability Density Functions and CDFs
Suppose that we have knowledge of the probability space corresponding to an experiment involving outcomes of the n-dimensional random variable X(n)ẳ (X1,. . .,Xm, Xm+1,. . .,Xn), but our real interest lies in assigning probabilities to events involving only the m-dimensional random variable X(m)ẳ(X1,. . .,Xm), m<n. In practical terms, this relates to an experiment in which n different
14The reader is reminded that we are suppressing the technical requirement that for every Borel set ofyvalues, the associated collection ofwvalues inSmust constitute aneventinSfor the functionYto be called a random variable. As we have remarked previously, this technical difficulty does not cause a problem in applied work.
characteristics were recorded for each outcome but we are specifically interested in analyzing only a subset of the characteristics. We will now examine the concept of amarginal probability density function (MPDF)forX(m), which will be derived from knowledge of the joint density function forX(n). Once defined, the MPDF can be used to identify the appropriate probability space only for the portion of the experiment characterized by the outcomes of (X1,. . .,Xm), and we will be able to use the MPDF in the usual way (summation in the discrete case, integration in the continuous case) to assign probabilities to events concerning (X1,. . .,Xm).
The key to understanding the definition of a marginal probability density is to establish the equivalence between events of the form (x1,. . .,xm) ∈Bin the probability space for (X1,. . .,Xm) and events of the form (x1,. . .,xn) ∈A in the probability space for (X1,. . .,Xn) since it is the latter events to which we can assign probabilities knowingf(x1,. . .,xn).
2.5.1 Bivariate Case
Letf(x1,x2) be the joint density function andR(X)be the range of the bivariate random variable (X1, X2). Suppose we want to assign probability to the event x1 ∈ B. Which event for thebivariaterandom variable is equivalent to eventB occurring for the univariate random variable X1? By definition, this event is given byAẳ{(x1,x2):x1 ∈ B, (x1,x2) ∈ R(X)}, i.e., the eventBoccurs forx1iff the outcome of (X1,X2) is inAso that x1 ∈ B. Then sinceBandAareequivalent events, the probability that we will observex1 ∈ Bis identically equal to the probability that we will observe (x1,x2) ∈ A(recall the discussion of equivalent events in Section2.2).
For the discrete case, the foregoing probability correspondence implies that PX1ðBị ẳP xð 12Bị ẳPðAị ẳ X
x1;x2
ð ị2A
f xð 1;x2ị:
Our convention of defining f(x1, x2)ẳ0 8(x1, x2)2=R(X) allows the following alternative representation ofPX1(B):
PX1ðBị ẳ X
x12B
X
x22R Xð ị2
f xð 1;x2ị
The equivalence of the two representations ofPX1(B) follows from the fact that the set of elementary events being summed over the latter case,Cẳ{(x1,x2):x1∈B, x2∈R(X2)} is such thatAC, andf(x1,x2)ẳ08(x1,x2)∈CA. The latter repre- sentation ofPX1(B) leads to the following definition of themarginal probability densityofX1:
f1ð ị ẳx1 X
x22R Xð ị2
f xð 1;x2ị:
This function, when summed over the points comprising the event x1 ∈ B, yields the probability thatx1 ∈ B, i.e.,
2.5 Marginal Probability Density Functions and CDFs 79
PX1ðBị ẳ X
x12B
f1ðx1ị ẳ X
x12B
X
x22RðX2ị
fðx1;x2ị:
Heuristically, one can think of the marginal density ofX1as having been defined by “summing out” the values of x2 in the bivariate PDF for (X1, X2). Having definedf1(x1), the probability space for the portion of the experiment involving onlyX1, can then be defined asfR Xð ị;1 ϒX1;PX1gwherePX1ðBị ẳP
X12Bf1ðx1ịfor B2ϒX1. Note that the order in which the random variables are originally listed is immaterial to the approach taken above, and the marginal density function and probability space forX2could be defined in an analogous manner by simply reversing the roles of X1andX2in the preceding arguments. The MPDF forX2 would be defined as
f2ðx2ị ẳ X
x12RðX1ị
fðx1;x2ị;
with the probability space forX2defined accordingly.
Example 2.19 Marginal PDFs in a Discrete Bivariate Case
Reexamine Example 1.16, in which an individual was to be drawn randomly from the work force of the Excelsior Corporation to receive a monthly “loyalty award.” Define the bivariate random variable (X1,X2) as
x1ẳ 0 1 ( )
if male female
( )
is drawn,
x2ẳ 0 1 2 8>
><
>>
: 9>
>=
>>
; if
sales clerical production 8>
><
>>
:
9>
>=
>>
;
worker is drawn;
so that the bivariate random variable is measuring two characteristics of the out- come of the experiment: gender and type of worker. The joint density of the bivariate random variable is represented in tabular form below, where the nonzero values off (x1,x2) are given in the cells formed by intersecting anx1-row with ax2-column.
R(X2)
0 1 2 f1(x1) 0
RðX1ị 1
.165 .135 .150 .450 .335 .165 .050 .550 f2(x2) .500 .300 .200
The nonzero values of the marginal density ofX2 are given in the bottom marginof the table, being the definition of the marginal density
f2ðx2ị ẳ X
x12RðX1ị
fðx1;x2ị ẳ X1
x1ẳ0
fðx1;x2ị ẳ:5If g0 ðx2ị ỵ:3If g1 ðx2ị ỵ:2If g2 ðx2ị:
The probability space forX2is thus {R(X2),ϒX2,PX2} withϒX2 ẳ{A:AR(X2)} and PX2(A)ẳ P
x22Af2ð ị. If one were interested in the probability that the individualx2
chosen was a sales or clerical worker, i.e., the eventAẳ{0,1}, then PX2(A)ẳ P1
x2ẳ0fx2(x2)ẳ.5ỵ.3ẳ.8.
The nonzero values of the marginal density forX1are given in the right-hand marginof the table, the definition of the density being
f1ð ị ẳx1 X
x22R Xð ị2
f xð 1;x2ị ẳ X2
x2ẳ0
f xð 1;x2ị ẳ:45Ið0ịð ị ỵx1 :55Ið1ịð ị:x1
The probability space forX1is thus {R(X1),ϒX1,PX1} withϒX1 ẳ{A:AR(X1)} and PX1(A)ẳ P
x12Af1ð ịx1 . If one were interested in the probability that the individ- ual chosen was male, i.e., the eventAẳ{0}, thenPX1(A) ẳ P0
x1ẳ0fx1ð ị ẳx1 .45.□ The preceding example provides a heuristic justification for the termmar- ginalin the bivariate case and reflects the historical basis for the namemarginal density function. In particular, by summing across the rows or columns of a tabular representation of the joint PDFf(x1,x2) one can calculate the marginal densities ofX1andX2in themarginsof the table.
We now examine the marginal density function concept for continuous random variables. Recall that the probability of event B occurring for the univariate random variable X1 is identical to the probability that the event Aẳ{(x1,x2):x1 ∈ B, (x1,x2) ∈ R(X)} occurs for the bivariate random variable Xẳ(X1,X2). Then
PX1ðBị ẳP xð 12Bị ẳPðAị ẳ ð
x1;x2
ð ị2A
f xð 1;x2ịdx1dx2:
Our convention of defining f(x1, x2) ẳ0 8(x1, x2)2= R(X) allows an alternative representation ofPX1ðBịto be given by
PX1ðBị ẳ ð
x12B
ð1
1f xð 1;x2ịdx2dx1:
The equivalence of the two representations follows from the fact that the set of elementary events being integrated over in the latter case,Cẳ{(x1,x2):x1∈B, x2 ∈(1,1)}, is such thatAC, andf(x1,x2)ẳ08(x1,x2)∈CA. The latter representation ofPX1(B) leads to the definition of the marginal density ofX1as f1ðx1ị ẳ
ð1
1fðx1;x2ịdx2:
This function, when integrated over the elementary events comprising the event x1∈B, yields the probability thatx1∈B, i.e.,
PX1ðBị ẳ ð
x12B
f1ð ịx1 dx1ẳ ð
x12B
ð1
1f xð 1;x2ịdx2dx1:
Heuristically, one might think of the marginal density ofX1as having been defined by “integrating out” the values ofX2in the bivariate density function for (X1,X2).
2.5 Marginal Probability Density Functions and CDFs 81
Having defined f1(x1), the probability space for the portion of the experiment involving onlyX1can then be defined as {R(X1),ϒX1,PX1} wherePX1ðAị ẳé
x12A
f1ðx1ịdx1 for A ∈ ϒX1. Since the order in which the random variables were originally listed is immaterial, the marginal density function and probability space forX2can be defined in an analogous manner by simply reversing the roles ofX1andX2in the preceding arguments. The MPDF forX2would be defined as f2ðx2ị ẳ
ð1
1fðx1;x2ịdx1
with the probability space forX2defined accordingly.
Example 2.20 Marginal PDFs in a Continuous Bivariate Case
The Seafresh Fish Processing Company operates two fish processing plants. The proportion of processing capacity at which each of the plants operates on any given day is the outcome of a bivariate random variable having joint density functionf(x1,x2)ẳ(x1 ỵx2)I[0,1](x1)I[0,1](x2). The marginal density function for the proportion of processing capacity at which plant 1 operates can be defined by integrating outx2fromf(x1,x2) as
f1ðx1ị ẳ ð1
1fðx1;x2ịdx2ẳ ð1
1ðx1ỵx2ịIẵ0;1ðx1ịIẵ0;1ðx2ịdx2
ẳ ð1
0
x1þx2
ð ịIẵ0;1ðx1ịdx2ẳ x1 x2ỵx22 2
Iẵ0;1ðx1ị
1
0
ẳ ðx1ỵ1=2ịIẵ0;1ðx1ị:
The probability space for plant 1 outcomes is given byfR Xð ị;1 ϒX1;PX1g, where R(X1) ẳ[0,1], ϒX1 ẳ{A: Ais a Borel setR(X1)}, and PX1ðAị ẳé
x12Af1ðx1ịdx1, 8A∈ϒX1. If one were interested in the probability that plant 1 will operate at less than half of capacity on a given day, i.e., the eventAẳ[0, .5), then
PX1ðx1:5ị ẳ ð:5
0
x1þ1 2
Iẵ0;1ð ịdxx1 1ẳx21 2 þx1
2
:5
0
ẳ:375: □
Regarding other properties of marginal density functions, note that the significance of the termmarginalis only to indicate the context in which the density was derived, i.e., the marginal density of X1 is deduced from the joint density for (X1, X2). Otherwise, the MPDF has no special properties that differ from the basic properties of any other PDF.
2.5.2 n-Variate Case
The concept of a discrete MPDF can be straightforwardly generalized to the n-variate case, in which case the marginal densities may themselves be joint density functions. For example, if we have the density function f(x1,x2,x3) for the trivariate random variable (X1,X2,X3), then we may conceive of six marginal
density functions: f1ðx1ị;f2ðx2ị;f3ðx3ị; f12(x1, x2), f13(x1, x3), f23ðx2;x3ị. In general, for an n-variate random variable, there are (2n2) possible MPDFs that can be defined from knowledge of f(x1,. . .,xn). We present the n-variate generalization in the following definition. We use the notationfj1:::jmðxj1; :::;xjmị to represent the MPDF of the m-variate random variableðXj1; :::;Xjmịwith theji’s being the indices that identify the particular random vector of interest. The motivation for the definition is analogous to the argument in the bivariate case upon identifying the equivalent events ðxj1; :::;xjmị 2B and Aẳ fx:ðxj1; :::;xjmị 2B;x2Rð ịgX is left to the reader as an exercise.
Definition 2.21 Discrete Marginal Probability Density Functions
Letf(x1,. . .,xn) be the joint discrete PDF for then-dimensional random vari- able (X1,. . .,Xn). LetJẳ{j1,j2,. . .,jm}, 1 m<n, be a set of indices selected from the index setIẳ{1, 2,. . .,n}. Then the marginal density function for the m-dimensional discrete random variable (Xj1,. . .,Xjm) is given by
fj1...jm xj1;. . .;xjm
ẳ X X
xi2R Xð ị;i i2IJ
ð ị
fðx1; :::;xnị:
In other words, to define a MPDF in the general discrete case, we simply
“sum out” the variables that are not of interest in the joint density function. We are left with the marginal density function for the random variable in which we are interested. For example, ifnẳ3, so thatIẳ{1,2,3}, and ifJẳ{j1,j2}ẳ{1,3} so that I-Jẳ{2}, then Definition 2.21 indicates that the MPDF of the random variable (X1,X3) is given by
f13ðx1;x3ị ẳ X
x22RðX2ị
fðx1;x2;x3ị:
Similarly, the marginal density forx1would be defined by f1ð ị ẳx1 X
x22RðX2ị
X
x32RðX3ị
fðx1;x2;x3ị:
The concept of a continuous MPDF can be generalized to then-variate case as follows:
Definition 2.22 Continuous Marginal Probability Density Functions
Letf(x1,. . .,xn) be the joint continuous PDF for then-variate random variable (X1,. . .,Xn). LetJẳ{j1,j2,. . .,jm}, 1m<n, be a set of indices selected from the index set Iẳ{1, 2,. . .,n}. Then the marginal density function for the m-variate continuous random variable (Xj1,. . .,Xjm) is given by
fj1...jm xj1;. . .;xjm
ẳ ð1
1 ð1
1f xð 1;. . .;xnị Y
i2IJ
dxi:
In other words, to define a MPDF function in the general continuous case, we simply “integrate out” the variables in the joint density function that are not
2.5 Marginal Probability Density Functions and CDFs 83
of interest. We are left with the marginal density function for the random variables in which we are interested. An example of marginal densities in the context of a trivariate random variable will be presented in Section2.8.
2.5.3 Marginal Cumulative Distribution Functions (MCDFs)
Marginal CDFs are simply CDFs that have been derived for a subset of the random variables in Xẳ(X1,. . .,Xn) from initial knowledge of the joint PDF or joint CDF of X. For example, ordering the elements of a continuous random variable (X1,. . .,Xn) so that the first m<n random variables are of primary interest, the MCDF of (X1,. . .,Xm) can be defined as
F1:::mðb1; :::;bmị ẳPX1:::Xmðxibi;iẳ1; :::;mịðDef:of CDFị
ẳPðxibi;iẳ1; :::;m;xi<1;iẳmỵ1; :::;nịðequivalent eventsị
ẳFðb1; :::;bm;1; :::;1ịðDef:in terms of joint CDFị
ẳ ðb1
1:::
ðbm 1
ð1
1:::
ð1
1fðx1; :::;xnịdxn:::dx1ðDef:in terms of joint PDFị
ẳ ðb1
1:::
ðbm
1 f1:::mðx1; :::;xmịdxm:::dx1ðDef:in terms of marginal PDFị:
In the case of an arbitrary subset ðXj1; :::;Xjmị,m<nof the random variables
(X1,. . .,Xn), the MCDF in terms of the joint CDF or marginal PDF can be
represented as
Fj1:::jmðbj1; :::;bjmị ẳFðbị ẳ ðbj
1
1:::
ðbjm
1fj1:::jmðxj1; :::;xjmịdxjm:::dxj1
wherebji is thejith entry inbandbiẳ 1ifi2{j= 1,. . .,jm}.
Examples of marginal CDFs in the trivariate case are presented in Section2.8.
The discrete case is analogous, with summation replacing integration.