Handbook of mathematics for engineers and scienteists part 162 pps

, X n lies in this region, then the null hypothesis H0is rejected and the alternative hypothesis H1is accepted.. , X n lies in this region, then the null hypothesis H0 is accepted and th

Example 1 The hypothesis that the theoretical distribution function is normal with zero expectation is a

parametric hypothesis.

Example 2 The hypothesis that the theoretical distribution function is normal is a nonparametric

hypoth-esis.

Example 3 The hypothesis H0that the variance of a random variable X is equal to σ2, i.e., H0: Var {X} =

σ2, is simple For an alternative hypothesis one can take one of the following hypotheses: H1: Var {X}> σ2 (composite hypothesis), H1: Var {X}< σ2(composite hypothesis), H1: Var {X} ≠σ2(composite hypothesis),

or H1: Var {X}= σ2(simple hypothesis).

21.3.1-2 Statistical test Type I and Type II errors

1◦ A statistical test (or simply a test) is a rule that permits one, on the basis of a sample

X1, , X n alone, to accept or reject the null hypothesis H0(respectively, reject or accept

the alternative hypothesis H1) Any test is characterized by two disjoint regions:

1 The critical region W is the region in the n-dimensional space Rn such that if the sample X1, , X n lies in this region, then the null hypothesis H0is rejected (and the

alternative hypothesis H1is accepted)

2 The acceptance region W (W =Rn \W ) is the region in the n-dimensional space R n such that if the sample X1, , X n lies in this region, then the null hypothesis H0 is

accepted (and the alternative hypothesis H1is rejected)

2◦ Suppose that there are two hypotheses H0and H1, i.e., two disjoint subsetsΓ0andΓ1

are singled out from the set of all distribution functions We consider the null hypothesis

H0that the sample X1, , X n is drawn from a population with theoretical distribution function F (x) belonging to the subsetΓ0 and the alternative hypothesis that the sample is

drawn from a population with theoretical distribution function F (x) belonging to the subset

Γ1 Suppose, also, that a test for verifying these hypotheses is given; i.e., the critical region

W and the admissible region W are given Since the sample is random, there may be errors

of two types:

i) Type I error is the error of accepting the hypothesis H1(the hypothesis H0is rejected),

while the null hypothesis H0is true

ii) Type II error is the error of accepting the hypothesis H0(the hypothesis H1is rejected),

while the alternative hypothesis H1is true

The probability α of Type I error is called the false positive rate, or size of the test, and

is determined by the formula

α = P [(X1, , X n)W]

=

⎧

⎨

⎩



P (X1)P (X2) P (X n) in the discrete case,



.



p(X1)p(X2) p(X n ) dx1 dx n in the continuous case;

here P (x) or p(x) is the distribution series or the distribution density of the random vari-able X under the assumption that the null hypothesis H0 is true, and the summation or

integration is performed over all points (x1, , x n) W The number1– α is called the

specificity of the test If the hypothesis H0is composite, then the size α = α[F (x)] depends

on the actual theoretical distribution function F (x)Γ0 If, moreover, H0is a parametric

hypothesis, i.e.,Γ0 is a parametric family of distribution functions F (x; θ) depending on

the parameter θ with rangeΘ0 Θ, where Θ is the region of all possible values θ, then,

instead of notation α[F (x)], the notation α(θ) is used under the assumption that θ Θ0.

The probability 2β of Type II error is called the false negative rate The power β =1– 2β

of the test is the probability that Type II error does not occur, i.e., the probability of rejecting

the false hypothesis H0and accepting the hypothesis H1 The test power is determined by

the same formula as the test specificity, but in this case, the distribution series P (x) or the density function p(x) are taken under the assumption that the alternative hypothesis H1 is

true If the hypothesis H1is composite, then the power β = β[F (x)] depends on the actual theoretical distribution function F (x) Γ1 If, moreover, H1 is a parametric hypothesis,

then, instead of the notation β[F (x)], the notation β(θ) is used under the assumption θΘ1,

whereΘ1is the range of the unknown parameter θ under the assumption that the hypothesis

H1is true.

The difference between the test specificity and the test power is that the specificity

1– α[F (x)] is determined for the theoretical distribution functions F (x)  Γ0, and the

power β(θ) is determined for the theoretical distribution functions F (x)Γ1

3◦ Depending on the form of the alternative hypothesis H1, the critical regions are

classi-fied as one-sided (right-sided and left-sided) and two-sided:

1 The right-sided critical region (Fig 21.3a) consisting of the interval (t Rcr;∞), where the boundary t Rcris determined by the condition

P [S(X1, , X n ) > t Rcr] = α; (21.3.1.1)

x

Figure 21.3 Right-sided (a), left-sided (b), and two-sided (c) critical region.

2 The left-sided critical region (Fig 21.3b) consisting of the interval (–∞; t L

cr), where the

boundary t Lcris determined by the condition

P [S(X1, , X n ) < t Lcr] = α; (21.3.1.2)

3 The two-sided critical region (Fig 21.3c) consisting of the intervals (– ∞; t L

cr) and

(t Rcr;∞), where the points t L

crand t Rcrare determined by the conditions

P [S(X1, , X n ) < t Lcr] = α

2 and P[S(X1, , X n ) > t Rcr] = α

2. (21.3.1.3)

21.3.1-3 Simple hypotheses

Suppose that a sample X1, , X nis selected from a population with theoretical distribution

function F (x) about which there are two simple hypotheses, the null hypothesis H0: F (x) =

F0(x) and the alternative hypothesis H1: F (x) = F1(x), where F0(x) and F1(x) are known

distribution functions In this case, there is a test that is most powerful for a given size α;

this is called the likelihood ratio test The likelihood ratio test is based on the statistic called the likelihood ratio,

Λ = Λ(X1, , X n) = L1(X1, , X n)

L0(X1, , X n), (21.3.1.4)

where L0(X1, , X n) is the likelihood function under the assumption that the null

hypoth-esis H0 is true, and L1(X1, , X n) is the likelihood function under the assumption that

the alternative hypothesis H1is true

The critical region W of the likelihood ratio test consists of all points (x1, , x n) for whichΛ(X1, , X n ) is larger than a critical value C.

NEYMAN–PEARSON LEMMA Of all tests of given size α testing two simple hypotheses

H0and H1, the likelihood ratio test is most powerful

21.3.1-4 Sequential analysis Wald test

Sequential analysis is the method of statistical analysis in which the sample size is not

fixed in advance but is determined in the course of experiment The ideas of sequential analysis are most often used for testing statistical hypotheses Suppose that observations

X1, X2, are performed successively; after each trial, one can stop the trials and accept one of the hypotheses H0 and H1 The hypothesis H0 is that the random variables X i have the probability distribution with density p0(x) in the continuous case or the probability distribution determined by probabilities P0(X i ) in the discrete case The hypothesis H1

is that the random variables X i have the probability distribution with density p1(x) in the continuous case or the probability distribution determined by probabilities P1(X i) in the discrete case

WALD TEST Of all tests with given size α, power β, finite mean number N0 of

observations under the assumption that the hypotheses H0is true, and finite mean number N1

of observations under the assumption that the hypothesis H1is true, the sequential likelihood

ratio test minimizes both N0and N1

The decision in the Wald test is made as follows One specifies critical values A and B,

0< A < B The result X1of the first observation determines the logarithm of the likelihood ratio

λ(X1) =

⎧

⎪

⎪

lnP1(X1)

P0(X1) in the discrete case,

lnp1(X1)

p0(X1) in the continuous case.

If λ(X1)≥B, then the hypothesis H1is accepted; if λ(X1)≤A, then the hypothesis H1is

accepted; and if A < λ(X1) < B, then the second trial is performed The logarithm of the

likelihood ratio

λ(X1, X2) = λ(X1) + λ(X2)

is again determined If λ(X1, X2)≥B, then the hypothesis H1is accepted; if λ(X1, X2)≤A, then the hypothesis H1is accepted; and if A < λ(X1, X2) < B, then the third trial is performed.

The logarithm of the likelihood ratio

λ(X1, X2, X3) = λ(X1) + λ(X2) + λ(X3)

is again determined, and so on The graphical scheme of trials is shown in Fig 21.4

For the size α and the power β of the Wald test, the following approximate estimates

hold:

α≈ 1– e A

e B – e A, β ≈ e B(1– e A)

e B – e A .

B

(1, ( ))λ X

(2, ( , ))λ X X

Acceptance region for hypothesis H

Acceptance region for hypothesis H

1

0

(3, ( , , ))λ X X X λ

N

1

1

1

1

2

2 3

Figure 21.4 The graphical scheme of the Wald test.

For given α and β, these estimates result in the following approximate expressions for the critical values A and B:

A≈ln β

1– α, B≈lnβ

α

For the mean numbers N0and N1of observations, the following approximate estimates

hold under the assumptions that the hypothesis H0or H1is true:

N0≈ αB+ (1– α)A

M[λ(X)|H0] , N1≈ βB+ (1– β)A

M[λ(X)|H1] ,

where

E{λ(X)|H0}=

L



i=1

lnP1(b i)

P0(b i)P0(b i), E{λ(X)|H1}=

L



i=1

lnP1(b i)

P0(b i)P1(b i)

in the discrete case and

E{λ(X)|H0}=

–∞ln

p1(x)

p0(x) p0(x) dx, E{λ(X)|H1}=

–∞ln

p1(x)

p0(x) p1(x) dx

in the continuous case

21.3.2 Goodness-of-Fit Tests

21.3.2-1 Statement of problem

Suppose that there is a random sample X1, , X n drawn from a population X with

unknown theoretical distribution function It is required to test the simple nonparametric

hypothesis H0: F (x) = F0(x) against the composite alternative hypothesis H1: F (x)≠F0(x), where F0(x) is a given theoretical distribution function There are several methods for

solving this problem that differ in the form of the measure of discrepancy between the empirical and hypothetical distribution laws For example, in the Kolmogorov test (see Paragraph 21.3.2-2) and the Smirnov test (see Paragraph 21.3.2-3), this measure is a function

of the difference between the empirical distribution function F ∗ (x) and the theoretical distribution function F (x), i.e.,

ρ = ρ[F ∗ (x) – F (x)];

and in the χ2-test, this measure is a function of the difference between the theoretical

probabilities p T i = P (H i ) of the random events H1, , H Land their relative frequencies

p ∗

i = n i /n, i.e.,

ρ = ρ(p T i – p ∗ i)

21.3.2-2 Kolmogorov test.

To test a hypothesis concerning the distribution law, the statistic

ρ = ρ(X1, , X n) =√

n sup

–∞<x<∞|F ∗ (x) – F (x)| (21.3.2.1)

is used to measure the compatibility (goodness of fit) of the hypothesis in the Kolmogorov

test A right-sided region is chosen to be the critical region in the Kolmogorov test For

a given size α, the boundary t Rcr of the right-sided critical region can be found from the relation

t R

cr = F–1(1– α).

Table 21.1 presents values depending on the size and calculated by formula (21.3.2.1)

TABLE 21.1 Boundary of right-sided critical region

α 0 5 0 1 0 05 0 01 0 001

t Rcr 0 828 1 224 1 385 1 627 1 950

As n → ∞, the distribution of the statistic ρ converges to the Kolmogorov distribution and the boundary t Rcr of the right-sided critical region coincides with the (1– α)-quantile

k1 –αof the Kolmogorov distribution.

The advantages of the Kolmogorov test are its simplicity and the absence of complicated calculations But this test has several essential drawbacks:

1 The use of the test requires considerable a priori information about the theoretical law

of distribution; i.e., in addition to the form of the distribution law, one must know the values of all parameters of the distribution

2 The test deals only with the maximal deviation of the empirical distribution function from the theoretical one and does not take into account the variations of this deviation

on the entire range of the random sample

21.3.2-3 Smirnov test (ω2-test)

In contrast to the Kolmogorov test, the Smirnov test takes the mean value of a function

of the difference between the empirical and theoretical distribution functions on the entire domain of the distribution function to be the measure of discrepancy between the empirical distribution function and the theoretical one; this eliminates the drawback of the Kolmogorov test

In the general case, the statistic

ω2= ω2(X

1, , X n) =

–∞ [F

∗ (x) – F (x)]2dF (x) (21.3.2.2)

is used Using the series X1∗ , , X n ∗ of order statistics, one can rewrite the statistic ω2in the form

ω2= 1

n

n



i=1

*

F ∗ (X ∗

i) – 2i–1

2n

+2

12n2. (21.3.2.3)

A right-sided region is chosen to be the critical region in the Smirnov test For a given

size α, the boundary t Rcrof the right-sided critical region can be found from the relation

t R

cr = F–1(1– α). (21.3.2.4)

Table 21.2 presents the values of t Rcr depending on the size and calculated by formula (21.3.2.4)

TABLE 21.2 Boundary of right-sided critical region

α 0 5 0 1 0 05 0 01 0 001

t Rcr 0 118 0 347 0 461 0 620 0 744

As n → ∞, the distribution of the statistic ω2 converges to the ω2-distribution and

the boundary t Rcr of the right-sided critical region coincides with the (1– α)-quantile of an

ω2-distribution.

21.3.2-4 Pearson test (χ2-test)

1◦ The χ2-test is used to measure the compatibility (goodness of fit) of the theoretical

probabilities p k = P (H k ) of random events H1, , H L with their relative frequencies

p ∗

k = n k /n in a sample of n independent observations The χ2-test permits comparing the

theoretical distribution of the population with its empirical distribution

The goodness of fit is measured by the statistic

χ2=L

k=1

(n k – np k)2

np k =

L



k=1

n2

k

np k – n, (21.3.2.5)

whose distribution as n → ∞ tends to the chi-square distribution with v = L –1degrees of

freedom According to the χ2-test, there are no grounds to reject the theoretical probabilities

for a given confidence level γ if the inequality χ2 < χ2γ (v) holds, where χ2γ (v) is the γ-quantile of a χ2-distribution with v degrees of freedom For v > 30, instead of the chi-square distribution, one can use the normal distribution of the random variable

2χ2with

expectation√

2v–1and variance1

Remark. The condition n k> 5is a necessary condition for the χ2-test to be used.

2◦ χ2-test with estimated parameters.

Suppose that X1, , X n is a sample drawn from a population X with unknown distribution function F (x) We test the null hypothesis H0 stating that the population is

distributed according to the law with the distribution function F (x) equal to the function

F0(x), i.e., the null hypothesis H0: F (x) = F0(x) is tested Then the alternative hypothesis

is H1 : F (x)≠F0(x).

In this case, the statistic (21.3.2.5) as n → ∞ tends to the chi-square distribution with

v = L – q –1degrees of freedom, where q is the number of estimated parameters Thus, for example, q =2for the normal distribution and q =1for the Poisson distribution The null

hypothesis H0for a given confidence level α is accepted if χ2< χ2α (L – q –1)

21.3.3 Problems Related to Normal Samples

21.3.3-1 Testing hypotheses about numerical values of parameters of normal

distribution

Suppose that a random sample X1, , X n is drawn from a population X with normal

distribution Table 21.3 presents several tests for hypotheses about numerical values of the parameters of the normal distribution

TABLE 21.3

Several tests related to normal populations with parameters (a, σ2)

No. Hypothesisto be tested Test statistic Statistic distribution for a given sizeCritical region

1 H0: a = a0,

H1: a≠a0 U = m

∗ – a σ

√

n standard normal |U|> u1–α/2

2 H0: a≤a0,

H1: a > a0 U = m

∗ – a σ

√

n standard normal U > u1 –α

3 H0: a≥a0,

H1: a < a0 U =

m ∗ – a σ

√

n standard normal U > –u1–α

4 H0: a = a0 ,

H1: a≠a0 T = n

s2∗(m ∗ – a) n–t-distribution with1 degrees of freedom |T|> t1–α/2

5 H0: a≤a0,

H1: a > a0 T =

n

s2∗(m ∗ – a) n–t-distribution with1 degrees of freedom T > t1–α

6 H0: a≥a0,

H1: a < a0 T =

n

s2∗(m ∗ – a) n–t-distribution with1 degrees of freedom T > –t1–α

7 H0: σ2= σ2,

H1: σ2≠σ2 χ

2 = s

2∗

σ2(n –1 ) χ2-distribution with

n– 1 degrees of freedom

χ2α/2 > χ2–α/2,

χ2> χ2–α/2

8 H0: σ2≤σ2,

H1: σ2> σ2 χ

2 = s

2∗

σ2(n –1 ) χ2-distribution with

n– 1 degrees of freedom χ

2> χ2–α

9 H0: σ2≥σ2,

H1: σ2< σ2 χ

2 = s

2∗

σ2(n –1 ) χ2-distribution with

n– 1 degrees of freedom χ2< χ2α

Remark 1. In items 1–6 σ2is known.

Remark 2. In items 1–3 u α is α-quantile of standard normal distribution.

21.3.3-2 Goodness-of-fit tests

Suppose that a sample X1, , X n is drawn from a population X with theoretical distribu-tion funcdistribu-tion F (x) It is required to test the composite null hypothesis, H0: F (x) is normal with unknown parameters (a, σ2), against the composite alternative hypothesis, H1: F (x)

is not normal Since the parameters a and σ2are decisive for the normal law, the sample

mean m ∗ (or X) and the adjusted sample variance s2 are used to estimate these parameters

1◦ Romanovskii test To test the null hypothesis, the following statistic (Romanovskii

ratio) is used:

ρrom = ρrom(X1, , X n) = χ

2(m) – m

√

2m , (21.3.3.1)

v = L – q –1degrees of freedom, where q is the number of estimated parameters Thus, for example, q =2for the normal distribution and q = 1for the Poisson distribution The... Samples

21.3.3-1 Testing hypotheses about numerical values of parameters of normal

distribution

Suppose that a random sample X1, , X n is drawn... with normal

distribution Table 21.3 presents several tests for hypotheses about numerical values of the parameters of the normal distribution

TABLE 21.3

Several