Handbook of mathematics for engineers and scienteists part 153 ppsx

A classF of subsets of the space Ω is called an algebra of sets events if∅F, ΩF, and the following conditions are satisfied: 1.. An algebra of setsF of subsets of the space Ω is called a

Events H1, , H n form a complete group of pairwise incompatible events (are

hy-potheses) if exactly one of the events necessarily occurs for each realization of the condition

setΣ, i.e., if

H1∪ · · · ∪ H n=Ω and H i ∩ H j =∅ (i≠j)

Main properties of random events:

1 A ∪ B = B ∪ A and A ∩ B = B ∩ A (commutativity).

2 (A ∪ B) ∩ C = (A ∩ C) ∪ (B ∩ C) and (A ∩ B) ∪ C = (A ∪ C) ∩ (B ∪ C) (distributivity).

3 (A ∪ B) ∪ C = A ∪ (B ∪ C) and (A ∩ B) ∩ C = A ∩ (B ∩ C) (associativity).

4 A ∪ A = A and A ∩ A = A.

5 A ∪ Ω = Ω and A ∩ Ω = A.

6 A ∪ A = Ω and A ∩ A =∅

7 ∅=Ω, Ω =∅, and A = A;

8 A \B = A ∩ B.

9 A ∪ B = A ∩ B and A ∩ B = A ∪ B (de Morgan’s laws).

20.1.1-2 Axiomatic definition of probability

In the case of an uncountable sample spaceΩ, not all subsets of Ω but only subsets belonging

to certain classes, called algebras of sets and σ-algebras, are viewed as events.

A classF of subsets of the space Ω is called an algebra of sets (events) if∅F, ΩF,

and the following conditions are satisfied:

1 If AF, then AF.

2 If AF and BF, then A ∪ BF and A ∩ B F.

The simplest example of an algebra of events is the systemF ={∅,Ω} Indeed, applying any of the operations listed above to any elements of the classF, we obtain an element of

this class: ∅∪ Ω = Ω,∅∩ Ω =∅,∅=Ω, and∅=Ω

An algebra of setsF of subsets of the space Ω is called a σ-algebra if the following

condition is satisfied: if A nF, n =1,2, , then∪

n A nF and ∩ n A nF.

The elements ofF are called random events.

The probability of an event is defined to be a single-valued real function P (A) defined

on the σ-algebra of events F and satisfying the following three axioms:

1 Nonnegativity: P (A)≥ 0for any AF.

2 Normalization: P (Ω) =1

3 Additivity: P ∪

n A n

n P (A n ), provided that A i ∩ Aj =∅whenever i≠j

A probability space is a triple ( Ω, F, P ), where Ω = Ω(ω) is a space of elementary

events,F is a σ-algebra of subsets of Ω, called random events, and P (A) is a probability

defined on the σ-algebra F.

Properties of probability:

1 The probability of an impossible event is zero; i.e., P (∅) =0

2 The probability of the event A opposite to an event A is equal to P (A) =1– P (A).

3 Probability is a bounded function; i.e.,0 ≤P (A)≤ 1

4 If an event A implies an event B (A ⊆ B), then P (A)≤P (B).

5 If events H1, , H n form a complete group of pairwise incompatible events, then

n



i=1P (H i) =1

20.1.1-3 Discrete probability space Classical definition of probability.

Suppose thatΩ ={ω1, , ω n}is a finite space The σ-algebra F of events includes all2n

subsets ofΩ To each elementary event ω iΩ (i =1,2, , n) there corresponds a number

p (ω i ), called the probability of the elementary event ω i Thus a real function satisfying the following two conditions is defined on the setΩ:

1 Nonnegativity condition: p(ω i)≥ 0for any ω iΩ

2 Normalization condition: n

i=1p (ω i) =1

The probability P (A) of an event A for any subset A ⊂ Ω is defined to be the sum of

probabilities of the elementary events that form A; i.e.,

ω i A

The triple (Ω, F, P ) thus defined is a finite discrete probability space.

A special case of the definition of probability (20.1.1.1) is the classical definition of

probability, in which all elementary events are equiprobable: p(ω1) =· · · = p(ω n) =1/n

Then the probability P (A) of an event A ={ω i1, , ω i m}is equal to the ratio of the number

of elementary events ω i contained in A (the number of outcomes that favor A) to the total

number of elementary events inΩ (the number of all possible outcomes),

P (A) = |A|

|Ω| =

m

Example 1 Let two dice be thrown Under the assumption that the elementary events are equiprobable,

find the probability of the event A that the sum of numbers shown is greater than10 Obviously, the sample space can be represented as Ω = {(i, j) : i, j =1 , 2 , 3 , 4 , 5 , 6}, where i is the number shown by the first die and j is the number shown by the second die The total number of elementary events is| Ω | = 36 The

event A corresponds to the subset A ={ ( 5 , 6 ), ( 6 , 5 ), ( 6 , 6 ) } of Ω Since |A| = 3 , formula (20.1.1.2) gives

P (A) =|A|/| Ω | = 3/36 = 1/12

20.1.1-4 Sampling without replacement

LetΔ ={1, , n}be the set of n numbers, and let ω = (i1, , i m) be an ordered sequence

of m elements of Δ Random sampling without replacement is the sampling scheme in

which

Ω ={ω = (i1, , i m ) : i kΔ, k =1, , m, and all i1, , i mare distinct} (20.1.1.3)

and all elementary events ω are equiprobable.

When calculating probabilities by formula (20.1.1.2), the following combinatorial for-mulas are often useful Suppose that a setΔ ={a1, , a n}of n elements is given Subsets

ofΔ are called combinations The number of distinct combinations of n elements of Δ taken m at a time is denoted by C n morn

m

 The following formula holds:

C m

n ≡m n= n!

Ordered sequences a i1, , a i m of distinct elements ofΔ are called arrangements, or

permutations of n elements taken m at a time The number of arrangements of m out of n

elements, i.e., the number of ordered sequences of m distinct elements selected from n elements, is denoted by A m n One has

A m

n = n(n –1) (n – m +1) = n!

Arrangements with m = n are called permutations The number of distinct permutations

P n on n elements is given by the formula

20.1.1-5 Sampling with replacement

The sampling scheme in which

Ω ={ω = (i1, , i m ) : i kΔ, k =1,2, , n} (20.1.1.7)

and all elementary events ω are equiprobable is called random sampling with replacement.

If, for random sampling of m out of n elements with replacement, no subsequent ordering is performed (i.e., each of the n elements can occur 0, 1, , or m times in

any combination), then one speaks of combinations with repetitions The number C m n of

all distinct combinations with repetitions of n elements taken m at a time is given by the

formula

C m

If, for random sampling of m out of n elements with replacement, the chosen elements are ordered in some way, then one speaks of arrangements with repetitions The number

A m

n of distinct arrangements with repetitions of n elements taken m at a time is given by

the formula

A m

Suppose that a set of n elements contains k distinct elements, of which the first occurs

n1times, the second occurs n2times, , and the kth occurs n k times, n1+· · · + n k = n Permutations of n elements of this set are called permutations with repetitions on n elements The number P n (n1, , n k ) of permutations with repetitions on n elements is given by the

formula

P n (n1, , n k) = n!

Example 2 Consider the set{1 , 2 , 3}of n =3elements The elements of this set give P3 = 3 ! = 6

permutations: ( 1 , 2 , 3 ), ( 1 , 3 , 2 ), ( 2 , 1 , 3 ), ( 2 , 3 , 1 ), ( 3 , 1 , 2 ), ( 3 , 2 , 1) For m =2 , there are

1 C2≡32= 3 combinations without repetitions [( 1 , 2 ), ( 1 , 3 ), ( 2 , 3 )].

2 C2= C2+2–1= C2 = 6 combinations with repetitions [( 1 , 2 ), ( 1 , 3 ), ( 2 , 3 ), ( 1 , 1 ), ( 2 , 2 ), ( 3 , 3 )].

3 A2= 3 arrangements without repetitions [( 1 , 2 ), ( 1 , 3 ), ( 2 , 3 ), ( 2 , 1 ), ( 3 , 2 ), ( 3 , 1 )].

4 A23= 3 2 = 9 arrangements with repetitions [( 1 , 2 ), ( 1 , 3 ), ( 2 , 3 ), ( 2 , 1 ), ( 3 , 2 ), ( 3 , 1 ), ( 1 , 1 ), ( 2 , 2 ), ( 3 , 3 )].

20.1.1-6 Geometric definition of probability

LetΩ be a set of positive finite measure μ(Ω) in the n-dimensional Euclidean space, and let the σ-algebra F consist of all measurable (i.e., having a measure) subsets A ⊆ Ω The geometric probability of an event A is defined to be the ratio of the measure of A to that

ofΩ,

P (A) = μ (A)

The notion of geometric probability is not invariant under transformations of the domain

Ω and depends on how the measure μ(A) is introduced.

Example 3 A point is randomly thrown into a disk of radius R =1 Find the probability of the event that

the point lands in the disk of radius r = 12 centered at the same point.

First method Let A be the event that the point lands in the smaller disk We find the probability P (A) as

the ratio of the area of the smaller disk to that of the larger disk:

P (A) = πr

2

πR2 = 1

4.

Second method Consider the polar coordinate system in which the position of a point is determined by

the angle ϕ between the position vector of the point and the axis OX and by the distance ρ from the point

to the origin Since all points equidistant from the center either belong or do not belong to the smaller disk simultaneously, it follows that the probability of landing in this disk is equal to the ratio of the radii:

P (A) = r

R = 1

2.

Thus we have obtained two different answers in the same problem The cause is that the notion of geometric probability is not invariant under transformations of the domainΩ and depends on how the measure μ(A) is

introduced.

20.1.2 Conditional Probability and Simplest Formulas

20.1.2-1 Probability of the union of events

The probability of realization of at least one of two events H1and H2is given by the formula

P (H1∪ H2) = P (H1) + P (H2) – P (H1∩ H2). (20.1.2.1)

In particular, for H1∩ H2=∅, we have

The probability of realization of at least one of n events is given by the formula

P (H1∪ · · · ∪ H n) =

n



k=1

P (H k) – 

1≤k1 <k2 ≤n

P (H k1∩ H k2)

1≤k1 <k2 <k3 ≤n

P (H k1∩ H k2 ∩ H k3) –· · · + (–1)n–1P (H

1∩ · · · ∩ H n) (20.1.2.3)

20.1.2-2 Conditional probability

The conditional probability P (A| H ), or P H (A), of an event A given the occurrence of some other event H is defined by the formula

P (A| H) = P (A ∩ H)

The conditional probability P (A| H ) can be treated as the probability of the event A under the condition that the event H occurs.

Relation (20.1.2.4) can be written as the “probability multiplication theorem”

P (A ∩ H) = P (H) P (A|H) (20.1.2.5) The formula

P (A1∩ · · · ∩ A n ) = P (A1) P (A2|A1) P (A3|A1∩ A2) P (A n|A1∩ · · · ∩ A n–1)

is a generalization of (20.1.2.5)

20.1.2-3 Independence of events.

Two random events A and B are said to be statistically independent if the conditional probability of A, given B, coincides with the unconditional probability of A,

Random events A1, , A nare jointly statistically independent if the relation

P

:m k=1

A i k



=

m



k=1

holds whenever1 ≤i1 <· · · < im ≤n and m≤n

The pairwise independence of the events A i and A j for all i≠j (i, j =1,2, , n) does

not imply that the events A1, , A nare jointly independent

Example 1 Suppose that the experiment is to draw one of four balls Let three of them be labeled by the

numbers 1 , 2 , and 3, and let the fourth ball bear all these numbers By A i (i =1 , 2 , 3 ) we denote the event that

the chosen ball bears the number i Are the events A1, A2, and A3 dependent?

Since each number is encountered twice, P (A1) = P (A2) = P (A3 ) = 1/2 Since any two distinct numbers

are present only on one of the balls, we have P (A1A2) = P (A2A3) = P (A1A3) = 1/4 , and hence the events

A1, A2, and A3are pairwise independent All three distinct numbers are present only on one of the balls, and

P (A1A2A3) = 1/4 ≠P (A1) P (A2) P (A3) = 1/8

Thus we see that the events A1, A2, and A3are jointly dependent, even though they are pairwise independent.

20.1.2-4 Total probability formula Bayes’ formula

Suppose that a complete group of pairwise incompatible events H1, , H n is given and

the unconditional probabilities P (H1), , P (H n), as well as the conditional probabilities

P (A| H1), , P (A| H n ) of an event A, are known Then the probability of A can be

determined by the total probability formula

P (A) =

n



k=1

Example 2 There are two urns; the first urn contains a white and b black balls, and the second urn contains

c white and d black balls We take one ball from the first urn and put it into the second urn After this, we draw

one ball from the second urn Find the probability of the event that this ball is white.

Let A be the event of drawing a white ball Consider the following complete group of events:

H1, a white ball is taken from the first urn and put into the second urn.

H2, a black ball is taken from the first urn and put into the second urn Obviously,

P (H1) = a

a + b, P (H2) =

b

a + b; P (A|H1) = c+1

c + d +1, P (A|H2) =

c

c + d +1.

Now by the total probability formula (20.1.2.8) we obtain

P (A) = P (H1) P (A|H1) + P (H2) P (A|H2) = a

a + b

c+ 1

c + d +1+

b

a + b

c

c + d +1.

If it is known that the event A has occurred but it is unknown which of the events

H1, , H n has occurred, then Bayes’ formula is used:

P (H k|A) = P (H k ) P (A| H k)

Example 3 The urn contains one ball It is known that this ball is either white or black We put a black

ball into the urn, then thoroughly shuffle the balls and draw one of the balls, which turns out to be black Find the probability of the event that the ball remaining in the urn is black.

We consider the following random events that form a complete group of pairwise incompatible events:

H1, the urn initially contained a white ball.

H2, the urn initially contained a black ball.

We have P (H1) = P (H2 ) = 1/2 Suppose that the event A is that a black ball is drawn Then P (A|H1) = 1/2

and P (A|H2) = 1 The desired probability of the event H2 |Ais determined by Bayes’ formula:

P (H2 |A) = P (H2) P (A|H2)

1 × 1 2

1 × 1

2 +12 × 1 2

= 2

3.

20.1.3 Sequences of Trials

20.1.3-1 Mathematical model of a sequence of n independent trials.

Trials in which events occurring in distinct trials are independent are said to be independent Each trial S k can be viewed as a probability space (Ωk,F k , P k) Independent trials are described by the probability space (Ω, F, P ) that is the direct product of the probability

spaces (Ωk,F k , P k ) Here the probability of each event A of the form A = A1×· · ·×A nis

defined as P (A) = P (A1) P (A n)

A sequence of n independent trials is also called a Bernoulli scheme Let p k = P (A k)

Then the probability of the event that n1 events A1, n2 events A2, , and n k events A k occur in n independent trials is equal to

P n (n1, , n k) = n n!

1! n k!p

n1

1 p n k k. (20.1.3.1)

The probability (20.1.3.1) is the coefficient of x n11 x n k

k in the expansion of the

poly-nomial (p1x1+· · · + p k x k)n in powers of x1, , x k.

The probability P n (n1, , n k) can be found by the technique of generating functions,

since this probability is the coefficient of z n1

1 z k n k in the generating function

ϕ X (z1, , z k ) = (p1z1+· · · + p k z k)n.

20.1.3-2 Bernoulli process

The special case of the Bernoulli scheme with N =2 is called the Bernoulli process In this case, some event A occurs with probability p = P (A) (the probability of “success”) and does not occur with probability q = P (A) =1– P (A) =1– p (the probability of “failure”) in each trial If μ n is the number of occurrences of the event A (the number of “successes”) in

n independent Bernoulli trials, then the probability that A occurs exactly k times is given

by the formula

P (μ n = k) = p n (k) = C n k p k(1– p) n–k (k =0,1, , n) (20.1.3.2)

Relation (20.1.3.2) is called the Bernoulli formula (binomial distribution).

The probability that the event occurs at least m times in n independent trials is calculated

by the formula

p n (k≥m) =

n



k=m

p n (k) =1–

m–1



k=0

p n (k).

The probability that the event occurs at least once in n independent trials is calculated

by the formula

p n (k≥ 1) =1– (1– p) n

The number n of independent trials necessary for the event to occur at least once with probability at least P is given by the formula

n≥ ln(1– P ) ln(1– p).

Example (Banach’s problem). A smoker mathematician has two matchboxes on him, each of which

initially contains exactly n matches Each time he needs to light a cigarette, he selects a matchbox at random Find the probability of the event that as the mathematician takes out an empty box for the first time, precisely k matches will be left in the other box (k≤n).

The mathematician has taken matches 2n – k times, n out of them from the box that is eventually empty.

This scheme corresponds to the scheme of 2n – k independent Bernoulli trials with n “successes.” The

probability of a “success” in a single trial is equal to 0 5 The desired probability can be found by the formula

P (μ2n–k = n) = p2n–k (n) = C n2n–k p ( 1– p) n–k = C n2n–k

1

2

2n–k .

20.1.3-3 Limit theorems for Bernoulli process

It is very difficult to use Bernoulli’s formula (20.1.3.2) for large n and m In this case, one has to use approximate formulas for calculating p n (k) with desired accuracy.

Poisson formula If the number of independent trials increases unboundedly (n → ∞)

and the probability p simultaneously decays (p →0) so that their product np is a constant

(np = λ = const), then the probability P (μ n = k) = p n (k) satisfies the limit relation

lim

n→∞ P (μ n = k) =

λ k

k!e

Local theorem of de Moivre–Laplace Suppose that n → ∞, p = const,0< p <1, and

0< c1 ≤x n,k = (k – np)[np(1 – p)]–1 2≤c2<∞; then

√2 πnp(1– p) e

–x2

n,k / [1+ O(1 / √

uniformly with respect to x n,k [c1, c2]

Integral theorem of de Moivre–Laplace Suppose that n → ∞ and p = const,0< p <1; then

P

*

x1< √ μ n – np

np(1– p) < x2

+

→ √1

2π

 x2

x1

e–t2/ dt (20.1.3.5)

uniformly with respect to x1and x2

The approximate formula (20.1.3.3) is normally used for n ≥ 50 and np≤ 10 The

approximate formulas (20.1.3.4) and (20.1.3.5) are used for np (1 – p) > 9 The error in formulas (20.1.3.4) and (20.1.3.5) can increase owing to the fact that the prelimit distribution

is discrete This error is of the order of O([np(1 – p)]–1 2)

The limit expression in (20.1.3.4) can readily be expressed via the probability density

ϕ (x) = √1

2π e

–x2/

of the standard normal distribution, and the limit expression in (20.1.3.5) can be expressed

in terms of the cumulative distribution function

Φ(x) = √1

2π

 x –∞ e

–t2/ dt

of the standard normal distribution