applied probability 6 2

GIÁO TRÌNH TOÁN XÁC SUẤT TỪ ĐẠI HỌC CHÂU ÂU, CHUẨN ĐỀ TOÁN XÁC SUẤT, CUNG CẤP GIÁO TRÌNH THAM KHẢO CHUẨN CHO CẤP BẬC ĐẠI HỌC SAU ĐẠI HỌC. ĐẶC BIỆT PHÙ HỢP VỚI CÁC BẠN ĐANG MONG MUỐN TRỞ THÀNH DU HỌC SINH MĨ, ANH, CANADA. HOẶC MONG MUỐN TÌM HIỂU LÝ THUYẾT TOÁN XÁC XUẤT QUỐC TÊ

Applied Probability

By:

Paul E Pfeiffer

Creative Commons Attribution 3.0 license (http://creativecommons.org/licenses/by/3.0/).

Collection structure revised: August 31, 2009

PDF generated: October 26, 2012

For copyright and attribution information for the modules contained in this collection, see p 618

Table of Contents

Preface to Pfeier Applied Probability 1

1 Probability Systems 1.1 Likelihood 5

1.2 Probability Systems 9

1.3 Interpretations 14

1.4 Problems on Probability Systems 19

Solutions 23

2 Minterm Analysis 2.1 Minterms 25

2.2 Minterms and MATLAB Calculations 34

2.3 Problems on Minterm Analysis 43

Solutions 48

3 Conditional Probability 3.1 Conditional Probability 61

3.2 Problems on Conditional Probability 70

Solutions 74

4 Independence of Events 4.1 Independence of Events 79

4.2 MATLAB and Independent Classes 83

4.3 Composite Trials 89

4.4 Problems on Independence of Events 95

Solutions 101

5 Conditional Independence 5.1 Conditional Independence 109

5.2 Patterns of Probable Inference 114

5.3 Problems on Conditional Independence 123

Solutions 129

6 Random Variables and Probabilities 6.1 Random Variables and Probabilities 135

6.2 Problems on Random Variables and Probabilities 148

Solutions 152

7 Distribution and Density Functions 7.1 Distribution and Density Functions 161

7.2 Distribution Approximations 174

7.3 Problems on Distribution and Density Functions 184

Solutions 189

8 Random Vectors and joint Distributions 8.1 Random Vectors and Joint Distributions 195

8.2 Random Vectors and MATLAB 202

8.3 Problems On Random Vectors and Joint Distributions 211

Solutions 215

9 Independent Classes of Random Variables 9.1 Independent Classes of Random Variables 231

9.2 Problems on Independent Classes of Random Variables 242

Solutions 247

10 Functions of Random Variables 10.1 Functions of a Random Variable 257

10.2 Function of Random Vectors 263

10.3 The Quantile Function 278

10.4 Problems on Functions of Random Variables 287

Solutions 294

11 Mathematical Expectation 11.1 Mathematical Expectation: Simple Random Variables 301

11.2 Mathematical Expectation; General Random Variables 309

11.3 Problems on Mathematical Expectation 326

Solutions 334

12 Variance, Covariance, Linear Regression 12.1 Variance 345

12.2 Covariance and the Correlation Coecient 356

12.3 Linear Regression 361

12.4 Problems on Variance, Covariance, Linear Regression 366

Solutions 374

13 Transform Methods 13.1 Transform Methods 385

13.2 Convergence and the central Limit Theorem 395

13.3 Simple Random Samples and Statistics 404

13.4 Problems on Transform Methods 408

Solutions 412

14 Conditional Expectation, Regression 14.1 Conditional Expectation, Regression 419

14.2 Problems on Conditional Expectation, Regression 437

Solutions 444

15 Random Selection 15.1 Random Selection 455

15.2 Some Random Selection Problems 464

15.3 Problems on Random Selection 476

Solutions 482

16 Conditional Independence, Given a Random Vector 16.1 Conditional Independence, Given a Random Vector 495

16.2 Elements of Markov Sequences 503

16.3 Problems on Conditional Independence, Given a Random Vector 523

Solutions 527

17 Appendices 17.1 Appendix A to Applied Probability: Directory of m-functions and m-procedures 531

17.2 Appendix B to Applied Probability: some mathematical aids 592

17.3 Appendix C: Data on some common distributions 596

17.4 Appendix D to Applied Probability: The standard normal distribution 598

17.5 Appendix E to Applied Probability: Properties of mathematical expectation 599

17.6 Appendix F to Applied Probability: Properties of conditional expectation, given a random vector 601

17.7 Appendix G to Applied Probability: Properties of conditional independence, given a random vector 602

17.8 Matlab les for "Problems" in "Applied Probability" 603

Index 615Attributions 618

Preface to Pfeier Applied Probability

The course

This is a "rst course" in the sense that it presumes no previous course in probability The units aremodules taken from the unpublished text: Paul E Pfeier, ELEMENTS OF APPLIED PROBABILITY,USING MATLAB The units are numbered as they appear in the text, although of course they may be used

in any desired order For those who wish to use the order of the text, an outline is provided, with indication

of which modules contain the material

The mathematical prerequisites are ordinary calculus and the elements of matrix algebra A few standardseries and integrals are used, and double integrals are evaluated as iterated integrals The reader who canevaluate simple integrals can learn quickly from the examples how to deal with the iterated integrals used

in the theory of expectation and conditional expectation Appendix B (Section 17.2) provides a convenientcompendium of mathematical facts used frequently in this work And the symbolic toolbox, implementingMAPLE, may be used to evaluate integrals, if desired

In addition to an introduction to the essential features of basic probability in terms of a precise matical model, the work describes and employs user dened MATLAB procedures and functions (which werefer to as m-programs, or simply programs) to solve many important problems in basic probability Thisshould make the work useful as a stand alone exposition as well as a supplement to any of several currenttextbooks

mathe-Most of the programs developed here were written in earlier versions of MATLAB, but have been revisedslightly to make them quite compatible with MATLAB 7 In a few cases, alternate implementations areavailable in the Statistics Toolbox, but are implemented here directly from the basic MATLAB program,

so that students need only that program (and the symbolic mathematics toolbox, if they desire its aid inevaluating integrals)

Since machine methods require precise formulation of problems in appropriate mathematical form, it

is necessary to provide some supplementary analytical material, principally the so-called minterm analysis.This material is not only important for computational purposes, but is also useful in displaying some of thestructure of the relationships among events

A probability model

Much of "real world" probabilistic thinking is an amalgam of intuitive, plausible reasoning and of statisticalknowledge and insight Mathematical probability attempts to to lend precision to such probability analysis

by employing a suitable mathematical model, which embodies the central underlying principles and structure

A successful model serves as an aid (and sometimes corrective) to this type of thinking

Certain concepts and patterns have emerged from experience and intuition The mathematical lation (the mathematical model) which has most successfully captured these essential ideas is rooted inmeasure theory, and is known as the Kolmogorov model, after the brilliant Russian mathematician A.N.Kolmogorov (1903-1987)

formu-1 This content is available online at <http://cnx.org/content/m23242/1.8/>.

Available for free at Connexions <http://cnx.org/content/col10708/1.6>

One cannot prove that a model is correct Only experience may show whether it is useful (and notincorrect) The usefulness of the Kolmogorov model is established by examining its structure and show-ing that patterns of uncertainty and likelihood in any practical situation can be represented adequately.Developments, such as in this course, have given ample evidence of such usefulness.

The most fruitful approach is characterized by an interplay of

• A formulation of the problem in precise terms of the model and careful mathematical analysis of theproblem so formulated

• A grasp of the problem based on experience and insight This underlies both problem formulationand interpretation of analytical results of the model Often such insight suggests approaches to theanalytical solution process

MATLAB: A tool for learning

In this work, we make extensive use of MATLAB as an aid to analysis I have tried to write the MATLABprograms in such a way that they constitute useful, ready-made tools for problem solving Once the userunderstands the problems they are designed to solve, the solution strategies used, and the manner in whichthese strategies are implemented, the collection of programs should provide a useful resource

However, my primary aim in exposition and illustration is to aid the learning process and to deepeninsight into the structure of the problems considered and the strategies employed in their solution Severalfeatures contribute to that end

1 Application of machine methods of solution requires precise formulation The data available and thefundamental assumptions must be organized in an appropriate fashion The requisite discipline forsuch formulation often contributes to enhanced understanding of the problem

2 The development of a MATLAB program for solution requires careful attention to possible solutionstrategies One cannot instruct the machine without a clear grasp of what is to be done

3 I give attention to the tasks performed by a program, with a general description of how MATLABcarries out the tasks The reader is not required to trace out all the programming details However,

it is often the case that available MATLAB resources suggest alternative solution strategies Hence,for those so inclined, attention to the details may be fruitful I have included, as a separate collection,the m-les written for this work These may be used as patterns for extensions as well as programs inMATLAB for computations Appendix A (Section 17.1) provides a directory of these m-les

4 Some of the details in the MATLAB script are presentation details These are renements which arenot essential to the solution of the problem But they make the programs more readily usable Andthey provide illustrations of MATLAB techniques for those who may wish to write their own programs

I hope many will be inclined to go beyond this work, modifying current programs or writing new ones

An Invitation to Experiment and Explore

Because the programs provide considerable freedom from the burden of computation and the tyranny oftables (with their limited ranges and parameter values), standard problems may be approached with a newspirit of experiment and discovery When a program is selected (or written), it embodies one method ofsolution There may be others which are readily implemented The reader is invited, even urged, to explore!The user may experiment to whatever degree he or she nds useful and interesting The possibilities areendless

Acknowledgments

After many years of teaching probability, I have long since lost track of all those authors and books whichhave contributed to the treatment of probability in this work I am aware of those contributions and am

most eager to acknowledge my indebtedness, although necessarily without specic attribution.

The power and utility of MATLAB must be attributed to to the long-time commitment of Cleve Moler,who made the package available in the public domain for several years The appearance of the professionalversions, with extended power and improved documentation, led to further appreciation and utilization ofits potential in applied probability

The Mathworks continues to develop MATLAB and many powerful "tool boxes," and to provide ship in many phases of modern computation They have generously made available MATLAB 7 to aid inchecking for compatibility the programs written with earlier versions I have not utilized the full potential

leader-of this version for developing prleader-ofessional quality user interfaces, since I believe the simpler implementationsused herein bring the student closer to the formulation and solution of the problems studied

CONNEXIONS

The development and organization of the CONNEXIONS modules has been achieved principally by twopeople: C.S.(Sid) Burrus a former student and later a faculty colleague, then Dean of Engineering, and mostimportantly a long time friend; and Daniel Williamson, a music major whose keyboard skills have enabledhim to set up the text (especially the mathematical expressions) with great accuracy, and whose dedication

to the task has led to improvements in presentation I thank them and others of the CONNEXIONS teamwho have contributed to the publication of this work

Paul E Pfeier

Rice University

Probability has roots that extend far back into antiquity The notion of chance played a central role inthe ubiquitous practice of gambling But chance acts were often related to magic or religion For example,there are numerous instances in the Hebrew Bible in which decisions were made by lot or some otherchance mechanism, with the understanding that the outcome was determined by the will of God In theNew Testament, the book of Acts describes the selection of a successor to Judas Iscariot as one of theTwelve. Two names, Joseph Barsabbas and Matthias, were put forward The group prayed, then drew lots,which fell on Matthias.

Early developments of probability as a mathematical discipline, freeing it from its religious and magicalovertones, came as a response to questions about games of chance played repeatedly The mathematicalformulation owes much to the work of Pierre de Fermat and Blaise Pascal in the seventeenth century Thegame is described in terms of a well dened trial (a play); the result of any trial is one of a specic set ofdistinguishable outcomes Although the result of any play is not predictable, certain statistical regularities

of results are observed The possible results are described in ways that make each result seem equally likely

If there are N such possible equally likely results, each is assigned a probability 1/N

The developers of mathematical probability also took cues from early work on the analysis of statisticaldata The pioneering work of John Graunt in the seventeenth century was directed to the study of vitalstatistics, such as records of births, deaths, and various diseases Graunt determined the fractions of people

in London who died from various diseases during a period in the early seventeenth century Some thirtyyears later, in 1693, Edmond Halley (for whom the comet is named) published the rst life insurance tables

To apply these results, one considers the selection of a member of the population on a chance basis One

1 This content is available online at <http://cnx.org/content/m23243/1.8/>.

Available for free at Connexions <http://cnx.org/content/col10708/1.6>

then assigns the probability that such a person will have a given disease The trial here is the selection of

a person, but the interest is in certain characteristics We may speak of the event that the person selectedwill die of a certain disease say consumption. Although it is a person who is selected, it is death fromconsumption which is of interest Out of this statistical formulation came an interest not only in probabilities

as fractions or relative frequencies but also in averages or expectatons These averages play an essential role

in modern probability

We do not attempt to trace this history, which was long and halting, though marked by ashes ofbrilliance Certain concepts and patterns which emerged from experience and intuition called for clarica-tion We move rather directly to the mathematical formulation (the mathematical model) which has mostsuccessfully captured these essential ideas This is the model, rooted in the mathematical system known asmeasure theory, is called the Kolmogorov model, after the brilliant Russian mathematician A.N Kolmogorov(1903-1987) Kolmogorov succeeded in bringing together various developments begun at the turn of the cen-tury, principally in the work of E Borel and H Lebesgue on measure theory Kolmogorov published hisepochal work in German in 1933 It was translated into English and published in 1956 by Chelsea PublishingCompany

1.1.2 Outcomes and events

Probability applies to situations in which there is a well dened trial whose possible outcomes are foundamong those in a given basic set The following are typical

• A pair of dice is rolled; the outcome is viewed in terms of the numbers of spots appearing on the topfaces of the two dice If the outcome is viewed as an ordered pair, there are thirty six equally likelyoutcomes If the outcome is characterized by the total number of spots on the two die, then there areeleven possible outcomes (not equally likely)

• A poll of a voting population is taken Outcomes are characterized by responses to a question Forexample, the responses may be categorized as positive (or favorable), negative (or unfavorable), oruncertain (or no opinion)

• A measurement is made The outcome is described by a number representing the magnitude of thequantity in appropriate units In some cases, the possible values fall among a nite set of integers Inother cases, the possible values may be any real number (usually in some specied interval)

• Much more sophisticated notions of outcomes are encountered in modern theory For example, incommunication or control theory, a communication system experiences only one signal stream in itslife But a communication system is not designed for a single signal stream It is designed for one of

an innite set of possible signals The likelihood of encountering a certain kind of signal is important

in the design Such signals constitute a subset of the larger set of all possible signals

These considerations show that our probability model must deal with

• A trial which results in (selects) an outcome from a set of conceptually possible outcomes The trial

is not successfully completed until one of the outcomes is realized

• Associated with each outcome is a certain characteristic (or combination of characteristics) pertinent

to the problem at hand In polling for political opinions, it is a person who is selected That personhas many features and characteristics (race, age, gender, occupation, religious preference, preferencesfor food, etc.) But the primary feature, which characterizes the outcome, is the political opinion onthe question asked Of course, some of the other features may be of interest for analysis of the poll.Inherent in informal thought, as well as in precise analysis, is the notion of an event to which a probabilitymay be assigned as a measure of the likelihood the event will occur on any trial A successful mathematicalmodel must formulate these notions with precision An event is identied in terms of the characteristic ofthe outcome observed The event a favorable response to a polling question occurs if the outcome observedhas that characteristic; i.e., i (if and only if) the respondent replies in the armative A hand of ve cards

is drawn The event one or more aces occurs i the hand actually drawn has at least one ace If that same

hand has two cards of the suit of clubs, then the event two clubs has occurred These considerations lead

to the following denition

Denition The event determined by some characteristic of the possible outcomes is the set of thoseoutcomes having this characteristic The event occurs i the outcome of the trial is a member of that set(i.e., has the characteristic determining the event)

• The event of throwing a seven with a pair of dice (which we call the event SEVEN) consists of theset of those possible outcomes with a total of seven spots turned up The event SEVEN occurs i theoutcome is one of those combinations with a total of seven spots (i.e., belongs to the event SEVEN).This could be represented as follows Suppose the two dice are distinguished (say by color) and apicture is taken of each of the thirty six possible combinations On the back of each picture, write thenumber of spots Now the event SEVEN consists of the set of all those pictures with seven on theback Throwing the dice is equivalent to selecting randomly one of the thirty six pictures The eventSEVEN occurs i the picture selected is one of the set of those pictures with seven on the back

• Observing for a very long (theoretically innite) time the signal passing through a communicationchannel is equivalent to selecting one of the conceptually possible signals Now such signals have manycharacteristics: the maximum peak value, the frequency spectrum, the degree of dierentibility, theaverage value over a given time period, etc If the signal has a peak absolute value less than ten volts,

a frequency spectrum essentially limited from 60 herz to 10,000 herz, with peak rate of change 10,000volts per second, then it is one of the set of signals with those characteristics The event "the signal hasthese characteristics" has occured This set (event) consists of an uncountable innity of such signals.One of the advantages of this formulation of an event as a subset of the basic set of possible outcomes is that

we can use elementary set theory as an aid to formulation And tools, such as Venn diagrams and indicatorfunctions (Section 1.3) for studying event combinations, provide powerful aids to establishing and visualizingrelationships between events We formalize these ideas as follows:

• Let Ω be the set of all possible outcomes of the basic trial or experiment We call this the basic space

or the sure event, since if the trial is carried out successfully the outcome will be in Ω; hence, the event

Ω is sure to occur on any trial We must specify unambiguously what outcomes are possible. In

ipping a coin, the only accepted outcomes are heads and tails. Should the coin stand on its edge,say by leaning against a wall, we would ordinarily consider that to be the result of an improper trial

• As we note above, each outcome may have several characteristics which are the basis for describingevents Suppose we are drawing a single card from an ordinary deck of playing cards Each card ischaracterized by a face value (two through ten, jack, queen, king, ace) and a suit (clubs, hearts,diamonds, spades) An ace is drawn (the event ACE occurs) i the outcome (card) belongs to theset (event) of four cards with ace as face value A heart is drawn i the card belongs to the set ofthirteen cards with heart as suit Now it may be desirable to specify events which involve variouslogical combinations of the characteristics Thus, we may be interested in the event the face value

is jack or king and the suit is heart or spade The set for jack or king is represented by the union

J ∪ K and the set for heart or spade is the union H ∪ S The occurrence of both conditions means theoutcome is in the intersection (common part) designated by ∩ Thus the event referred to is

The notation of set theory thus makes possible a precise formulation of the event E

• Sometimes we are interested in the situation in which the outcome does not have one of the teristics Thus the set of cards which does not have suit heart is the set of all those outcomes not inevent H In set theory, this is the complementary set (event) Hc

charac-• Events are mutually exclusive i not more than one can occur on any trial This is the condition thatthe sets representing the events are disjoint (i.e., have no members in common)

• The notion of the impossible event is useful The impossible event is, in set terminology, the emptyset∅ Event ∅ cannot occur, since it has no members (contains no outcomes) One use of ∅ is to

provide a simple way of indicating that two sets are mutually exclusive To say AB = ∅ (here weuse the alternate AB for A ∩ B) is to assert that events A and B have no outcome in common, hencecannot both occur on any given trial.

• Set inclusion provides a convenient way to designate the fact that event A implies event B, in the sensethat the occurrence of A requires the occurrence of B The set relation A ⊂ B signies that everyelement (outcome) in A is also in B If a trial results in an outcome in A (event A occurs), then thatoutcome is also in B (so that event B has occurred)

The language and notaton of sets provide a precise language and notation for events and their combinations

We collect below some useful facts about logical (often called Boolean) combinations of events (as sets) Thenotion of Boolean combinations may be applied to arbitrary classes of sets For this reason, it is sometimesuseful to use an index set to designate membership We say the index J is countable if it is nite or countablyinnite; otherwise it is uncountable In the following it may be arbitrary

{Ai: i ∈ J } is the class of sets Ai, one for each index i in the index set J (1.2)For example, if J = {1, 2, 3} then {Ai: i ∈ J }is the class {A1, A2, A3}, and

If event E is the union of a class of events, then event E occurs i at least one event in the class occurs If

F is the intersection of a class of events, then event F occurs i all events in the class occur on the trial.The role of disjoint unions is so important in probability that it is useful to have a symbol indicatingthe union of a disjoint class We use the big V to indicate that the sets combined in the union are disjoint.Thus, for example, we write

A =

n_

i=1

Ai to signify A =

n[

i=1

Ai with the proviso that the Ai form a disjoint class (1.5)

Example 1.1: Events derived from a class

Consider the class {E1, E2, E3}of events Let Ak be the event that exactly k occur on a trial and

Bk be the event that k or more occur on a trial Then

The unions are disjoint since each pair of terms has Ei in one and Eic in the other, for at least

one i Now the Bk can be expressed in terms of the Ak For example

B2= A2

_

The union in this expression for B2 is disjoint since we cannot have exactly two of the Ei occur

and exactly three of them occur on the same trial We may express B2 directly in terms of the Ei

as follows:

Here the union is not disjoint, in general However, if one pair, say {E1, E3} is disjoint, then

E1E3= ∅ and the pair {E1E2, E2E3}is disjoint (draw a Venn diagram) Suppose C is the eventthe rst two occur or the last two occur but no other combination Then

Two important patterns in set theory known as DeMorgan's rules are useful in the handling of events For

an arbitrary class {Ai: i ∈ J }of events,

Example 1.2: Continuation of Example 1.1 (Events derived from a class)

Express the event of no more than one occurrence of the events in {E1, E2, E3}as B2c

In the module "Likelihood" (Section 1.1) we introduce the notion of a basic space Ω of all possible outcomes

of a trial or experiment, events as subsets of the basic space determined by appropriate characteristics ofthe outcomes, and logical or Boolean combinations of the events (unions, intersections, and complements)corresponding to logical combinations of the dening characteristics

Occurrence or nonoccurrence of an event is determined by characteristics or attributes of the outcomeobserved on a trial Performing the trial is visualized as selecting an outcome from the basic set Anevent occurs whenever the selected outcome is a member of the subset representing the event As described

so far, the selection process could be quite deliberate, with a prescribed outcome, or it could involve theuncertainties associated with chance. Probability enters the picture only in the latter situation Before thetrial is performed, there is uncertainty about which of these latent possibilities will be realized Probabilitytraditionally is a number assigned to an event indicating the likelihood of the occurrence of that event onany trial

We begin by looking at the classical model which rst successfully formulated probability ideas in ematical form We use modern terminology and notation to describe it

math-Classical probability

1 The basic space Ω consists of a nite number N of possible outcomes

- There are thirty six possible outcomes of throwing two dice

2 This content is available online at <http://cnx.org/content/m23244/1.8/>.

- There are C (52, 5) = 52!

5!47! = 2598960dierent hands of ve cards (order not important)

- There are 25= 32results (sequences of heads or tails) of ipping ve coins

2 Each possible outcome is assigned a probability 1/N

3 If event (subset) A has NA elements, then the probability assigned event A is

With this denition of probability, each event A is assigned a unique probability, which may be determined

by counting NA, the number of elements in A (in the classical language, the number of outcomes "favorable"

to the event) and N the total number of possible outcomes in the sure event Ω

Example 1.3: Probabilities for hands of cards

Consider the experiment of drawing a hand of ve cards from an ordinary deck of 52 playing cards

The number of outcomes, as noted above, is N = C (52, 5) = 2598960 What is the probability

of drawing a hand with exactly two aces? What is the probability of drawing a hand with two or

more aces? What is the probability of not more than one ace?

SOLUTION

Let A be the event of exactly two aces, B be the event of exactly three aces, and C be the event

of exactly four aces In the rst problem, we must count the number NAof ways of drawing a hand

with two aces We select two aces from the four, and select the other three cards from the 48 non

There are two or more aces i there are exactly two or exactly three or exactly four Thus the

event D of two or more is D = A W B W C Since A, B, C are mutually exclusive,

ND = NA+ NB+ NC = C (4, 2) C (48, 3) + C (4, 3) C (48, 2) + C (4, 4) C (48, 1) =

103776 + 4512 + 48 = 108336

(1.15)

so that P (D) ≈ 0.0417 There is one ace or none i there are not two or more aces We thus

want P (Dc) Now the number in Dc is the number not in D which is N − ND, so that

This example illustrates several important properties of the classical probability

1 P (A) = NA/N is a nonnegative quantity

2 P (Ω) = N/N = 1

3 If A, B, C are mutually exclusive, then the number in the disjoint union is the sum of the numbers inthe individual events, so that

Several other elementary properties of the classical probability may be identied It turns out that they can

be derived from these three Although the classical model is highly useful, and an extensive theory has beendeveloped, it is not really satisfactory for many applications (the communications problem, for example)

We seek a more general model which includes classical probability as a special case and is thus an extension

of it We adopt the Kolmogorov model (introduced by the Russian mathematician A N Kolmogorov) which

captures the essential ideas in a remarkably successful way Of course, no model is ever completely successful.Reality always seems to escape our logical nets.

The Kolmogorov model is grounded in abstract measure theory A full explication requires a level ofmathematical sophistication inappropriate for a treatment such as this But most of the concepts and many

of the results are elementary and easily grasped And many technical mathematical considerations are notimportant for applications at the level of this introductory treatment and may be disregarded We borrowfrom measure theory a few key facts which are either very plausible or which can be understood at a practicallevel This enables us to utilize a very powerful mathematical system for representing practical problems in

a manner that leads to both insight and useful strategies of solution

Our approach is to begin with the notion of events as sets introduced above, then to introduce probability

as a number assigned to events subject to certain conditions which become denitive properties Gradually

we introduce and utilize additional concepts to build progressively a powerful and useful discipline Thefundamental properties needed are just those illustrated in Example 1.3 (Probabilities for hands of cards)for the classical case

Denition

A probability system consists of a basic set Ω of elementary outcomes of a trial or experiment, a class ofevents as subsets of the basic space, and a probability measure P (·) which assigns values to the events inaccordance with the following rules:

(P1): For any event A, the probability P (A) ≥ 0

(P2): The probability of the sure event P (Ω) = 1

(P3): Countable additivity If {Ai : 1 ∈ J } is a mutually exclusive, countable class of events, then theprobability of the disjoint union is the sum of the individual probabilities

The necessity of the mutual exclusiveness (disjointedness) is illustrated in Example 1.3 (Probabilities forhands of cards) If the sets were not disjoint, probability would be counted more than once in the sum Aprobability, as dened, is abstractsimply a number assigned to each set representing an event But we cangive it an interpretation which helps to visualize the various patterns and relationships encountered We maythink of probability as mass assigned to an event The total unit mass is assigned to the basic set Ω Theadditivity property for disjoint sets makes the mass interpretation consistent We can use this interpretation

as a precise representation Repeatedly we refer to the probability mass assigned a given set The mass

is proportional to the weight, so sometimes we speak informally of the weight rather than the mass Now

a mass assignment with three properties does not seem a very promising beginning But we soon expandthis rudimentary list of properties We use the mass interpretation to help visualize the properties, but areprimarily concerned to interpret them in terms of likelihoods

(P4): P (Ac) = 1 − P (A) This follows from additivity and the fact that

(P5): P (∅) = 0 The empty set represents an impossible event It has no members, hence cannot occur

It seems reasonable that it should be assigned zero probability (mass) Since ∅ = Ωc, this followslogically from (P4) ("(P4)", p 11) and (P2) ("(P2)", p 11)

Figure 1.1: Partitions of the union A ∪ B.

(P6): If A ⊂ B, then P (A) ≤ P (B) From the mass point of view, every point in A is also in B, so that Bmust have at least as much mass as A Now the relationship A ⊂ B means that if A occurs, B mustalso Hence B is at least as likely to occur as A From a purely formal point of view, we have

B = A_AcB so that P (B) = P (A) + P (AcB) ≥ P (A) since P (AcB) ≥ 0 (1.19)

(P7): P (A ∪ B) = P (A) + P (A

cB) = P (B) + P (ABc) = P (ABc) + P (AB) + P (AcB)

= P (A) + P (B) − P (AB)The rst three expressions follow from additivity and partitioning of A ∪ B as follows (see Figure 1.1)

If we add the rst two expressions and subtract the third, we get the last expression In terms ofprobability mass, the rst expression says the probability in A ∪ B is the probability mass in A plusthe additional probability mass in the part of B which is not in A A similar interpretation holds forthe second expression The third is the probability in the common part plus the extra in A and theextra in B If we add the mass in A and B we have counted the mass in the common part twice Thelast expression shows that we correct this by taking away the extra common mass

(P8): If {Bi: i ∈ J }is a countable, disjoint class and A is contained in the union, then

(P9): Subadditivity If A = S∞

i=1Ai, then P (A) ≤ P∞

i=1P (Ai) This follows from countable additivity,property (P6) ("(P6)", p 12), and the fact (Partitions)

A =

∞[

i=1

Ai=

∞_

i=1

Bi, where Bi= AiAc1Ac2· · · Aci−1⊂ Ai (1.22)This includes as a special case the union of a nite number of events

Some of these properties, such as (P4) ("(P4)", p 11), (P5) ("(P5)", p 11), and (P6) ("(P6)", p 12), are

so elementary that it seems they should be included in the dening statement This would not be incorrect,but would be inecient If we have an assignment of numbers to the events, we need only establish (P1)("(P1)", p 11), (P2) ("(P2)", p 11), and (P3) ("(P3)", p 11) to be able to assert that the assignmentconstitutes a probability measure And the other properties follow as logical consequences

Flexibility at a price

In moving beyond the classical model, we have gained great exibility and adaptability of the model

It may be used for systems in which the number of outcomes is innite (countably or uncountably) Itdoes not require a uniform distribution of the probability mass among the outcomes For example, thedice problem may be handled directly by assigning the appropriate probabilities to the various numbers oftotal spots, 2 through 12 As we see in the treatment of conditional probability (Section 3.1), we makenew probability assignments (i.e., introduce new probability measures) when partial information about theoutcome is obtained

But this freedom is obtained at a price In the classical case, the probability value to be assigned an event

is clearly dened (although it may be very dicult to perform the required counting) In the general case,

we must resort to experience, structure of the system studied, experiment, or statistical studies to assignprobabilities

The existence of uncertainty due to chance or randomness does not necessarily imply that the act ofperforming the trial is haphazard The trial may be quite carefully planned; the contingency may be the result

of factors beyond the control or knowledge of the experimenter The mechanism of chance (i.e., the source

of the uncertainty) may depend upon the nature of the actual process or system observed For example, intaking an hourly temperature prole on a given day at a weather station, the principal variations are not due

to experimental error but rather to unknown factors which converge to provide the specic weather patternexperienced In the case of an uncorrected digital transmission error, the cause of uncertainty lies in theintricacies of the correction mechanisms and the perturbations produced by a very complex environment Apatient at a clinic may be self selected Before his or her appearance and the result of a test, the physicianmay not know which patient with which condition will appear In each case, from the point of view of theexperimenter, the cause is simply attributed to chance. Whether one sees this as an act of the gods orsimply the result of a conguration of physical or behavioral causes too complex to analyze, the situation isone of uncertainty, before the trial, about which outcome will present itself

If there were complete uncertainty, the situation would be chaotic But this is not usually the case.While there is an extremely large number of possible hourly temperature proles, a substantial subset ofthese has very little likelihood of occurring For example, proles in which successive hourly temperaturesalternate between very high then very low values throughout the day constitute an unlikely subset (event).One normally expects trends in temperatures over the 24 hour period Although a trac engineer does notknow exactly how many vehicles will be observed in a given time period, experience provides some idea whatrange of values to expect While there is uncertainty about which patient, with which symptoms, will appear

at a clinic, a physician certainly knows approximately what fraction of the clinic's patients have the disease

in question In a game of chance, analyzed into equally likely outcomes, the assumption of equal likelihood

is based on knowledge of symmetries and structural regularities in the mechanism by which the game iscarried out And the number of outcomes associated with a given event is known, or may be determined

In each case, there is some basis in statistical data on past experience or knowledge of structure, regularity,and symmetry in the system under observation which makes it possible to assign likelihoods to the occurrence

of various events It is this ability to assign likelihoods to the various events which characterizes applied

probability However determined, probability is a number assigned to events to indicate their likelihood ofoccurrence The assignments must be consistent with the dening properties (P1) ("(P1)", p 11), (P2)("(P2)", p 11), (P3) ("(P3)", p 11) along with derived properties (P4) through (P9) (p 11) (plus otherswhich may also be derived from these) Since the probabilities are not built in, as in the classical case, aprime role of probability theory is to derive other probabilities from a set of given probabilites.

1.3 Interpretations3

1.3.1 What is Probability?

The formal probability system is a model whose usefulness can only be established by examining its structureand determining whether patterns of uncertainty and likelihood in any practical situation can be representedadequately With the exception of the sure event and the impossible event, the model does not tell us how toassign probability to any given event The formal system is consistent with many probability assignments,just as the notion of mass is consistent with many dierent mass assignments to sets in the basic space.The dening properties (P1) ("(P1)", p 11), (P2) ("(P2)", p 11), (P3) ("(P3)", p 11) and derivedproperties provide consistency rules for making probability assignments One cannot assign negative proba-bilities or probabilities greater than one The sure event is assigned probability one If two or more eventsare mutually exclusive, the total probability assigned to the union must equal the sum of the probabilities

of the separate events Any assignment of probability consistent with these conditions is allowed

One may not know the probability assignment to every event Just as the dening conditions putconstraints on allowable probability assignments, they also provide important structure A typical appliedproblem provides the probabilities of members of a class of events (perhaps only a few) from which todetermine the probabilities of other events of interest We consider an important class of such problems inthe next chapter

There is a variety of points of view as to how probability should be interpreted These impact the manner

in which probabilities are assigned (or assumed) One important dichotomy among practitioners

• One group believes probability is objective in the sense that it is something inherent in the nature ofthings It is to be discovered, if possible, by analysis and experiment Whether we can determine it ornot, it is there.

• Another group insists that probability is a condition of the mind of the person making the probabilityassessment From this point of view, the laws of probability simply impose rational consistency uponthe way one assigns probabilities to events Various attempts have been made to nd objective ways

to measure the strength of one's belief or degree of certainty that an event will occur The probability

P (A)expresses the degree of certainty one feels that event A will occur One approach to characterizing

an individual's degree of certainty is to equate his assessment of P (A) with the amount a he is willing

to pay to play a game which returns one unit of money if A occurs, for a gain of (1 − a), and returnszero if A does not occur, for a gain of −a Behind this formulation is the notion of a fair game, inwhich the expected or average gain is zero

The early work on probability began with a study of relative frequencies of occurrence of an event underrepeated but independent trials This idea is so imbedded in much intuitive thought about probability thatsome probabilists have insisted that it must be built into the denition of probability This approach has notbeen entirely successful mathematically and has not attracted much of a following among either theoretical orapplied probabilists In the model we adopt, there is a fundamental limit theorem, known as Borel's theorem,which may be interpreted if a trial is performed a large number of times in an independent manner, thefraction of times that event A occurs approaches as a limit the value P (A) Establishing this result (which

we do not do in this treatment) provides a formal validation of the intuitive notion that lay behind theearly attempts to formulate probabilities Inveterate gamblers had noted long-run statistical regularities,

3 This content is available online at <http://cnx.org/content/m23246/1.8/>.

and sought explanations from their mathematically gifted friends From this point of view, probability ismeaningful only in repeatable situations Those who hold this view usually assume an objective view ofprobability It is a number determined by the nature of reality, to be discovered by repeated experiment.There are many applications of probability in which the relative frequency point of view is not feasible.Examples include predictions of the weather, the outcome of a game or a horse race, the performance of anindividual on a particular job, the success of a newly designed computer These are unique, nonrepeatabletrials As the popular expression has it, You only go around once. Sometimes, probabilities in thesesituations may be quite subjective As a matter of fact, those who take a subjective view tend to think

in terms of such problems, whereas those who take an objective view usually emphasize the frequencyinterpretation

Example 1.4: Subjective probability and a football game

The probability that one's favorite football team will win the next Superbowl Game may well

be only a subjective probability of the bettor This is certainly not a probability that can bedetermined by a large number of repeated trials The game is only played once However, thesubjective assessment of probabilities may be based on intimate knowledge of relative strengthsand weaknesses of the teams involved, as well as factors such as weather, injuries, and experience.There may be a considerable objective basis for the subjective assignment of probability In fact,there is often a hidden frequentist element in the subjective evaluation There is an assessment(perhaps unrealized) that in similar situations the frequencies tend to coincide with the valuesubjectively assigned

Example 1.5: The probability of rain

Newscasts often report that the probability of rain of is 20 percent or 60 percent or some other

gure There are several diculties here

• To use the formal mathematical model, there must be precision in determining an event

An event either occurs or it does not How do we determine whether it has rained or not?Must there be a measurable amount? Where must this rain fall to be counted? During whattime period? Even if there is agreement on the area, the amount, and the time period, thereremains ambiguity: one cannot say with logical certainty the event did occur or it did notoccur Nevertheless, in this and other similar situations, use of the concept of an event may behelpful even if the description is not denitive There is usually enough practical agreementfor the concept to be useful

• What does a 30 percent probability of rain mean? Does it mean that if the prediction is correct,

30 percent of the area indicated will get rain (in an agreed amount) during the specied timeperiod? Or does it mean that 30 percent of the occasions on which such a prediction is madethere will be signicant rainfall in the area during the specied time period? Again, the latteralternative may well hide a frequency interpretation Does the statement mean that it rains

30 percent of the times when conditions are similar to current conditions?

Regardless of the interpretation, there is some ambiguity about the event and whether it hasoccurred And there is some diculty with knowing how to interpret the probability gure Whilethe precise meaning of a 30 percent probability of rain may be dicult to determine, it is generallyuseful to know whether the conditions lead to a 20 percent or a 30 percent or a 40 percent probabilityassignment And there is no doubt that as weather forecasting technology and methodology continue

to improve the weather probability assessments will become increasingly useful

Another common type of probability situation involves determining the distribution of some characteristicover a populationusually by a survey These data are used to answer the question: What is the probability(likelihood) that a member of the population, chosen at random (i.e., on an equally likely basis) will have

a certain characteristic?

Example 1.6: Empirical probability based on survey data

A survey asks two questions of 300 students: Do you live on campus? Are you satised withthe recreational facilities in the student center? Answers to the latter question were categorized

reasonably satised, unsatised, or no denite opinion. Let C be the event on campus; O

be the event o campus; S be the event reasonably satised; U be the event unsatised; and

N be the event no denite opinion. Data are shown in the following table

If an individual is selected on an equally likely basis from this group of 300, the probability of any

of the events is taken to be the relative frequency of respondents in each category corresponding

to an event There are 200 on campus members in the population, so P (C) = 200/300 and

P (O) = 100/300 The probability that a student selected is on campus and satised is taken to be

P (CS) = 127/300 The probability a student is either on campus and satised or o campus andnot satised is

If there is reason to believe that the population sampled is representative of the entire studentbody, then the same probabilities would be applied to any student selected at random from theentire student body

It is fortunate that we do not have to declare a single position to be the correct viewpoint and interpretation.The formal model is consistent with any of the views set forth We are free in any situation to make theinterpretation most meaningful and natural to the problem at hand It is not necessary to t all problemsinto one conceptual mold; nor is it necessary to change mathematical model each time a dierent point ofview seems appropriate

1.3.2 Probability and odds

Often we nd it convenient to work with a ratio of probabilities If A and B are events with positiveprobability the odds favoring A over B is the probability ratio P (A) /P (B) If not otherwise specied, B istaken to be Ac and we speak of the odds favoring A

1.3.3 Partitions and Boolean combinations of events

The countable additivity property (P3) ("(P3)", p 11) places a premium on appropriate partitioning ofevents

Denition A partition is a mutually exclusive class

• A partition (no qualier) is taken to be a partition of the sure event Ω

• If class {Bi : ß ∈ J } is mutually exclusive and A ⊂ B = W

P

∞[

i=1

P (Bi) ≤

∞X

i=1

The representation of a union as a disjoint union points to an important strategy in the solution of probabilityproblems If an event can be expressed as a countable disjoint union of events, each of whose probabilities isknown, then the probability of the combination is the sum of the individual probailities In in the module onPartitions and Minterms (Section 2.1.2: Partitions and minterms), we show that any Boolean combination

of a nite class of events can be expressed as a disjoint union in a manner that often facilitates systematicdetermination of the probabilities

1.3.4 The indicator function

One of the most useful tools for dealing with set combinations (and hence with event combinations) is theindicator function IE for a set E ⊂ Ω It is dened very simply as follows:

IE(ω) = { 1 for ω ∈ E

Remark Indicator fuctions may be dened on any domain We have occasion in various cases to denethem on the real line and on higher dimensional Euclidean spaces For example, if M is the interval [a, b]

on the real line then IM(t) = 1 for each t in the interval (and is zero otherwise) Thus we have a stepfunction with unit value over the interval M In the abstract basic space Ω we cannot draw a graph so easily.However, with the representation of sets on a Venn diagram, we can give a schematic representation, as inFigure 1.2

Figure 1.2: Representation of the indicator function IEfor event E

Much of the usefulness of the indicator function comes from the following properties

(IF1): IA≤ IB i A ⊂ B If IA≤ IB, then ω ∈ A implies IA(ω) = IB(ω) = 1, so ω ∈ B If A ⊂ B, then

IA(ω) = 1implies ω ∈ A implies ω ∈ B implies IB(ω) = 1

(IF2): IA= IB i A = B

A = B i both A ⊂ B and B ⊂ A i IA≤ IB and IB ≤ IA i IA= IB (1.31)

(IF3): IA c = 1 − IAThis follows from the fact IA c(ω) = 1i IA(ω) = 0

(IF4): IAB = IAIB = min{IA, IB} (extends to any class) An element ω belongs to the intersection i itbelongs to all i the indicator function for each event is one i the product of the indicator functions

is one

(IF5): IA∪B = IA+ IB− IAIB = max{IA, IB} (the maximum rule extends to any class) The maximumrule follows from the fact that ω is in the union i it is in any one or more of the events in the union iany one or more of the individual indicator function has value one i the maximum is one The sumrule for two events is established by DeMorgan's rule and properties (IF2), (IF3), and (IF4)

IA∪B = 1 − IAc B c= 1 − [1 − IA] [1 − IB] = 1 − 1 + IB+ IA− IAIB (1.32)

(IF6): If the pair {A, B} is disjoint, IA W B = IA+ IB (extends to any disjoint class)

The following example illustrates the use of indicator functions in establishing relationships between setcombinations Other uses and techniques are established in the module on Partitions and Minterms (Sec-tion 2.1.2: Partitions and minterms)

Example 1.7: Indicator functions and set combinations

Suppose {Ai: 1 ≤ i ≤ n}is a partition

If B =

n_

i=1

AiCi, then Bc

=

n_

i=1

Note that since the Ai form a partition, we have Pn

i=1IAi = 1, so that the indicator function forthe complementary event is

IB c= 1 −

nX

i=1

IAiICi =

nX

i=1

IAi−

nX

i=1

IAiICi =

nX

i=1

IAi[1 − ICi] =

nX

1.3.5 A technical comment on the class of events

The class of events plays a central role in the intuitive background, the application, and the formal ematical structure Events have been modeled as subsets of the basic space of all possible outcomes of thetrial or experiment In the case of a nite number of outcomes, any subset can be taken as an event In thegeneral theory, involving innite possibilities, there are some technical mathematical reasons for limiting theclass of subsets to be considered as events The practical needs are these:

math-1 If A is an event, its complementary set must also be an event

2 If {Ai: i ∈ J }is a nite or countable class of events, the union and the intersection of members of theclass need to be events

A simple argument based on DeMorgan's rules shows that if the class contains complements of all its setsand countable unions, then it contains countable intersections Likewise, if it contains complements of all itssets and countable intersections, then it contains countable unions A class of sets closed under complementsand countable unions is known as a sigma algebra of sets In a formal, measure-theoretic treatment, a basicassumption is that the class of events is a sigma algebra and the probability measure assigns probabilities tomembers of that class Such a class is so general that it takes very sophisticated arguments to establish thefact that such a class does not contain all subsets But precisely because the class is so general and inclusive

in ordinary applications we need not be concerned about which sets are permissible as events

A primary task in formulating a probability problem is identifying the appropriate events and the tionships between them The theoretical treatment shows that we may work with great freedom in formingevents, with the assurrance that in most applications a set so produced is a mathematically valid event.The so called measurability question only comes into play in dealing with random processes with continuousparameters Even there, under reasonable assumptions, the sets produced will be events

Let Ω consist of the set of positive integers Consider the subsets

4 This content is available online at <http://cnx.org/content/m24071/1.5/>.

d The even integers greater than 12.

e The positive integers which are multiples of six

f The integers which are even and no greater than 6 or which are odd and greater than 12

Let Ω be the set of integers 0 through 10 Let A = {5, 6, 7, 8}, B = the odd integers in Ω, and

C =the integers in Ω which are even or less than three Describe the following sets by listing theirelements

A group of ve persons consists of two men and three women They are selected one-by-one in arandom manner Let Ei be the event a man is selected on the ith selection Write an expressionfor the event that both men have been selected by the third selection

Two persons play a game consecutively until one of them is successful or there are ten unsuccessfulplays Let Ei be the event of a success on the ith play of the game Let A, B, C be the respectiveevents that player one, player two, or neither wins Write an expression for each of these events interms of the events Ei, 1 ≤ i ≤ 10

Suppose the game in Exercise 1.4.5 could, in principle, be played an unlimited number of times.Write an expression for the event D that the game will be terminated with a success in a nitenumber of times Write an expression for the event F that the game will never terminate

Find the (classical) probability that among three random digits, with each digit (0 through 9)being equally likely and each triple equally likely:

a All three are alike

b No two are alike

c The rst digit is 0

d Exactly two are alike

Exercise 1.4.8 (Solution on p 23.)

The classical probability model is based on the assumption of equally likely outcomes Some caremust be shown in analysis to be certain that this assumption is good A well known example is thefollowing Two coins are tossed One of three outcomes is observed: Let ω1 be the outcome bothare heads, ω2 the outcome that both are tails, and ω3 be the outcome that they are dierent

Is it reasonable to suppose these three outcomes are equally likely? What probabilities would youassign?

An extension of the classical model involves the use of areas A certain region L (say of land) istaken as a reference For any subregion A, dene P (A) = area (A) /area (L) Show that P (·) is aprobability measure on the subregions of L

John thinks the probability the Houston Texans will win next Sunday is 0.3 and the probabilitythe Dallas Cowboys will win is 0.7 (they are not playing each other) He thinks the probability bothwill win is somewhere betweensay, 0.5 Is that a reasonable assumption? Justify your answer

Suppose P (A) = 0.5 and P (B) = 0.3 What is the largest possible value of P (AB)? Usingthe maximum value of P (AB), determine P (ABc), P (AcB), P (AcBc)and P (A ∪ B) Are thesevalues determined uniquely?

Use fundamental properties of probability to show

a P (AB) ≤ P (A) ≤ P (A ∪ B) ≤ P (A) + P (B)

P (E) =

nX

i=1

ciPi(E) , where the Pi are probabilities measures, ci > 0, and

nX

i=1

ciP (EAi) /

nX

i=1

Show that Q (·) us a probability measure

Solutions to Exercises in Chapter 1

Each triple has probability 1/103= 1/1000

a Ten triples, all alike: P = 10/1000

b 10 × 9 × 8 triples all dierent: P = 720/1000

c 100 triples with rst one zero: P = 100/1000

d C (3, 2) = 3 ways to pick two positions alike; 10 ways to pick the common value; 9 ways to pick theother P = 270/1000

Solution to Exercise 1.4.13 (p 21)

Draw a Venn diagram, or use algebraic expressions P (ABc) = P (A) − P (AB) = 0.2

P (AcB) = P (B) − P (AB) = 0 P (AcBc) = P (Ac) − P (AcB) = 0.5 P (A ∪ B) = 0.5 (1.41)Solution to Exercise 1.4.14 (p 21)

i=1

Ei implies P (E) = c1

∞X

i=1

P1(Ei) + c2

∞X

i=1

P2(Ei) =

∞X

k=1

Ek, then P (EAi) =

∞X

k=1

Interchanging the order of summation shows that Q is countably additive

Chapter 2

Minterm Analysis

2.1 Minterms1

2.1.1 Introduction

A fundamental problem in elementary probability is to nd the probability of a logical (Boolean) combination

of a nite class of events, when the probabilities of certain other combinations are known If we partition anevent F into component events whose probabilities can be determined, then the additivity property impliesthe probability of F is the sum of these component probabilities Frequently, the event F is a Booleancombination of members of a nite class say, {A, B, C} or {A, B, C, D} For each such nite class, there

is a fundamental partition determined by the class The members of this partition are called minterms AnyBoolean combination of members of the class can be expressed as the disjoint union of a unique subclass ofthe minterms If the probability of every minterm in this subclass can be determined, then by additivity theprobability of the Boolean combination is determined We examine these ideas in more detail

2.1.2 Partitions and minterms

To see how the fundamental partition arises naturally, consider rst the partition of the basic space produced

by a single event A

Now if B is a second event, then

A = AB_ABc and Ac= AcB_AcBc, so that Ω = AcBc_AcB_ABc_AB (2.2)The pair {A, B} has partitioned Ω into {AcBc, AcB, ABc, AB} Continuation is this way leads systemat-ically to a partition by three events {A, B, C}, four events {A, B, C, D}, etc

We illustrate the fundamental patterns in the case of four events {A, B, C, D} We form the minterms

as intersections of members of the class, with various patterns of complementation For a class of four events,there are 24= 16such patterns, hence 16 minterms These are, in a systematic arrangement,

AcBcCcDc AcBCcDc ABcCcDc ABCcDc

AcBcCcD AcBCcD ABcCcD ABCcD

AcBcC Dc AcBC Dc ABcC Dc ABC Dc

1 This content is available online at <http://cnx.org/content/m23247/1.8/>.

Available for free at Connexions <http://cnx.org/content/col10708/1.6>

Table 2.1

No element can be in more than one minterm, because each diers from the others by complementation

of at least one member event Each element ω is assigned to exactly one of the minterms by determining theanswers to four questions:

Is it in A? Is it in B? Is it in C? Is it in D?

Suppose, for example, the answers are: Yes, No, No, Yes Then ω is in the minterm ABcCcD In asimilar way, we can determine the membership of each ω in the basic space Thus, the minterms form apartition That is, the minterms represent mutually exclusive events, one of which is sure to occur on eachtrial The membership of any minterm depends upon the membership of each generating set A, B, C or D,and the relationships between them For some classes, one or more of the minterms are empty (impossibleevents) As we see below, this causes no problems

An examination of the development above shows that if we begin with a class of n events, there are

2n minterms To aid in systematic handling, we introduce a simple numbering system for the minterms,which we illustrate by considering again the four events A, B, C, D , in that order The answers to the fourquestions above can be represented numerically by the scheme

No ∼ 0 and Yes ∼ 1

Thus, if ω is in AcBcCcDc, the answers are tabulated as 0 0 0 0 If ω is in ABcCcD, then this is designated

1 0 0 1 With this scheme, the minterm arrangement above becomes

We may view these quadruples of zeros and ones as binary representations of integers, which may also

be represented by their decimal equivalents, as shown in the table Frequently, it is useful to refer tothe minterms by number If the members of the generating class are treated in a xed order, then eachminterm number arrived at in the manner above species a minterm uniquely Thus, for the generating class{A, B, C, D}, in that order, we may designate

AcBcCcDc= M0 (minterm 0) ABcCcD = M9 (minterm 9), etc (2.3)

We utilize this numbering scheme on special Venn diagrams called minterm maps These are illustrated inFigure 2.1, for the cases of three, four, and ve generating events Since the actual content of any mintermdepends upon the sets A, B, C, and D in the generating class, it is customary to refer to these sets asvariables In the three-variable case, set A is the right half of the diagram and set C is the lower half; but set

B is split, so that it is the union of the second and fourth columns Similar splits occur in the other cases.Remark Other useful arrangements of minterm maps are employed in the analysis of switching circuits

D E

Figure 1.1 M interm m aps for three, four, and five variables.

Figure 2.1: Minterm maps for three, four, or ve variables

2.1.3 Minterm maps and the minterm expansion

The signicance of the minterm partition of the basic space rests in large measure on the following fact.Minterm expansion

Each Boolean combination of the elements in a generating class may be expressed as the disjoint union

of an appropriate subclass of the minterms This representation is known as the minterm expansion for the

In deriving an expression for a given Boolean combination which holds for any class {A, B, C , D} of fourevents, we include all possible minterms, whether empty or not If a minterm is empty for a given class, itspresence does not modify the set content or probability assignment for the Boolean combination

The existence and uniqueness of the expansion is made plausible by simple examples utilizing mintermmaps to determine graphically the minterm content of various Boolean combinations Using the arrangementand numbering system introduced above, we let Mi represent the ith minterm (numbering from zero) andlet p (i) represent the probability of that minterm When we deal with a union of minterms in a mintermexpansion, it is convenient to utilize the shorthand illustrated in the following

M (1, 3, 7) = M1

_

M3_

Figure 2.2: E = AB ∪ Ac

(B ∪ Cc)c = M (1, 6, 7) Minterm expansion for Example 2.1 ( Mintermexpansion)

Consider the following simple example

Example 2.1: Minterm expansion

Suppose E = AB ∪ Ac(B ∪ Cc)c Examination of the minterm map in Figure 2.2 shows that

AB consists of the union of minterms M6, M7, which we designate M (6, 7) The combinationB∪Cc= M (0, 2, 3, 4, 6, 7), so that its complement (B ∪ Cc)c= M (1, 5) This leaves the commonpart Ac(B ∪ Cc)c= M1 Hence, E = M (1, 6, 7) Similarly, F = A ∪ BcC = M (1, 4, 5, 6, 7)

A key to establishing the expansion is to note that each minterm is either a subset of the combination or isdisjoint from it The expansion is thus the union of those minterms included in the combination A generalverication using indicator functions is sketched in the last section of this module

2.1.4 Use of minterm maps

A typical problem seeks the probability of certain Boolean combinations of a class of events when theprobabilities of various other combinations is given We consider several simple examples and illustrate theuse of minterm maps in formulation and solution

Example 2.2: Survey on software

Statistical data are taken for a certain student population with personal computers An individual

is selected at random Let A = the event the person selected has word processing, B = the event

he or she has a spread sheet program, and C = the event the person has a data base program Thedata imply

• The probability is 0.80 that the person has a word processing program: P (A) = 0.8

• The probability is 0.65 that the person has a spread sheet program: P (B) = 0.65

• The probability is 0.30 that the person has a data base program: P (C) = 0.3

• The probability is 0.10 that the person has all three: P (ABC) = 0.1

• The probability is 0.05 that the person has neither word processing nor spread sheet:

P (AcBc) = 0.05

• The probability is 0.65 that the person has at least two: P (AB ∪ AC ∪ BC) = 0.65

• The probability of word processor and data base, but no spread sheet is twice the probabilty

of spread sheet and data base, but no word processor: P (ABcC) = 2P (AcBC)

a What is the probability that the person has exactly two of the programs?

b What is the probability that the person has only the data base program?

Several questions arise:

• Are these data consistent?

• Are the data sucient to answer the questions?

• How may the data be utilized to anwer the questions?

Figure 2.3: Minterm maps for software survey, Example 2.2 (Survey on software)

Example 2.3: Survey on personal computers

A survey of 1000 students shows that 565 have PC compatible desktop computers, 515 have tosh desktop computers, and 151 have laptop computers 51 have all three, 124 have both PC andlaptop computers, 212 have at least two of the three, and twice as many own both PC and laptop

Macin-as those who have both Macintosh desktop and laptop A person is selected at random from thispopulation What is the probability he or she has at least one of these types of computer? What

is the probability the person selected has only a laptop?

Figure 2.4: Minterm probabilities for computer survey, Example 2.3 (Survey on personal computers)

SOLUTION

Let A = the event of owning a PC desktop, B = the event of owning a Macintosh desktop, and

C =the event of owning a laptop We utilize a minterm map for three variables to help determineminterm patterns For example, the event AC = M5W M7so that P (AC) = p (5) + p (7) = p (5, 7).The data, expressed in terms of minterm probabilities, are:

Figure 2.5: Minterm probabilities for opinion survey, Example 2.4 (Opinion survey)

Example 2.4: Opinion survey

A survey of 1000 persons is made to determine their opinions on four propositions Let A, B, C, D

be the events a person selected agrees with the respective propositions Survey results show thefollowing probabilities for various combinations:

P (A) = 0.200, P (B) = 0.500, P (C) = 0.300, P (D) = 0.700, P (A (B ∪ Cc) Dc) = 0.055 (2.7)

P (A ∪ BC ∪ Dc) = 0.520, P (AcBCcD) = 0.200, P (ABCD) = 0.015, P (ABcC) = 0.030 (2.8)

P (AcBcCcD) = 0.195, P (AcBC) = 0.120, P (AcBcDc) = 0.120, P (ACc) = 0.140 (2.9)

Determine the probabilities for each minterm and for each of the following combinations

Ac(BCc∪ BcC) that is, not A and (B or C, but not both)

A ∪ BCc  that is, A or (B and not C)

SOLUTION

At the outset, it is not clear that the data are consistent or sucient to determine the mintermprobabilities However, an examination of the data shows that there are sixteen items (including thefact that the sum of all minterm probabilities is one) Thus, there is hope, but no assurance, that asolution exists A step elimination procedure, as in the previous examples, shows that all mintermscan in fact be calculated The results are displayed on the minterm map in Figure 2.5 It would bedesirable to be able to analyze the problem systematically The formulation above suggests a moresystematic algebraic formulation which should make possible machine aided solution