The books a lso a re use ful for se lf-study or a s a re fe re nce for pe ople e nga ge d in the de ve lopme nt of compute r syste ms or a pplica tions progra ms be ca usethe y conta in
Trang 1Algorithms in Java: Parts 1-4, Third Edition
By Robert Sedgewick
Publishe r: Addison W e sle yPub Da te : July 23, 2002ISBN: 0-201-36120-5, 768 pa ge s
Se dge w ick ha s a re a l gift for e xpla ining conce pts in a w a y tha t ma ke s the m e a sy to unde rsta nd The use of re a l progra ms in pa ge -size (or le ss)chunks tha t ca n be e a sily unde rstood is a re a l plus The figure s, progra ms, a nd ta ble s a re a significa nt contribution to the le a rning e xpe rie nce ofthe re a de r; the y ma ke this book distinctive -W illia m A W a rd, Unive rsity of South Ala ba ma
This e dition of Robe rt Se dge w ick's popula r w ork provide s curre nt a nd compre he nsive cove ra ge of importa nt a lgorithms for Ja va progra mme rs.Micha e l Schidlow sky a nd Se dge w ick ha ve de ve lope d ne w Ja va imple me nta tions tha t both e xpre ss the me thods in a concise a nd dire ct ma nne r
a nd provide progra mme rs w ith the pra ctica l me a ns to te st the m on re a l a pplica tions
Ma ny ne w a lgorithms a re pre se nte d, a nd the e xpla na tions of e a ch a lgorithm a re much more de ta ile d tha n in pre vious e ditions A ne w te xt de sign
a nd de ta ile d, innova tive figure s, w ith a ccompa nying comme nta ry, gre a tly e nha nce the pre se nta tion The third e dition re ta ins the succe ssfulble nd of the ory a nd pra ctice tha t ha s ma de Se dge w ick's w ork a n inva lua ble re source for more tha n 400,000 progra mme rs!
This pa rticula r book, Pa rts 1-4, re pre se nts the e sse ntia l first ha lf of Se dge w ick's comple te w ork It provide s e xte nsive cove ra ge of funda me nta l
da ta structure s a nd a lgorithms for sorting, se a rching, a nd re la te d a pplica tions Although the substa nce of the book a pplie s to progra mming in
a ny la ngua ge , the imple me nta tions by Schidlow sky a nd Se dge w ick a lso e xploit the na tura l ma tch be tw e e n Ja va cla sse s a nd a bstra ct da ta type(ADT) imple me nta tions
Highlights
Ja va cla ss imple me nta tions of more tha n 100 importa nt pra ctica l a lgorithms
Empha sis on ADTs, modula r progra mming, a nd obje ct-orie nte d progra mming
Exte nsive cove ra ge of a rra ys, linke d lists, tre e s, a nd othe r funda me nta l da ta structure s
Thorough tre a tme nt of a lgorithms for sorting, se le ction, priority que ue ADT imple me nta tions, a nd symbol ta ble ADT imple me nta tions(se a rch a lgorithms)
Comple te imple me nta tions for binomia l que ue s, multiw a y ra dix sorting, ra ndomize d BSTs, spla y tre e s, skip lists, multiw a y trie s, B tre e s,
e xte ndible ha shing, a nd ma ny othe r a dva nce d me thods
Qua ntita tive informa tion a bout the a lgorithms tha t give s you a ba sis for compa ring the m
More tha n 1,000 e xe rcise s a nd more tha n 250 de ta ile d figure s to he lp you le a rn prope rtie s of the a lgorithms
W he the r you a re le a rning the a lgorithms for the first time or w ish to ha ve up-to-da te re fe re nce ma te ria l tha t incorpora te s ne w progra mming
Trang 2Algorithms in Java: Parts 1-4, Third Edition
Copyright
Pre fa ce
Scope
Use in the Curriculum
Algorithms of Pra ctica l Use
Progra mming La ngua ge
Se ction 1.2 A Sa mple Proble m: Conne ctivity
Se ction 1.3 Union–Find Algorithms
Se ction 1.4 Pe rspe ctive
Se ction 1.5 Summa ry of Topics
Chapter 2 Principles of Algorithm Analysis
Se ction 2.1 Imple me nta tion a nd Empirica l Ana lysis
Se ction 2.2 Ana lysis of Algorithms
Se ction 2.3 Grow th of Functions
Se ction 2.4 Big-Oh Nota tion
Se ction 2.5 Ba sic Re curre nce s
Se ction 2.6 Exa mple s of Algorithm Ana lysis
Se ction 2.7 Gua ra nte e s, Pre dictions, a nd Limita tions
Re fe re nce s for Pa rt One
Part II: Data Structures
Chapter 3 Elementary Data Structures
Se ction 3.1 Building Blocks
Se ction 3.2 Arra ys
Se ction 3.3 Linke d Lists
Se ction 3.4 Ele me nta ry List Proce ssing
Se ction 3.5 Me mory Alloca tion for Lists
Se ction 3.6 Strings
Se ction 3.7 Compound Da ta Structure s
Chapter 4 Abstract Data Types
Exe rcise s
Se ction 4.1 Colle ctions of Ite ms
Se ction 4.2 Pushdow n Sta ck ADT
Se ction 4.3 Exa mple s of Sta ck ADT Clie nts
Se ction 4.4 Sta ck ADT Imple me nta tions
Se ction 4.5 Ge ne ric Imple me nta tions
Se ction 4.6 Cre a tion of a Ne w ADT
Se ction 4.7 FIFO Que ue s a nd Ge ne ra lize d Que ue s
Se ction 4.8 Duplica te a nd Inde x Ite ms
Se ction 4.9 First-Cla ss ADTs
Se ction 4.10 Applica tion-Ba se d ADT Exa mple
Se ction 4.11 Pe rspe ctive
Chapter 5 Recursion and Trees
Se ction 5.1 Re cursive Algorithms
Se ction 5.2 Divide a nd Conque r
Se ction 5.3 Dyna mic Progra mming
Se ction 5.4 Tre e s
Se ction 5.5 Ma the ma tica l Prope rtie s of Bina ry Tre e s
Se ction 5.6 Tre e Tra ve rsa l
Se ction 5.7 Re cursive Bina ry-Tre e Algorithms
Se ction 5.8 Gra ph Tra ve rsa l
Se ction 5.9 Pe rspe ctive
Re fe re nce s for Pa rt Tw o
Part III: Sorting
Chapter 6 Elementary Sorting Methods
Se ction 6.1 Rule s of the Ga me
Se ction 6.2 Ge ne ric Sort Imple me nta tions
Se ction 6.3 Se le ction Sort
Se ction 6.4 Inse rtion Sort
Se ction 6.5 Bubble Sort
Se ction 6.6 Pe rforma nce Cha ra cte ristics of Ele me nta ry Sorts
Se ction 6.7 Algorithm Visua liza tion
Se ction 6.8 She llsort
Se ction 6.9 Sorting of Linke d Lists
Trang 3Se ction 7.1 The Ba sic Algorithm
Se ction 7.2 Pe rforma nce Cha ra cte ristics of Quicksort
Se ction 7.3 Sta ck Size
Se ction 7.4 Sma ll Subfile s
Se ction 7.5 Me dia n-of-Thre e Pa rtitioning
Se ction 7.6 Duplica te Ke ys
Se ction 7.7 Strings a nd Ve ctors
Se ction 7.8 Se le ction
Chapter 8 Merging and Mergesort
Se ction 8.1 Tw o-W a y Me rging
Se ction 8.2 Abstra ct In-Pla ce Me rge
Se ction 8.3 Top-Dow n Me rge sort
Se ction 8.4 Improve me nts to the Ba sic Algorithm
Se ction 8.5 Bottom-Up Me rge sort
Se ction 8.6 Pe rforma nce Cha ra cte ristics of Me rge sort
Se ction 8.7 Linke d-List Imple me nta tions of Me rge sort
Se ction 8.8 Re cursion Re visite d
Chapter 9 Priority Q ueues and Heapsort
Se ction 9.5 Priority-Que ue ADT
Se ction 9.6 Priority Que ue s for Clie nt Arra ys
Se ction 9.7 Binomia l Que ue s
Chapter 10 Radix Sorting
Se ction 10.1 Bits, Byte s, a nd W ords
Se ction 10.2 Bina ry Quicksort
Se ction 10.3 MSD Ra dix Sort
Se ction 10.4 Thre e -W a y Ra dix Quicksort
Se ction 10.5 LSD Ra dix Sort
Se ction 10.6 Pe rforma nce Cha ra cte ristics of Ra dix Sorts
Se ction 10.7 Subline a r-Time Sorts
Chapter 11 Special-Purpose Sorting Methods
Se ction 11.1 Ba tche r's Odd–Eve n Me rge sort
Se ction 11.2 Sorting Ne tw orks
Se ction 11.3 Sorting In Pla ce
Se ction 11.4 Exte rna l Sorting
Se ction 11.5 Sort–Me rge Imple me nta tions
Se ction 11.6 Pa ra lle l Sort–Me rge
Re fe re nce s for Pa rt Thre e
Part IV: Searching
Chapter 12 Symbol Tables and Binary Search Trees
Se ction 12.1 Symbol-Ta ble Abstra ct Da ta Type
Se ction 12.2 Ke y-Inde xe d Se a rch
Se ction 12.3 Se que ntia l Se a rch
Se ction 12.4 Bina ry Se a rch
Se ction 12.5 Inde x Imple me nta tions w ith Symbol Ta ble s
Se ction 12.6 Bina ry Se a rch Tre e s
Se ction 12.7 Pe rforma nce Cha ra cte ristics of BSTs
Se ction 12.8 Inse rtion a t the Root in BSTs
Se ction 12.9 BST Imple me nta tions of Othe r ADT Ope ra tions
Chapter 13 Balanced Trees
Exe rcise s
Se ction 13.1 Ra ndomize d BSTs
Se ction 13.2 Spla y BSTs
Se ction 13.3 Top-Dow n 2-3-4 Tre e s
Se ction 13.4 Re d–Bla ck Tre e s
Se ction 13.5 Skip Lists
Se ction 13.6 Pe rforma nce Cha ra cte ristics
Chapter 14 Hashing
Se ction 14.1 Ha sh Functions
Se ction 14.2 Se pa ra te Cha ining
Se ction 14.3 Line a r Probing
Se ction 14.4 Double Ha shing
Se ction 14.5 Dyna mic Ha sh Ta ble s
Se ction 14.6 Pe rspe ctive
Chapter 15 Radix Search
Se ction 15.1 Digita l Se a rch Tre e s
Se ction 15.2 Trie s
Se ction 15.3 Pa tricia Trie s
Se ction 15.4 Multiw a y Trie s a nd TSTs
Se ction 15.5 Te xt-String–Inde x Algorithms
Chapter 16 External Searching
Se ction 16.1 Rule s of the Ga me
Se ction 16.2 Inde xe d Se que ntia l Acce ss
Trang 4Se ction 16.4 Exte ndible Ha shing
Se ction 16.5 Pe rspe ctive
Re fe re nce s for Pa rt Four
Appendix
Exe rcise s
Top
Trang 6Ma ny of the de signa tions use d by ma nufa cture rs a nd se lle rs to distinguish the ir products a re cla ime d a s tra de ma rks W he re those de signa tions
a ppe a r in this book a nd Addison-We sle y w a s a w a re of a tra de ma rk cla im, the de signa tions ha ve be e n printe d in initia l ca pita l le tte rs or a ll
ca pita ls
The a uthor a nd publishe r ha ve ta ke n ca re in the pre pa ra tion of this book, but ma ke no e xpre sse d or implie d w a rra nty of a ny kind a nd a ssume no
re sponsibility for e rrors or omissions No lia bility is a ssume d for incide nta l or conse que ntia l da ma ge s in conne ction w ith or a rising out of the use
of the informa tion or progra ms conta ine d he re in
The publishe r offe rs discounts on this book w he n orde re d in qua ntity for spe cia l sa le s For more informa tion, ple a se conta ct:
U.S Corpora te a nd Gove rnme nt Sa le s
(800) 382-3410
corpsa le s@pe a rsonte chgroup.com
For sa le s outside of the Unite d Sta te s, ple a se conta ct:
Inte rna tiona l Sa le s
(317) 581-3793
inte rna tiona l@pe a rsonte chgroup.com
Visit Addison-W e sle y on the W e b: w w w a w profe ssiona l.com
Library of Congress Cataloging-in-Publication Data
Se dge w ick, Robe rt, 1946 –
Algorithms in Ja va / Robe rt Se dge w ick — 3d e d
p cm
Include s bibliogra phica l re fe re nce s a nd inde x
Conte nts: v 1, pts 1–4 Funda me nta ls, da ta structure s, sorting, se a rching
1 Ja va (Compute r progra m la ngua ge ) 2 Compute r a lgorithms
I Title
QA76.73.C15S 2003
005.13'3—dc20 92-901
CIP
Copyright © 2003 by Pe a rson Educa tion, Inc
All rights re se rve d No pa rt of this publica tion ma y be re produce d, store d in a re trie va l syste m, or tra nsmitte d, in a ny form or by a ny me a ns,
e le ctronic, me cha nica l, photocopying, re cording, or othe rw ise , w ithout the prior w ritte n pe rmission of the publishe r Printe d in the Unite d Sta te s
of Ame rica Publishe d simulta ne ously in Ca na da
For informa tion on obta ining pe rmission for use of ma te ria l from this w ork, ple a se submit a w ritte n re que st to:
Pe a rson Educa tion, Inc
75 Arlington Stre e t, Suite 300
Trang 7This book is the first of thre e volume s tha t a re inte nde d to surve y the most importa nt compute r a lgorithms in use toda y This first volume (Pa rtsI–IV) cove rs funda me nta l conce pts (Pa rt I), da ta structure s (Pa rt II), sorting a lgorithms (Pa rt III), a nd se a rching a lgorithms (Pa rt IV); the se condvolume (Pa rt 5) cove rs gra phs a nd gra ph a lgorithms; a nd the (ye t to be publishe d) third volume (Pa rts 6–8) cove rs strings (Pa rt 6),computa tiona l ge ome try (Pa rt 7), a nd a dva nce d a lgorithms a nd a pplica tions (Pa rt 8)
The books a re use ful a s te xts e a rly in the compute r scie nce curriculum, a fte r stude nts ha ve a cquire d ba sic progra mming skills a nd fa milia rity w ithcompute r syste ms, but be fore the y ha ve ta ke n spe cia lize d course s in a dva nce d a re a s of compute r scie nce or compute r a pplica tions The books
a lso a re use ful for se lf-study or a s a re fe re nce for pe ople e nga ge d in the de ve lopme nt of compute r syste ms or a pplica tions progra ms be ca usethe y conta in imple me nta tions of use ful a lgorithms a nd de ta ile d informa tion on the se a lgorithms' pe rforma nce cha ra cte ristics The broa d
pe rspe ctive ta ke n ma ke s the se rie s a n a ppropria te introduction to the fie ld
Toge the r the thre e volume s comprise the Third Edition of a book tha t ha s be e n w ide ly use d by stude nts a nd progra mme rs a round the w orld for
ma ny ye a rs I ha ve comple te ly re w ritte n the te xt for this e dition, a nd I ha ve a dde d thousa nds of ne w e xe rcise s, hundre ds of ne w figure s,doze ns of ne w progra ms, a nd de ta ile d comme nta ry on a ll the figure s a nd progra ms This ne w ma te ria l provide s both cove ra ge of ne w topics a ndfulle r e xpla na tions of ma ny of the cla ssic a lgorithms A ne w e mpha sis on a bstra ct da ta type s throughout the books ma ke s the progra ms morebroa dly use ful a nd re le va nt in mode rn obje ct-orie nte d progra mming e nvironme nts Pe ople w ho ha ve re a d pre vious e ditions w ill find a w e a lth of
ne w informa tion throughout; a ll re a de rs w ill find a w e a lth of pe da gogica l ma te ria l tha t provide s e ffe ctive a cce ss to e sse ntia l conce pts
The se books a re not just for progra mme rs a nd compute r scie nce stude nts Eve ryone w ho use s a compute r w a nts it to run fa ste r or to solve
la rge r proble ms The a lgorithms tha t w e conside r re pre se nt a body of know le dge de ve lope d during the la st 50 ye a rs tha t is the ba sis for the
e fficie nt use of the compute r for a broa d va rie ty of a pplica tions From N-body simula tion proble ms in physics to ge ne tic-se que ncing proble ms in
mole cula r biology, the ba sic me thods de scribe d he re ha ve be come e sse ntia l in scie ntific re se a rch; a nd from da ta ba se syste ms to Inte rne t se a rch
e ngine s, the y ha ve be come e sse ntia l pa rts of mode rn softw a re syste ms As the scope of compute r a pplica tions be come s more w ide spre a d, sogrow s the impa ct of ba sic a lgorithms The goa l of this book is to se rve a s a re source so tha t stude nts a nd profe ssiona ls ca n know a nd ma keinte llige nt use of the se funda me nta l a lgorithms a s the ne e d a rise s in w ha te ve r compute r a pplica tion the y might unde rta ke
Previous Section
Top
Trang 8This book, Algorithms in Java, Third Edition, Parts 1-4, conta ins 16 cha pte rs groupe d into four ma jor pa rts: funda me nta ls, da ta structure s, sorting,
a nd se a rching The de scriptions he re a re inte nde d to give re a de rs a n unde rsta nding of the ba sic prope rtie s of a s broa d a ra nge of funda me nta l
a lgorithms a s possible The a lgorithms de scribe d he re ha ve found w ide spre a d use for ye a rs, a nd re pre se nt a n e sse ntia l body of know le dge forboth the pra cticing progra mme r a nd the compute r-scie nce stude nt The se cond volume is de vote d to gra ph a lgorithms, a nd the third consists offour a dditiona l pa rts tha t cove r strings, ge ome try, a nd a dva nce d topics My prima ry goa l in de ve loping the se books ha s be e n to bring toge the rfunda me nta l me thods from the se a re a s, to provide a cce ss to the be st me thods know n for solving proble ms by compute r
You w ill most a ppre cia te the ma te ria l he re if you ha ve ha d one or tw o pre vious course s in compute r scie nce or ha ve ha d e quiva le nt progra mming
e xpe rie nce : one course in progra mming in a high-le ve l la ngua ge such a s Ja va , C, or C++, a nd pe rha ps a nothe r course tha t te a che s funda me nta lconce pts of progra mming syste ms This book is thus inte nde d for a nyone conve rsa nt w ith a mode rn progra mming la ngua ge a nd w ith the ba sic
fe a ture s of mode rn compute r syste ms Re fe re nce s tha t might he lp to fill in ga ps in your ba ckground a re sugge ste d in the te xt
Most of the ma the ma tica l ma te ria l supporting the a na lytic re sults is se lf-conta ine d (or is la be le d a s be yond the scope of this book), so littlespe cific pre pa ra tion in ma the ma tics is re quire d for the bulk of the book, a lthough ma the ma tica l ma turity is de finite ly he lpful
Previous Section
Top
Trang 9Use in the Curriculum
The re is a gre a t de a l of fle xibility in how the ma te ria l he re ca n be ta ught, de pe nding on the ta ste of the instructor a nd the pre pa ra tion of thestude nts The re is sufficie nt cove ra ge of ba sic ma te ria l for the book to be use d to te a ch da ta structure s to be ginne rs, a nd the re is sufficie nt
de ta il a nd cove ra ge of a dva nce d ma te ria l for the book to be use d to te a ch the de sign a nd a na lysis of a lgorithms to uppe r-le ve l stude nts Someinstructors ma y w ish to e mpha size imple me nta tions a nd pra ctica l conce rns; othe rs ma y w ish to e mpha size a na lysis a nd the ore tica l conce pts
An e le me nta ry course on da ta structure s a nd a lgorithms might e mpha size the ba sic da ta structure s in Pa rt II a nd the ir use in theimple me nta tions in Pa rts III a nd IV A course on de sign a nd a na lysis of a lgorithms might e mpha size the funda me nta l ma te ria l in Pa rt I a ndCha pte r 5, the n study the w a ys in w hich the a lgorithms in Pa rts III a nd IV a chie ve good a symptotic pe rforma nce A course on softw a re
e ngine e ring might omit the ma the ma tica l a nd a dva nce d a lgorithmic ma te ria l, a nd e mpha size how to inte gra te the imple me nta tions give n he reinto la rge progra ms or syste ms A course on a lgorithms might ta ke a surve y a pproa ch a nd introduce conce pts from a ll the se a re a s
Ea rlie r e ditions of this book tha t a re ba se d on othe r progra mming la ngua ge s ha ve be e n use d a t score s of colle ge s a nd unive rsitie s a s a te xt forthe se cond or third course in compute r scie nce a nd a s supple me nta l re a ding for othe r course s At Prince ton, our e xpe rie nce ha s be e n tha t thebre a dth of cove ra ge of ma te ria l in this book provide s our ma jors w ith a n introduction to compute r scie nce tha t ca n be e xpa nde d on in la te rcourse s on a na lysis of a lgorithms, syste ms progra mming, a nd the ore tica l compute r scie nce , w hile providing the grow ing group of stude nts fromothe r discipline s w ith a la rge se t of te chnique s tha t the se pe ople ca n put to good use imme dia te ly
The e xe rcise s—ne a rly a ll of w hich a re ne w to this third e dition—fa ll into se ve ra l type s Some a re inte nde d to te st unde rsta nding of ma te ria l inthe te xt, a nd simply a sk re a de rs to w ork through a n e xa mple or to a pply conce pts de scribe d in the te xt Othe rs involve imple me nting a nd puttingtoge the r the a lgorithms, or running e mpirica l studie s to compa re va ria nts of the a lgorithms a nd to le a rn the ir prope rtie s Still othe rs a re a
re pository for importa nt informa tion a t a le ve l of de ta il tha t is not a ppropria te for the te xt Re a ding a nd thinking a bout the e xe rcise s w ill pa ydivide nds for e ve ry re a de r
Previous Section
Top
Trang 10Algorithms of Practical Use
Anyone w a nting to use a compute r more e ffe ctive ly ca n use this book for re fe re nce or for se lf-study Pe ople w ith progra mming e xpe rie nce ca nfind informa tion on spe cific topics throughout the book To a la rge e xte nt, you ca n re a d the individua l cha pte rs in the book inde pe nde ntly of theothe rs, a lthough, in some ca se s, a lgorithms in one cha pte r ma ke use of me thods from a pre vious cha pte r
The orie nta tion of the book is to study a lgorithms like ly to be of pra ctica l use The book provide s informa tion a bout the tools of the tra de to thepoint tha t re a de rs ca n confide ntly imple me nt, de bug, a nd put a lgorithms to w ork to solve a proble m or to provide functiona lity in a n a pplica tion.Full imple me nta tions of the me thods discusse d a re include d, a s a re de scriptions of the ope ra tions of the se progra ms on a consiste nt se t of
e xa mple s
Be ca use w e w ork w ith re a l code , ra the r tha n w rite pse udo-code , you ca n put the progra ms to pra ctica l use quickly Progra m listings a re a va ila blefrom the book's home pa ge You ca n use the se w orking progra ms in ma ny w a ys to he lp you study a lgorithms Re a d the m to che ck yourunde rsta nding of the de ta ils of a n a lgorithm, or to se e one w a y to ha ndle initia liza tions, bounda ry conditions, a nd othe r a w kw a rd situa tions tha tofte n pose progra mming cha lle nge s Run the m to se e the a lgorithms in a ction, to study pe rforma nce e mpirica lly a nd che ck your re sults a ga instthe ta ble s in the book, or to try your ow n modifica tions
Cha ra cte ristics of the a lgorithms a nd of the situa tions in w hich the y might be use ful a re discusse d in de ta il Conne ctions to the a na lysis of
a lgorithms a nd the ore tica l compute r scie nce a re de ve lope d in con-te xt W he n a ppropria te , e mpirica l a nd a na lytic re sults a re pre se nte d toillustra te w hy ce rta in a lgorithms a re pre fe rre d W he n inte re sting, the re la tionship of the pra ctica l a lgorithms be ing discusse d to pure ly the ore tica l
re sults is de scribe d Spe cific informa tion on pe rforma nce cha ra cte ristics of a lgorithms a nd imple me nta tions is synthe size d, e nca psula te d, a nddiscusse d throughout the book
Previous Section
Top
Trang 11Programming Language
The progra mming la ngua ge use d for a ll of the imple me nta tions is Ja va The progra ms use a w ide ra nge of sta nda rd Ja va idioms, a nd the te xtinclude s concise de scriptions of e a ch construct
Mike Schidlow sky a nd I de ve lope d a style of Ja va progra mming ba se d on a bstra ct da ta type s tha t w e fe e l is a n e ffe ctive w a y to pre se nt the
a lgorithms a nd da ta structure s a s re a l progra ms We ha ve strive n for e le ga nt, compa ct, e fficie nt, a nd porta ble imple me nta tions The style isconsiste nt w he ne ve r possible , so progra ms tha t a re simila r look simila r
For ma ny of the a lgorithms in this book, the simila ritie s hold re ga rdle ss of the la ngua ge : Quicksort is quicksort (to pick one promine nt e xa mple ),
w he the r e xpre sse d in Ada , Algol-60, Ba sic, C, C++, Fortra n, Ja va , Me sa , Modula -3, Pa sca l, PostScript, Sma llta lk, or countle ss othe r progra mming
la ngua ge s a nd e nvironme nts w he re it ha s prove d to be a n e ffe ctive sorting me thod On the one ha nd, our code is informe d by e xpe rie nce w ithimple me nting a lgorithms in the se a nd nume rous othe r la ngua ge s (C a nd C++ ve rsions of this book a re a lso a va ila ble ); on the othe r ha nd, some
of the prope rtie s of some of the se la ngua ge s a re informe d by the ir de signe rs' e xpe rie nce w ith some of the a lgorithms a nd da ta structure s tha t
w e conside r in this book
Cha pte r 1 constitute s a de ta ile d e xa mple of this a pproa ch to de ve loping e fficie nt Ja va imple me nta tions of our a lgorithms, a nd Cha pte r 2
de scribe s our a pproa ch to a na lyzing the m Cha pte rs 3 a nd 4 a re de vote d to de scribing a nd justifying the ba sic me cha nisms tha t w e use for da tatype a nd ADT imple me nta tions The se four cha pte rs se t the sta ge for the re st of the book
Previous Section
Top
Trang 12to tha nk the ma ny re a de rs w ho ha ve provide d me w ith comme nts a bout va rious e ditions, including Guy Alme s, Jon Be ntle y, Ma rc Brow n, Ja yGische r, Alla n He ydon, Ke nne dy Le mke , Udi Ma nbe r, Da na Richa rds, John Re if, M Rose nfe ld, Ste phe n Se idma n, Micha e l Quinn, a nd W illia m W a rd.
To produce this ne w e dition, I ha ve ha d the ple a sure of w orking w ith Pe te r Gordon a nd He le n Goldste in a t Addison-We sle y, w ho ha ve pa tie ntlyshe phe rde d this proje ct a s it ha s e volve d It ha s a lso be e n my ple a sure to w ork w ith se ve ra l othe r me mbe rs of the profe ssiona l sta ff a t Addison-
We sle y The na ture of this proje ct ma de the book a some w ha t unusua l cha lle nge for ma ny of the m, a nd I much a ppre cia te the ir forbe a ra nce In
pa rticula r, Ma rilyn Ra sh did a n outsta nding job ma na ging the book's production w ithin a tightly compre sse d sche dule
I ha ve ga ine d thre e ne w me ntors in w riting this book, a nd pa rticula rly w a nt to e xpre ss my a ppre cia tion to the m First, Ste ve Summit ca re fullyche cke d e a rly ve rsions of the ma nuscript on a te chnica l le ve l a nd provide d me w ith lite ra lly thousa nds of de ta ile d comme nts, pa rticula rly on theprogra ms Ste ve cle a rly unde rstood my goa l of providing e le ga nt, e fficie nt, a nd e ffe ctive imple me nta tions, a nd his comme nts not only he lpe d me
to provide a me a sure of consiste ncy a cross the imple me nta tions, but a lso he lpe d me to improve ma ny of the m substa ntia lly Se cond, Lyn Dupré e
a lso provide d me w ith thousa nds of de ta ile d comme nts on the ma nuscript, w hich w e re inva lua ble in he lping me not only to corre ct a nd a voidgra mma tica l e rrors, but a lso—more importa nt—to find a consiste nt a nd cohe re nt w riting style tha t he lps bind toge the r the da unting ma ss of
te chnica l ma te ria l he re Third, Chris Va n W yk, in a long se rie s of spirite d e le ctronic ma il e xcha nge s, pa tie ntly de fe nde d the ba sic pre ce pts ofobje ct-orie nte d progra mming a nd he lpe d me de ve lop a style of coding tha t e xhibits the a lgorithms w ith cla rity a nd pre cision w hile still ta king
a dva nta ge of w ha t obje ct-orie nte d progra mming ha s to offe r The ba sic a pproa ch tha t w e de ve lope d for the C++ ve rsion of this book ha ssubsta ntia lly influe nce d the Ja va code he re a nd w ill ce rta inly influe nce future volume s in both la ngua ge s (a nd C a s w e ll) I a m e xtre me ly gra te fulfor the opportunity to le a rn from Ste ve , Lyn, a nd Chris—the ir input w a s vita l in the de ve lopme nt of this book
Much of w ha t I ha ve w ritte n he re I ha ve le a rne d from the te a ching a nd w ritings of Don Knuth, my a dvisor a t Sta nford Although Don ha d nodire ct influe nce on this w ork, his pre se nce ma y be fe lt in the book, for it w a s he w ho put the study of a lgorithms on the scie ntific footing tha t
ma ke s a w ork such a s this possible My frie nd a nd colle a gue Philippe Fla jole t, w ho ha s be e n a ma jor force in the de ve lopme nt of the a na lysis of
a lgorithms a s a ma ture re se a rch a re a , ha s ha d a simila r influe nce on this w ork
I a m de e ply tha nkful for the support of Prince ton Unive rsity, Brow n Unive rsity, a nd the Institut Na tiona l de Re che rche e n Informa tique e tAutoma tique (INRIA), w he re I did most of the w ork on the book; a nd of the Institute for De fe nse Ana lyse s a nd the Xe rox Pa lo Alto Re se a rch
Ce nte r, w he re I did some w ork on the book w hile visiting Ma ny pa rts of the book a re de pe nde nt on re se a rch tha t ha s be e n ge ne rouslysupporte d by the Na tiona l Scie nce Founda tion a nd the Office of Na va l Re se a rch Fina lly, I tha nk Bill Bow e n, Aa ron Le monick, a nd Ne il Rude nstinefor the ir support in building a n a ca de mic e nvironme nt a t Prince ton in w hich I w a s a ble to pre pa re this book, de spite my nume rous othe r
re sponsibilitie s
Robert Sedgewick
Marly-le-Roi, France, 1983
Princeton, New Jersey, 1990, 1992
Jamestown, Rhode Island, 1997
Princeton, New Jersey, 1998, 2002
Previous Section
Top
Trang 13Java Consultant's Preface
In the pa st de ca de , Ja va ha s be come the la ngua ge of choice for a va rie ty of a pplica tions But Ja va de ve lope rs ha ve found the mse lve s re pe a te dly
re fe rring to re fe re nce s such a s Se dge w ick's Algorithms in C for solutions to common progra mming proble ms The re ha s long be e n a n e mpty spa ce
on the bookshe lf for a compa ra ble re fe re nce w ork for Ja va ; this book is he re to fill tha t spa ce
We w rote the sa mple progra ms a s utility me thods to be use d in a va rie ty of conte xts To tha t e nd, w e did not use the Ja va pa cka ge me cha nism
To focus on the a lgorithms a t ha nd (a nd to e xpose the a lgorithmic ba sis of ma ny funda me nta l libra ry cla sse s), w e a voide d the sta nda rd Ja valibra ry in fa vor of more funda me nta l type s Prope r e rror che cking a nd othe r de fe nsive pra ctice s w ould both substa ntia lly incre a se the a mount ofcode a nd distra ct the re a de r from the core a lgorithms De ve lope rs should introduce such code w he n using the progra ms in la rge r a pplica tions.Although the a lgorithms w e pre se nt a re la ngua ge inde pe nde nt, w e ha ve pa id close a tte ntion to Ja va -spe cific pe rforma nce issue s The timingsthroughout the book a re provide d a s one conte xt for compa ring a lgorithms, a nd w ill va ry de pe nding on the virtua l ma chine As Ja va
e nvironme nts e volve , progra ms w ill pe rform a s fa st a s na tive ly compile d code , but such optimiza tions w ill not cha nge the pe rforma nce of
a lgorithms re la tive to one a nothe r W e provide the timings a s a use ful re fe re nce for such compa risons
I w ould like to tha nk Mike Za ma nsky, for his me ntorship a nd de votion to the te a ching of compute r scie nce , a nd Da nie l Cha ske s, Ja son Sa nde rs,
a nd Ja me s Pe rcy, for the ir unw a ve ring support I w ould a lso like to tha nk my fa mily for the ir support a nd for the compute r tha t bore my firstprogra ms Bringing toge the r Ja va w ith the cla ssic a lgorithms of compute r scie nce w a s a n e xciting e nde a vor for w hich I a m ve ry gra te ful Tha nkyou, Bob, for the opportunity to do so
Michael Schidlowsky
Oakland Gardens, New York, 2002
Previous Section
Top
Trang 14Notes on Exercises
Cla ssifying e xe rcise s is a n a ctivity fra ught w ith pe ril be ca use re a de rs of a book such a s this come to the ma te ria l w ith va rious le ve ls ofknow le dge a nd e xpe rie nce None the le ss, guida nce is a ppropria te , so ma ny of the e xe rcise s ca rry one of four a nnota tions to he lp you de cide how
to a pproa ch the m
Exe rcise s tha t test your understanding of the ma te ria l a re ma rke d w ith a n ope n tria ngle , a s follow s:
9.57 Give the binomia l que ue tha t re sults w he n the ke ys E A S Y Q U E S T I O N a re inse rte d into a n initia lly e mpty binomia l
que ue
Most ofte n, such e xe rcise s re la te dire ctly to e xa mple s in the te xt The y should pre se nt no spe cia l difficulty, but w orking the m might te a ch you a
fa ct or conce pt tha t ma y ha ve e lude d you w he n you re a d the te xt
Exe rcise s tha t add new and thought-provoking informa tion to the ma te ria l a re ma rke d w ith a n ope n circle , a s follow s:
14.20 W rite a progra m tha t inse rts N ra ndom inte ge rs into a ta ble of size N/100 using se pa ra te cha ining, the n finds the le ngth of
the shorte st a nd longe st lists, for N = 103, 104, 105, a nd 106
Such e xe rcise s e ncoura ge you to think a bout a n importa nt conce pt tha t is re la te d to the ma te ria l in the te xt, or to a nsw e r a que stion tha t ma y
ha ve occurre d to you w he n you re a d the te xt You ma y find it w orthw hile to re a d the se e xe rcise s, e ve n if you do not ha ve the time to w ork the mthrough
Exe rcise s tha t a re inte nde d to challenge you a re ma rke d w ith a bla ck dot, a s follow s:
• 8.46 Suppose tha t me rge sort is imple me nte d to split the file a t a random position, ra the r tha n e xa ctly in the middle How ma ny
compa risons a re use d by such a me thod to sort N e le me nts, on the a ve ra ge ?
Such e xe rcise s ma y re quire a substa ntia l a mount of time to comple te , de pe nding on your e xpe rie nce Ge ne ra lly, the most productive a pproa ch is
to w ork on the m in a fe w diffe re nt sittings
A fe w e xe rcise s tha t a re extremely difficult (by compa rison w ith most othe rs) a re ma rke d w ith tw o bla ck dots, a s follow s:
•• 15.29 Prove tha t the he ight of a trie built from N ra ndom bitstrings is a bout 2lg N.
The se e xe rcise s a re simila r to que stions tha t might be a ddre sse d in the re se a rch lite ra ture , but the ma te ria l in the book ma y pre pa re you to
e njoy trying to solve the m (a nd pe rha ps succe e ding)
The a nnota tions a re inte nde d to be ne utra l w ith re spe ct to your progra mming a nd ma the ma tica l a bility Those e xe rcise s tha t re quire e xpe rtise inprogra mming or in ma the ma tica l a na lysis a re se lf-e vide nt All re a de rs a re e ncoura ge d to te st the ir unde rsta nding of the a lgorithms byimple me nting the m Still, a n e xe rcise such a s this one is stra ightforw a rd for a pra cticing progra mme r or a stude nt in a progra mming course , but
ma y re quire substa ntia l w ork for some one w ho ha s not re ce ntly progra mme d:
1.23 Modify Progra m 1.4 to ge ne ra te ra ndom pa irs of inte ge rs be tw e e n 0 a nd N - 1 inste a d of re a ding the m from sta nda rd input,
a nd to loop until N - 1 union ope ra tions ha ve be e n pe rforme d Run your progra m for N = 103, 104, 105, a nd 106 a nd print out the
tota l numbe r of e dge s ge ne ra te d for e a ch va lue of N.
In a simila r ve in, a ll re a de rs a re e ncoura ge d to strive to a ppre cia te the a na lytic unde rpinnings of our know le dge a bout prope rtie s of a lgorithms.Still, a n e xe rcise such a s this one is stra ightforw a rd for a scie ntist or a stude nt in a discre te ma the ma tics course , but ma y re quire substa ntia l
w ork for some one w ho ha s not re ce ntly done ma the ma tica l a na lysis:
1.13 Compute the average dista nce from a node to the root in a w orst-ca se tre e of 2 n node s built by the w e ighte d quick-union
Trang 15Part I: Fundamentals
Cha pte r 1 Introduction
Cha pte r 2 Principle s of Algorithm Ana lysis
Re fe re nce s for Pa rt One
Previous Section
Top
Trang 16Chapter 1 Introduction
The obje ctive of this book is to study a broa d va rie ty of importa nt a nd use ful algorithms—me thods for solving proble ms tha t a re suite d for
compute r imple me nta tion We sha ll de a l w ith ma ny diffe re nt a re a s of a pplica tion, a lw a ys conce ntra ting on funda me nta l a lgorithms tha t a reimporta nt to know a nd inte re sting to study We sha ll spe nd e nough time on e a ch a lgorithm to unde rsta nd its e sse ntia l cha ra cte ristics a nd to
re spe ct its subtle tie s Our goa l is to le a rn w e ll e nough to be a ble to use a nd a ppre cia te a la rge numbe r of the most importa nt a lgorithms use d
on compute rs toda y
The stra te gy tha t w e use for unde rsta nding the progra ms pre se nte d in this book is to imple me nt a nd te st the m, to e xpe rime nt w ith the ir
va ria nts, to discuss the ir ope ra tion on sma ll e xa mple s, a nd to try the m out on la rge r e xa mple s simila r to w ha t w e might e ncounte r in pra ctice
We sha ll use the Ja va progra mming la ngua ge to de scribe the a lgorithms, thus providing use ful imple me nta tions a t the sa me time Our progra ms
ha ve a uniform style tha t is a me na ble to tra nsla tion into othe r mode rn progra mming la ngua ge s a s w e ll
We a lso pa y ca re ful a tte ntion to pe rforma nce cha ra cte ristics of our a lgorithms in orde r to he lp us de ve lop improve d ve rsions, compa re diffe re nt
a lgorithms for the sa me ta sk, a nd pre dict or gua ra nte e pe rforma nce for la rge proble ms Unde rsta nding how the a lgorithms pe rform might re quire
e xpe rime nta tion or ma the ma tica l a na lysis or both We conside r de ta ile d informa tion for ma ny of the most importa nt a lgorithms, de ve loping
a na lytic re sults dire ctly w he n fe a sible , or ca lling on re sults from the re se a rch lite ra ture w he n ne ce ssa ry
To illustra te our ge ne ra l a pproa ch to de ve loping a lgorithmic solutions, w e conside r in this cha pte r a de ta ile d e xa mple comprising a numbe r of
a lgorithms tha t solve a pa rticula r proble m The proble m tha t w e conside r is not a toy proble m; it is a funda me nta l computa tiona l ta sk, a nd thesolution tha t w e de ve lop is of use in a va rie ty of a pplica tions We sta rt w ith a simple solution, the n se e k to unde rsta nd tha t solution's
pe rforma nce cha ra cte ristics, w hich he lp us to se e how to improve the a lgorithm Afte r a fe w ite ra tions of this proce ss, w e come to a n e fficie nt
a nd use ful a lgorithm for solving the proble m This prototypica l e xa mple se ts the sta ge for our use of the sa me ge ne ra l me thodology throughoutthe book
We conclude the cha pte r w ith a short discussion of the conte nts of the book, including brie f de scriptions of w ha t the ma jor pa rts of the book a re
a nd how the y re la te to one a nothe r
Previous Section
Top
Trang 171.1 Algorithms
W he n w e w rite a compute r progra m, w e a re ge ne ra lly imple me nting a me thod tha t ha s be e n de vise d pre viously to solve some proble m This
me thod is ofte n inde pe nde nt of the pa rticula r compute r to be use d—it is like ly to be e qua lly a ppropria te for ma ny compute rs a nd ma ny compute r
la ngua ge s It is the me thod, ra the r tha n the compute r progra m itse lf, tha t w e must study to le a rn how the proble m is be ing a tta cke d The te rm
algorithm is use d in compute r scie nce to de scribe a proble m-solving me thod suita ble for imple me nta tion a s a compute r progra m Algorithms a re
the stuff of compute r scie nce : The y a re ce ntra l obje cts of study in ma ny, if not most, a re a s of the fie ld
Most a lgorithms of inte re st involve me thods of orga nizing the da ta involve d in the computa tion Obje cts cre a te d in this w a y a re ca lle d data structures, a nd the y a lso a re ce ntra l obje cts of study in compute r scie nce Thus, a lgorithms a nd da ta structure s go ha nd in ha nd In this book w e
ta ke the vie w tha t da ta structure s e xist a s the byproducts or e nd products of a lgorithms a nd tha t w e must the re fore study the m in orde r tounde rsta nd the a lgorithms Simple a lgorithms ca n give rise to complica te d da ta structure s a nd, conve rse ly, complica te d a lgorithms ca n use simple
da ta structure s We sha ll study the prope rtie s of ma ny da ta structure s in this book; inde e d, the book might w e ll ha ve be e n ca lle d Algorithms and Data Structures in Java.
W he n w e use a compute r to he lp us solve a proble m, w e typica lly a re fa ce d w ith a numbe r of possible diffe re nt a pproa che s For sma ll proble ms,
it ha rdly ma tte rs w hich a pproa ch w e use , a s long a s w e ha ve one tha t solve s the proble m corre ctly For huge proble ms (or a pplica tions w he re
w e ne e d to solve huge numbe rs of sma ll proble ms), how e ve r, w e quickly be come motiva te d to de vise me thods tha t use time or spa ce a s
e fficie ntly a s possible
The prima ry re a son to le a rn a bout a lgorithm de sign is tha t this discipline give s us the pote ntia l to re a p huge sa vings, e ve n to the point of ma king
it possible to do ta sks tha t w ould othe rw ise be impossible In a n a pplica tion w he re w e a re proce ssing millions of obje cts, it is not unusua l to be
a ble to ma ke a progra m millions of time s fa ste r by using a w e ll-de signe d a lgorithm We sha ll se e such a n e xa mple in Se ction 1.2 a nd onnume rous othe r occa sions throughout the book By contra st, inve sting a dditiona l mone y or time to buy a nd insta ll a ne w compute r holds thepote ntia l for spe e ding up a progra m by pe rha ps a fa ctor of only 10 or 100 Ca re ful a lgorithm de sign is a n e xtre me ly e ffe ctive pa rt of the proce ss
of solving a huge proble m, w ha te ve r the a pplica tions a re a
W he n a huge or comple x compute r progra m is to be de ve lope d, a gre a t de a l of e ffort must go into unde rsta nding a nd de fining the proble m to besolve d, ma na ging its comple xity, a nd de composing it into sma lle r subta sks tha t ca n be imple me nte d e a sily Ofte n, ma ny of the a lgorithms
re quire d a fte r the de composition a re trivia l to imple me nt In most ca se s, how e ve r, the re a re a fe w a lgorithms w hose choice is critica l be ca usemost of the syste m re source s w ill be spe nt running those a lgorithms Those a re the type s of a lgorithms on w hich w e conce ntra te in this book
W e sha ll study a va rie ty of funda me nta l a lgorithms tha t a re use ful for solving huge proble ms in a broa d va rie ty of a pplica tions a re a s
The sha ring of progra ms in compute r syste ms is be coming more w ide spre a d, so a lthough w e might e xpe ct to be using a la rge fra ction of the
a lgorithms in this book, w e a lso might e xpe ct to ha ve to implement only a sma ll fra ction of the m For e xa mple , the Ja va libra rie s conta in
imple me nta tions of a host of funda me nta l a lgorithms How e ve r, imple me nting simple ve rsions of ba sic a lgorithms he lps us to unde rsta nd the m
be tte r a nd thus to more e ffe ctive ly use a nd tune a dva nce d ve rsions from a libra ry More importa nt, the opportunity to re imple me nt ba sic
a lgorithms a rise s fre que ntly The prima ry re a son to do so is tha t w e a re fa ce d, a ll too ofte n, w ith comple te ly ne w computing e nvironme nts(ha rdw a re a nd softw a re ) w ith ne w fe a ture s tha t old imple me nta tions ma y not use to be st a dva nta ge In othe r w ords, w e ofte n imple me nt ba sic
a lgorithms ta ilore d to our proble m, ra the r tha n de pe nding on a syste m routine , to ma ke our solutions more porta ble a nd longe r la sting Anothe rcommon re a son to re imple me nt ba sic a lgorithms is tha t, de spite the a dva nce s e mbodie d in Ja va , the me cha nisms tha t w e use for sha ringsoftw a re a re not a lw a ys sufficie ntly pow e rful to a llow us to conve nie ntly ta ilor libra ry progra ms to pe rform e ffe ctive ly on spe cific ta sks
Compute r progra ms a re ofte n ove roptimize d It ma y not be w orthw hile to ta ke pa ins to e nsure tha t a n imple me nta tion of a pa rticula r a lgorithm
is the most e fficie nt possible unle ss the a lgorithm is to be use d for a n e normous ta sk or is to be use d ma ny time s Othe rw ise , a ca re ful, re la tive lysimple imple me nta tion w ill suffice : We ca n ha ve some confide nce tha t it w ill w ork, a nd it is like ly to run pe rha ps 5 or 10 time s slow e r a t w orsttha n the be st possible ve rsion, w hich me a ns tha t it ma y run for a n e xtra fe w se conds By contra st, the prope r choice of a lgorithm in the firstpla ce ca n ma ke a diffe re nce of a fa ctor of 100 or 1000 or more , w hich might tra nsla te to minute s, hours, or e ve n more in running time In thisbook, w e conce ntra te on the simple st re a sona ble imple me nta tions of the be st a lgorithms We do pa y ca re ful a tte ntion to ca re fully coding thecritica l pa rts of the a lgorithms, a nd ta ke pa ins to note w he re low -le ve l optimiza tion e ffort could be most be ne ficia l
The choice of the be st a lgorithm for a pa rticula r ta sk ca n be a complica te d proce ss, pe rha ps involving sophistica te d ma the ma tica l a na lysis The
bra nch of compute r scie nce tha t comprise s the study of such que stions is ca lle d analysis of algorithms Ma ny of the a lgorithms tha t w e study ha ve
be e n show n through a na lysis to ha ve e xce lle nt pe rforma nce ; othe rs a re simply know n to w ork w e ll through e xpe rie nce Our prima ry goa l is to
le a rn re a sona ble a lgorithms for importa nt ta sks, ye t w e sha ll a lso pa y ca re ful a tte ntion to compa ra tive pe rforma nce of the me thods We shouldnot use a n a lgorithm w ithout ha ving a n ide a of w ha t re source s it might consume , a nd w e strive to be a w a re of how our a lgorithms might be
e xpe cte d to pe rform
Previous Section
Top
Trang 181.2 A Sample Problem: Connectivity
Suppose tha t w e a re give n a se que nce of pa irs of inte ge rs, w he re e a ch inte ge r re pre se nts a n obje ct of some type a nd w e a re to inte rpre t the
pa ir p-q a s me a ning "p is conne cte d to q." We a ssume the re la tion "is conne cte d to" to be tra nsitive : If p is conne cte d to q, a nd q is conne cte d
to r, the n p is conne cte d to r Our goa l is to w rite a progra m to filte r out e xtra ne ous pa irs from the se t: W he n the progra m inputs a pa ir p-q, it
should output the pa ir only if the pa irs it ha s se e n to tha t point do not imply tha t p is conne cte d to q If the pre vious pa irs do imply tha t p isconne cte d to q, the n the progra m should ignore p-q a nd should proce e d to input the ne xt pa ir Figure 1.1 give s a n e xa mple of this proce ss
Figure 1.1 Connectivity example
Given a sequence of pairs of integers representing connections between objects (left), the task of a connectivity algorithm is to output those pairs that provide new connections (center) For example, the pair 2-9 is not part of the output because the connection 2-3-4-9 is implied by
previous connections (this evidence is shown at right).
gra phics/01fig01.gif
Our proble m is to de vise a progra m tha t ca n re me mbe r sufficie nt informa tion a bout the pa irs it ha s se e n to be a ble to de cide w he the r or not a
ne w pa ir of obje cts is conne cte d Informa lly, w e re fe r to the ta sk of de signing such a me thod a s the connectivity problem This proble m a rise s in a
numbe r of importa nt a pplica tions W e brie fly conside r thre e e xa mple s he re to indica te the funda me nta l na ture of the proble m
For e xa mple , the inte ge rs might re pre se nt compute rs in a la rge ne tw ork, a nd the pa irs might re pre se nt conne ctions in the ne tw ork The n, ourprogra m might be use d to de te rmine w he the r w e ne e d to e sta blish a ne w dire ct conne ction for p a nd q to be a ble to communica te or w he the r
w e could use e xisting conne ctions to se t up a communica tions pa th In this kind of a pplica tion, w e might ne e d to proce ss millions of points a ndbillions of conne ctions, or more As w e sha ll se e , it w ould be impossible to solve the proble m for such a n a pplica tion w ithout a n e fficie nt
a lgorithm
Simila rly, the inte ge rs might re pre se nt conta ct points in a n e le ctrica l ne tw ork, a nd the pa irs might re pre se nt w ire s conne cting the points In this
ca se , w e could use our progra m to find a w a y to conne ct a ll the points w ithout a ny e xtra ne ous conne ctions, if tha t is possible The re is nogua ra nte e tha t the e dge s in the list w ill suffice to conne ct a ll the points—inde e d, w e sha ll soon se e tha t de te rmining w he the r or not the y w illcould be a prime a pplica tion of our progra m
Figure 1.2 illustra te s the se tw o type s of a pplica tions in a la rge r e xa mple Exa mina tion of this figure give s us a n a ppre cia tion for the difficulty of
the conne ctivity proble m: How ca n w e a rra nge to te ll quickly w he the r any give n tw o points in such a ne tw ork a re conne cte d?
Figure 1.2 A large connectivity example
The objects in a connectivity problem might represent connection points, and the pairs might be connections between them, as indicated in this idealized example that might represent wires connecting buildings in a city or components on a computer chip This graphical
representation makes it possible for a human to spot nodes that are not connected, but the algorithm has to work with only the pairs of
integers that it is given Are the two nodes marked with the large black dots connected?
gra phics/01fig02.gif
Still a nothe r e xa mple a rise s in ce rta in progra mming e nvironme nts w he re it is possible to de cla re tw o va ria ble na me s a s e quiva le nt The proble m
is to be a ble to de te rmine w he the r tw o give n na me s a re e quiva le nt, a fte r a se que nce of such de cla ra tions This a pplica tion is a n e a rly one tha tmotiva te d the de ve lopme nt of se ve ra l of the a lgorithms tha t w e a re a bout to conside r It dire ctly re la te s our proble m to a simple a bstra ction tha tprovide s us w ith a w a y to ma ke our a lgorithms use ful for a w ide va rie ty of a pplica tions, a s w e sha ll se e
Applica tions such a s the va ria ble -na me –e quiva le nce proble m de scribe d in the pre vious pa ra gra ph re quire tha t w e a ssocia te a n inte ge r w ith e a ch
Trang 19a ssume in this cha pte r, w ithout loss of ge ne ra lity, tha t w e ha ve N obje cts w ith inte ge r na me s, from 0 to N - 1.
We a re a sking for a progra m tha t doe s a spe cific a nd w e ll-de fine d ta sk The re a re ma ny othe r re la te d proble ms tha t w e might w a nt to ha ve
solve d a s w e ll One of the first ta sks tha t w e fa ce in de ve loping a n a lgorithm is to be sure tha t w e ha ve spe cifie d the problem in a re a sona ble
ma nne r The more w e re quire of a n a lgorithm, the more time a nd spa ce w e ma y e xpe ct it to ne e d to finish the ta sk It is impossible to qua ntifythis re la tionship a priori, a nd w e ofte n modify a proble m spe cifica tion on finding tha t it is difficult or e xpe nsive to solve or, in ha ppy circumsta nce s,
on finding tha t a n a lgorithm ca n provide informa tion more use ful tha n w a s ca lle d for in the origina l spe cifica tion
For e xa mple , our conne ctivity-proble m spe cifica tion re quire s only tha t our progra m some how know w he the r or not a ny give n pa ir p-q isconne cte d, a nd not tha t it be a ble to de monstra te a ny or a ll w a ys to conne ct tha t pa ir Adding a re quire me nt for such a spe cifica tion ma ke s theproble m more difficult a nd w ould le a d us to a diffe re nt fa mily of a lgorithms, w hich w e conside r brie fly in Cha pte r 5 a nd in de ta il in Pa rt 5
The spe cifica tions me ntione d in the pre vious pa ra gra ph a sk us for more informa tion tha n our origina l one did; w e could a lso a sk for less informa tion For e xa mple , w e might simply w a nt to be a ble to a nsw e r the que stion: "Are the M conne ctions sufficie nt to conne ct toge the r a ll N
obje cts?" This proble m illustra te s tha t to de ve lop e fficie nt a lgorithms w e ofte n ne e d to do high-le ve l re a soning a bout the a bstra ct obje cts tha t
w e a re proce ssing In this ca se , a funda me nta l re sult from gra ph the ory implie s tha t a ll N obje cts a re conne cte d if a nd only if the numbe r of pa irs output by the conne ctivity a lgorithm is pre cise ly N - 1 (se e Se ction 5.4) In othe r w ords, a conne ctivity a lgorithm w ill ne ve r output more tha n N - 1
pa irs be ca use , once it ha s output N - 1 pa irs, a ny pa ir tha t it e ncounte rs from tha t point on w ill be conne cte d Accordingly, w e ca n ge t a progra m
tha t a nsw e rs the ye s–no que stion just pose d by cha nging a progra m tha t solve s the conne ctivity proble m to one tha t incre me nts a counte r,
ra the r tha n w riting out e a ch pa ir tha t w a s not pre viously conne cte d, a nsw e ring "ye s" w he n the counte r re a che s N - 1 a nd "no" if it ne ve r doe s.
This que stion is but one e xa mple of a host of que stions tha t w e might w ish to a nsw e r re ga rding conne ctivity The se t of pa irs in the input is
ca lle d a graph, a nd the se t of pa irs output is ca lle d a spanning tree for tha t gra ph, w hich conne cts a ll the obje cts We conside r prope rtie s of
gra phs, spa nning tre e s, a nd a ll ma nne r of re la te d a lgorithms in Pa rt 5
It is w orthw hile to try to ide ntify the funda me nta l ope ra tions tha t w e w ill be pe rforming, a nd so to ma ke a ny a lgorithm tha t w e de ve lop for theconne ctivity ta sk use ful for a va rie ty of simila r ta sks Spe cifica lly, e a ch time tha t a n a lgorithm ge ts a ne w pa ir, it ha s first to de te rmine w he the r it
re pre se nts a ne w conne ction, the n to incorpora te the informa tion tha t the conne ction ha s be e n se e n into its unde rsta nding a bout the
conne ctivity of the obje cts such tha t it ca n che ck conne ctions to be se e n in the future We e nca psula te the se tw o ta sks a s abstract operations by
conside ring the inte ge r input va lue s to re pre se nt e le me nts in a bstra ct se ts a nd the n de signing a lgorithms a nd da ta structure s tha t ca n
Find the se t conta ining a give n ite m.
Re pla ce the se ts conta ining tw o give n ite ms by the ir union.
Orga nizing our a lgorithms in te rms of the se a bstra ct ope ra tions doe s not se e m to fore close a ny options in solving the conne ctivity proble m, a ndthe ope ra tions ma y be use ful for solving othe r proble ms De ve loping e ve r more pow e rful la ye rs of a bstra ction is a n e sse ntia l proce ss in compute rscie nce in ge ne ra l a nd in a lgorithm de sign in pa rticula r, a nd w e sha ll turn to it on nume rous occa sions throughout this book In this cha pte r, w euse a bstra ct thinking in a n informa l w a y to guide us in de signing progra ms to solve the conne ctivity proble m; in Cha pte r 4, w e sha ll se e how to
e nca psula te a bstra ctions in Ja va code
The conne ctivity proble m is e a sy to solve w ith the find a nd union a bstra ct ope ra tions We re a d a ne w pa ir from the input a nd pe rform a find
ope ra tion for e a ch me mbe r of the pa ir: If the me mbe rs of the pa ir a re in the sa me se t, w e move on to the ne xt pa ir; if the y a re not, w e do a
union ope ra tion a nd w rite out the pa ir The se ts re pre se nt connected components—subse ts of the obje cts w ith the prope rty tha t a ny tw o obje cts
in a give n compone nt a re conne cte d This a pproa ch re duce s the de ve lopme nt of a n a lgorithmic solution for conne ctivity to the ta sks of de fining a
da ta structure re pre se nting the se ts a nd de ve loping union a nd find a lgorithms tha t e fficie ntly use tha t da ta structure
The re a re ma ny w a ys to re pre se nt a nd proce ss a bstra ct se ts, some of w hich w e conside r in Cha pte r 4 In this cha pte r, our focus is on finding a
re pre se nta tion tha t ca n support e fficie ntly the union a nd find ope ra tions tha t w e se e in solving the conne ctivity proble m.
Exercises
1.1 Give the output tha t a conne ctivity a lgorithm should produce w he n give n the input 0-2, 1-4, 2-5, 3-6, 0-4, 6-0, a nd 1-3
1.2 List a ll the diffe re nt w a ys to conne ct tw o diffe re nt obje cts for the e xa mple in Figure 1.1.
1.3 De scribe a simple me thod for counting the numbe r of se ts re ma ining a fte r using the union a nd find ope ra tions to solve the
conne ctivity proble m a s de scribe d in the te xt
Previous Section
Top
Trang 201.3 Union–Find Algorithms
The first ste p in the proce ss of de ve loping a n e fficie nt a lgorithm to solve a give n proble m is to implement a simple algorithm that solves the problem If w e ne e d to solve a fe w pa rticula r proble m insta nce s tha t turn out to be e a sy, the n the simple imple me nta tion ma y finish the job for
us If a more sophistica te d a lgorithm is ca lle d for, the n the simple imple me nta tion provide s us w ith a corre ctne ss che ck for sma ll ca se s a nd a
ba se line for e va lua ting pe rforma nce cha ra cte ristics We a lw a ys ca re a bout e fficie ncy, but our prima ry conce rn in de ve loping the first progra m tha t
w e w rite to solve a proble m is to ma ke sure tha t the progra m is a correct solution to the proble m.
The first ide a tha t might come to mind is some how to sa ve a ll the input pa irs, the n to w rite a function to pa ss through the m to try to discove r
w he the r the ne xt pa ir of obje cts is conne cte d We sha ll use a diffe re nt a pproa ch First, the numbe r of pa irs might be sufficie ntly la rge to pre cludeour sa ving the m a ll in me mory in pra ctica l a pplica tions Se cond, a nd more to the point, no simple me thod imme dia te ly sugge sts itse lf for
de te rmining w he the r tw o obje cts a re conne cte d from the se t of a ll the conne ctions, e ve n if w e could sa ve the m a ll! We conside r a ba sic me thodtha t ta ke s this a pproa ch in Cha pte r 5, but the me thods tha t w e sha ll conside r in this cha pte r a re simple r, be ca use the y solve a le ss difficult
proble m, a nd more e fficie nt, be ca use the y do not re quire sa ving a ll the pa irs The y a ll use a n array of inte ge rs—one corre sponding to e a ch obje ct
—to hold the re quisite informa tion to be a ble to imple me nt union a nd find Arra ys a re e le me nta ry da ta structure s tha t w e discuss in de ta il in
Se ction 3.2 He re , w e use the m in the ir simple st form: w e cre a te a n a rra y tha t ca n hold N inte ge rs by w riting int id[] = new int[N]; the n
w e re fe r to the ith inte ge r in the a rra y by w riting id[i], for 0 i < 1000.
Program 1.1 Quick-find solution to connectivity problem
This progra m ta ke s a n inte ge r N from the comma nd line , re a ds a se que nce of pa irs of inte ge rs, inte rpre ts the pa ir p q to
me a n "conne ct obje ct p to obje ct q," a nd prints the pa irs tha t re pre se nt obje cts tha t a re not ye t conne cte d The progra m
ma inta ins the a rra y id such tha t id[p] a nd id[q] a re e qua l if a nd only if p a nd q a re conne cte d
T h e In a nd Out me thods tha t w e use for input a nd output a re de scribe d in the Appe ndix, a nd the sta nda rd Ja va
me cha nism for ta king pa ra me te r va lue s from the comma nd line is de scribe d in Se ction 3.7
public class QuickF
{ public static void main(String[] args)
{ int N = Integer.parseInt(args[0]);
int id[] = new int[N];
for (int i = 0; i < N ; i++) id[i] = i;
for( In.init(); !In.empty(); )
{ int p = In.getInt(), q = In.getInt();
int t = id[p];
if (t == id[q]) continue;
for (int i = 0;i<N;i++)
if (id[i] == t) id[i] = id[q];
Figure 1.3 show s the cha nge s to the a rra y for the union ope ra tions in the e xa mple in Figure 1.1 To imple me nt find, w e just te st the indica te d
a rra y e ntrie s for e qua lity—he nce the na me quick find The union ope ra tion, on the othe r ha nd, involve s sca nning through the w hole a rra y for
e a ch input pa ir
Figure 1.3 Example of quick find (slow union)
This sequence depicts the contents of the id array after each of the pairs at left is processed by the quick-find algorithm (Progra m 1.1).
Shaded entries are those that change for the union operation When we process the pair pq, we change all entries with the value id[p] to
have the value id[q].gra phics/01fig03.gif
Property 1.1
The quick-find algorithm executes at least MN instructions to solve a connectivity problem with N objects that involves M union operations For e a ch of the M union ope ra tions, w e ite ra te the for loop N time s Ea ch ite ra tion re quire s a t le a st one instruction (if only to che ck
w he the r the loop is finishe d)
We ca n e xe cute te ns or hundre ds of millions of instructions pe r se cond on mode rn compute rs, so this cost is not notice a ble if M a nd N a re sma ll,
but w e a lso might find ourse lve s w ith billions of obje cts a nd millions of input pa irs to proce ss in a mode rn a pplica tion The ine sca pa ble conclusion
Trang 21Figure 1.4 show s a gra phica l re pre se nta tion of Figure 1.3 We ma y think of some of the obje cts a s re pre se nting the se t to w hich the y be long,
a nd a ll of the othe r obje cts a s ha ving a link to the re pre se nta tive in the ir se t The re a son for moving to this gra phica l re pre se nta tion of the a rra y
w ill be come cle a r soon Obse rve tha t the conne ctions be tw e e n obje cts (links) in this re pre se nta tion a re not ne ce ssa rily the sa me a s the
conne ctions in the input pa irs—the y a re the informa tion tha t the a lgorithm choose s to re me mbe r to be a ble to know w he the r future pa irs a reconne cte d
Figure 1.4 Tree representation of quick find
This figure depicts graphical representations for the example in Figure 1.3 The connections in these figures do not necessarily represent the connections in the input For example, the structure at the bottom has the connection 1-7, which is not in the input, but which is made
because of the string of connections 7-3-4-9-5-6-1
gra phics/01fig04.gif
The ne xt a lgorithm tha t w e conside r is a comple me nta ry me thod ca lle d the quick-union algorithm It is ba se d on the sa me da ta structure —a n
a rra y inde xe d by obje ct na me s—but it use s a diffe re nt inte rpre ta tion of the va lue s tha t le a ds to more comple x a bstra ct structure s Ea ch obje ct
ha s a link to a nothe r obje ct in the sa me se t, in a structure w ith no cycle s To de te rmine w he the r tw o obje cts a re in the sa me se t, w e follow linksfor e a ch until w e re a ch a n obje ct tha t ha s a link to itse lf The obje cts a re in the sa me se t if a nd only if this proce ss le a ds the m to the sa meobje ct If the y a re not in the sa me se t, w e w ind up a t diffe re nt obje cts (w hich ha ve links to the mse lve s) To form the union, the n, w e just link
one to the othe r to pe rform the union ope ra tion; he nce the na me quick union.
Figure 1.5 show s the gra phica l re pre se nta tion tha t corre sponds to Figure 1.4 for the ope ra tion of the quick-union a lgorithm on the e xa mple ofFigure 1.1, a nd Figure 1.6 show s the corre sponding cha nge s to the id a rra y The gra phica l re pre se nta tion of the da ta structure ma ke s it
re la tive ly e a sy to unde rsta nd the ope ra tion of the a lgorithm—input pa irs tha t a re know n to be conne cte d in the da ta a re a lso conne cte d to one
a nothe r in the da ta structure As me ntione d pre viously, it is importa nt to note a t the outse t tha t the conne ctions in the da ta structure a re not
ne ce ssa rily the sa me a s the conne ctions in the a pplica tion implie d by the input pa irs; ra the r, the y a re constructe d by the a lgorithm to fa cilita te
e fficie nt imple me nta tion of union a nd find.
Figure 1.5 Tree representation of quick union
Trang 22Figure 1.6 Example of quick union (not-too-quick find)
This sequence depicts the contents of the id array after each of the pairs at left are processed by the quick-union algorithm (Progra m 1.2).
Shaded entries are those that change for the union operation (just one per operation) When we process the pair p q, we follow links from p
to get an entry i with id[i] == i; then, we follow links from q to get an entry j with id[j] == j; then, if i and j differ, we set id[i]
= id[j] For the find operation for the pair 5-8 (final line), i takes on the values 5 6901, and j takes on the values 801
find w he the r or not the y a re conne cte d Ea ch tre e ha s pre cise ly one obje ct tha t ha s a link to itse lf, w hich is ca lle d the root of the tre e The se
lf-link is not show n in the dia gra ms W he n w e sta rt a t a ny obje ct in the tre e , move to the obje ct to w hich its lf-link re fe rs, the n move to the obje ct to
w hich tha t obje ct's link re fe rs, a nd so forth, w e a lw a ys e ve ntua lly e nd up a t the root We ca n prove this prope rty to be true by induction: It is
true a fte r the a rra y is initia lize d to ha ve e ve ry obje ct link to itse lf, a nd if it is true be fore a give n union ope ra tion, it is ce rta inly true a fte rw a rd.
The dia gra ms in Figure 1.4 for the quick-find a lgorithm ha ve the sa me prope rtie s a s those de scribe d in the pre vious pa ra gra ph The diffe re nce
be tw e e n the tw o is tha t w e re a ch the root from a ll the node s in the quick-find tre e s a fte r follow ing just one link, w he re a s w e might ne e d tofollow se ve ra l links to ge t to the root in a quick-union tre e
Program 1.2 Quick-union solution to connectivity problem
If w e re pla ce the body of the for loop in Progra m 1.1 by this code , w e ha ve a progra m tha t me e ts the sa me spe cifica tions
a s Progra m 1.1, but doe s le ss computa tion for the union ope ra tion a t the e xpe nse of more computa tion for the find
ope ra tion The for loops a nd subse que nt if sta te me nt in this code spe cify the ne ce ssa ry a nd sufficie nt conditions on the
id a rra y for p a nd q to be conne cte d The a ssignme nt sta te me nt id[i] = j imple me nts the union ope ra tion.
int i, j, p = In.getInt(), q = In.getInt();
for (i = p; i != id[i]; i = id[i]);
for (j = q; j != id[j]; j = id[j]);
Trang 23more de pe nde nt on the na ture of the input By running e mpirica l studie s or doing ma the ma tica l a na lysis (se e Cha pte r 2), w e ca n convinceourse lve s tha t Progra m 1.2 is fa r more e fficie nt tha n Progra m 1.1, a nd tha t it is fe a sible to conside r using Progra m 1.2 for huge pra ctica lproble ms We sha ll discuss one such e mpirica l study a t the e nd of this se ction For the mome nt, w e ca n re ga rd quick union a s a n improve me nt
be ca use it re move s quick find's ma in lia bility (tha t the progra m re quire s a t le a st NM instructions to proce ss M union ope ra tions a mong N obje cts).
This diffe re nce be tw e e n quick union a nd quick find ce rta inly re pre se nts a n improve me nt, but quick union still ha s the lia bility tha t w e ca nnot
guarantee it to be substa ntia lly fa ste r tha n quick find in e ve ry ca se , be ca use the input da ta could conspire to ma ke the find ope ra tion slow
Property 1.2
For M > N, the quick-union algorithm could take more than MN/2 instructions to solve a connectivity problem with M pairs of N objects.
Suppose tha t the input pa irs come in the orde r 1-2, the n 2-3, the n 3-4, a nd so forth Afte r N - 1 such pa irs, w e ha ve N obje cts a ll in the
sa me se t, a nd the tre e tha t is forme d by the quick-union a lgorithm is a stra ight line , w ith N linking to N - 1, w hich links to N - 2, w hich links to N - 3, a nd so forth To e xe cute the find ope ra tion for obje ct N, the progra m ha s to follow N - 1 links Thus, the a ve ra ge numbe r of links follow e d for the first N pa irs is
gra phics/01icon01.gif
Now suppose tha t the re ma inde r of the pa irs a ll conne ct N to some othe r obje ct The find ope ra tion for e a ch of the se pa irs involve s a t
le a st (N - 1) links The gra nd tota l for the M find ope ra tions for this se que nce of input pa irs is ce rta inly gre a te r tha n MN/2
Fortuna te ly, the re is a n e a sy modifica tion to the a lgorithm tha t a llow s us to gua ra nte e tha t ba d ca se s such a s this one do not occur Ra the r tha n
a rbitra rily conne cting the se cond tre e to the first for union, w e ke e p tra ck of the numbe r of node s in e a ch tre e a nd a lw a ys conne ct the sma lle r
tre e to the la rge r This cha nge re quire s slightly more code a nd a nothe r a rra y to hold the node counts, a s show n in Progra m 1.3 , but it le a ds to
substa ntia l improve me nts in e fficie ncy W e re fe r to this a lgorithm a s the weighted quick-union algorithm.
Figure 1.7 show s the fore st of tre e s constructe d by the w e ighte d union–find a lgorithm for the e xa mple input in Figure 1.1 Eve n for this sma ll
e xa mple , the pa ths in the tre e s a re substa ntia lly shorte r tha n for the unw e ighte d ve rsion in Figure 1.5 Figure 1.8 illustra te s w ha t ha ppe ns in
the w orst ca se , w he n the size s of the se ts to be me rge d in the union ope ra tion a re a lw a ys e qua l (a nd a pow e r of 2) The se tre e structure s look
comple x, but the y ha ve the simple prope rty tha t the ma ximum numbe r of links tha t w e ne e d to follow to ge t to the root in a tre e of 2n node s is n.
Furthe rmore , w he n w e me rge tw o tre e s of 2n node s, w e ge t a tre e of 2n+1 node s, a nd w e incre a se the ma ximum dista nce to the root to n + 1.
This obse rva tion ge ne ra lize s to provide a proof tha t the w e ighte d a lgorithm is substa ntia lly more e fficie nt tha n the unw e ighte d a lgorithm
Figure 1.7 Tree representation of weighted quick union
This sequence depicts the result of changing the quick-union algorithm to link the root of the smaller of the two trees to the root of the larger
of the two trees The distance from each node to the root of its tree is small, so the find operation is efficient.
gra phics/01fig07.gif
Figure 1.8 Weighted quick union (worst case)
The worst scenario for the weighted quick-union algorithm is that each union operation links trees of equal size If the number of objects is
n
Trang 24Program 1.3 Weighted version of quick union
This progra m is a modifica tion to the quick-union a lgorithm (se e Progra m 1.2 ) tha t ke e ps a n a dditiona l a rra y sz for the
purpose of ma inta ining, for e a ch obje ct w ith id[i] == i, the numbe r of node s in the a ssocia te d tre e so tha t the union
ope ra tion ca n link the sma lle r of the tw o spe cifie d tre e s to the la rge r, thus pre ve nting the grow th of long pa ths in the
tre e s
public class QuickUW
{ public static void main(String[] args)
{ int N = Integer.parseInt(args[0]);
int id[] = new int[N], sz[] = new int[N];
for (int i = 0;i<N;i++)
{ id[i] = i; sz[i] = 1; }
for(In.init(); !In.empty(); )
{ int i, j, p = In.getInt(), q = In.getInt();
for (i = p; i != id[i]; i = id[i]);
for (j = q; j != id[j]; j = id[j]);
The weighted quick-union algorithm follows at most 2 lg N links to determine whether two of N objects are connected.
We ca n prove tha t the union ope ra tion pre se rve s the prope rty tha t the numbe r of links follow e d from a ny node to the root in a se t of k obje cts is no gre a te r tha n lg k (w e do not count the se lf-link a t the root) W he n w e combine a se t of i node s w ith a se t of j node s w ith i
j, w e incre a se the numbe r of links tha t must be follow e d in the sma lle r se t by 1, but the y a re now in a se t of size i + j, so the prope rty is pre se rve d be ca use 1 + lg i =lg(i + i) lg(i + j)
The pra ctica l implica tion of Prope rty 1.3 is tha t the w e ighte d quick-union a lgorithm use s at most a consta nt time s M lg N instruc-tions to proce ss
M e dge s on N obje cts (se e Exe rcise 1.9) This re sult is in sta rk contra st to our finding tha t quick find a lw a ys (a nd quick union some time s) use s at least MN/2 instructions The conclusion is tha t, w ith w e ighte d quick union, w e ca n gua ra nte e tha t w e ca n solve huge pra ctica l proble ms in a
re a sona ble a mount of time (se e Exe rcise 1.11) For the price of a fe w e xtra line s of code , w e ge t a progra m tha t is lite ra lly millions of time s fa ste rtha n the simple r a lgorithms for the huge proble ms tha t w e might e ncounte r in pra ctica l a pplica tions
It is e vide nt from the dia gra ms tha t re la tive ly fe w node s a re fa r from the root; inde e d, e mpirica l studie s on huge proble ms te ll us tha t the
w e ighte d quick-union a lgorithm of Progra m 1.3 typica lly ca n solve pra ctica l proble ms in linear time Tha t is, the cost of running the a lgorithm is
w ithin a consta nt fa ctor of the cost of re a ding the input W e could ha rdly e xpe ct to find a more e fficie nt a lgorithm
We imme dia te ly come to the que stion of w he the r or not w e ca n find a n a lgorithm tha t ha s guaranteed line a r pe rforma nce This que stion is a n
e xtre me ly difficult one tha t pla gue d re se a rche rs for ma ny ye a rs (se e Se ction 2.7) The re a re a numbe r of e a sy w a ys to improve the w e ighte dquick-union a lgorithm furthe r Ide a lly, w e w ould like e ve ry node to link dire ctly to the root of its tre e , but w e do not w a nt to pa y the price ofcha nging a la rge numbe r of links, a s w e did in the quick-union a lgorithm We ca n a pproa ch the ide a l simply by ma king a ll the node s tha t w e do
e xa mine link to the root This ste p se e ms dra stic a t first blush, but it is e a sy to imple me nt, a nd the re is nothing sa crosa nct a bout the structure of
the se tre e s: If w e ca n modify the m to ma ke the a lgorithm more e fficie nt, w e should do so We ca n e a sily imple me nt this me thod, ca lle d path compression, by a dding a nothe r pa ss through e a ch pa th during the union ope ra tion, se tting the id e ntry corre sponding to e a ch ve rte x
e ncounte re d a long the w a y to link to the root The ne t re sult is to fla tte n the tre e s a lmost comple te ly, a pproxima ting the ide a l a chie ve d by thequick-find a lgorithm, a s illustra te d in Figure 1.9 The a na lysis tha t e sta blishe s this fa ct is e xtre me ly comple x, but the me thod is simple a nd
Trang 25Figure 1.9 Path compression
We can make paths in the trees even shorter by simply making all the objects that we touch point to the root of the new tree for the union operation, as shown in these two examples The example at the top shows the result corresponding to Figure 1.7 For short paths, path compression has no effect, but when we process the pair 1 6, we make 1, 5, and 6 all point to 3 and get a tree flatter than the one in Figure 1.7 The example at the bottom shows the result corresponding to Figure 1.8 Paths that are longer than one or two links can develop in the
trees, but whenever we traverse them, we flatten them Here, when we process the pair 6 8, we flatten the tree by making 4, 6, and 8 all
point to 0.gra phics/01fig09.gif
Figure 1.11 A large example of the effect of path compression
This sequence depicts the result of processing random pairs from 100 objects with the weighted quick-union algorithm with path
compression All but two of the nodes in the tree are one or two steps from the root.
gra phics/01fig11.gif
The re a re ma ny othe r w a ys to imple me nt pa th compre ssion For e xa mple , Progra m 1.4 is a n imple me nta tion tha t compre sse s the pa ths by
ma king e a ch link skip to the ne xt node in the pa th on the w a y up the tre e , a s de picte d in Figure 1.10 This me thod is slightly e a sie r to imple me nt
tha n full pa th compre ssion (se e Exe rcise 1.16), a nd a chie ve s the sa me ne t re sult We re fe r to this va ria nt a s weighted quick-union with path compression by halving W hich of the se me thods is the more e ffe ctive ? Is the sa vings a chie ve d w orth the e xtra time re quire d to imple me nt pa th
compre ssion? Is the re some othe r te chnique tha t w e should conside r? To a nsw e r the se que stions, w e ne e d to look more ca re fully a t the
a lgorithms a nd imple me nta tions We sha ll re turn to this topic in Cha pte r 2 , in the conte xt of our discussion of ba sic a pproa che s to the a na lysis of
a lgorithms
Figure 1.10 Path compression by halving
We can nearly halve the length of paths on the way up the tree by taking two links at a time and setting the bottom one to point to the same node as the top one, as shown in this example The net result of performing this operation on every path that we traverse is asymptotically
the same as full path compression.
Program 1.4 Path compression by halving
If w e re pla ce the for loops in Progra m 1.3 by this code , w e ha lve the le ngth of a ny pa th tha t w e tra ve rse The ne t re sult
of this cha nge is tha t the tre e s be come a lmost comple te ly fla t a fte r a long se que nce of ope ra tions
for (i = p; i != id[i]; i = id[i])
cost of ga the ring the da ta More ove r, the a lgorithms a re online a lgorithms tha t conside r e a ch e dge once , using spa ce proportiona l to the numbe r
of obje cts, so the re is no limita tion on the numbe r of e dge s tha t the y ca n ha ndle The e mpirica l studie s in Ta ble 1.1 va lida te our conclusion tha t
Trang 261.4 Show the conte nts of the id a rra y a fte r e a ch union ope ra tion w he n you use the quick-find a lgorithm (Progra m 1.1 ) to solve
the conne ctivity proble m for the se que nce 0-2, 1-4, 2-5, 3-6, 0-4, 6-0, a nd 1-3 Also give the numbe r of time s the progra m
a cce sse s the id a rra y for e a ch input pa ir
1.5 Do Exe rcise 1.4, but use the quick-union a lgorithm (Progra m 1.2).
Table 1.1 Empirical study of union-find algorithms
The se re la tive timings for solving ra ndom conne ctivity proble ms using va rious union–find a lgorithms de monstra te the
e ffe ctive ne ss of the w e ighte d ve rsion of the quick-union a lgorithm The a dde d incre me nta l be ne fit due to pa th
compre ssion is le ss importa nt In the se e xpe rime nts, M is the numbe r of ra ndom conne ctions ge ne ra te d until a ll N
obje cts a re conne cte d This proce ss involve s substa ntia lly more find ope ra tions tha n union ope ra tions, so quick union
is substa ntia lly slow e r tha n quick find Ne ithe r quick find nor quick union is fe a sible for huge N The running time for
the w e ighte d me thods is e vide ntly roughly proportiona l to M.
F quick find (Progra m 1.1)
U quick union (Progra m 1.2)
W w e ighte d quick union (Progra m 1.3)
P w e ighte d quick union w ith pa th compre ssion (Exe rcise 1.16)
H w e ighte d quick union w ith ha lving (Progra m 1.4)
1.6 Give the conte nts of the id a rra y a fte r e a ch union ope ra tion for the w e ighte d quick-union a lgorithm running on the e xa mple s
corre sponding to Figure 1.7 a nd Figure 1.8
1.7 Do Exe rcise 1.4, but use the w e ighte d quick-union a lgorithm (Progra m 1.3).
1.8 Do Exe rcise 1.4, but use the w e ighte d quick-union a lgorithm w ith pa th compre ssion by ha lving (Progra m 1.4).
1.9 Prove a n uppe r bound on the numbe r of ma chine instructions re quire d to proce ss M conne ctions on N obje cts using Progra m
1.3 You ma y a ssume , for e xa mple , tha t a ny Ja va a ssignme nt sta te me nt a lw a ys re quire s le ss tha n c instructions, for some fixe d consta nt c.
1.10 Estima te the minimum a mount of time (in da ys) tha t w ould be re quire d for quick find (Progra m 1.1) to solve a proble m w ith 109
obje cts a nd 106 input pa irs, on a compute r ca pa ble of e xe cuting 109 instructions pe r se cond Assume tha t e a ch ite ra tion of the inne r
for loop re quire s a t le a st 10 instructions
1.11 Estima te the ma ximum a mount of time (in se conds) tha t w ould be re quire d for w e ighte d quick union (Progra m 1.3) to solve a
proble m w ith 109 obje cts a nd 106 input pa irs, on a compute r ca pa ble of e xe cuting 109 instructions pe r se cond Assume tha t e a chite ra tion of the oute r for loop re quire s a t most 100 instructions
1.12 Compute the average dista nce from a node to the root in a w orst-ca se tre e of 2 n node s built by the w e ighte d quick-union
a lgorithm
1.13 Dra w a dia gra m like Figure 1.10, sta rting w ith e ight node s inste a d of nine
1.14 Give a se que nce of input pa irs tha t ca use s the w e ighte d quick-union a lgorithm (Progra m 1.3) to produce a pa th of le ngth 4.
1.15 Give a se que nce of input pa irs tha t ca use s the w e ighte d quick-union a lgorithm w ith pa th compre ssion by ha lving (Progra m
1.4) to produce a pa th of le ngth 4
1.16 Show how to modify Progra m 1.3 to imple me nt full pa th compre ssion, w he re w e comple te e a ch union ope ra tion by ma king
e ve ry node tha t w e touch link to the root of the ne w tre e
1.17 Answ e r Exe rcise 1.4, but use the w e ighte d quick-union a lgorithm w ith full pa th compre ssion (Exe rcise 1.16).
1.18 Give a se que nce of input pa irs tha t ca use s the w e ighte d quick-union a lgorithm w ith full pa th compre ssion (Exe rcise
1.16) to produce a pa th of le ngth 4
1.19 Give a n e xa mple show ing tha t modifying quick union (Progra m 1.2) to imple me nt full pa th compre ssion (se e Exe rcise 1.16) is
not sufficie nt to e nsure tha t the tre e s ha ve no long pa ths
1.20 Modify Progra m 1.3 to use the height of the tre e s (longe st pa th from a ny node to the root), inste a d of the w e ight, to de cide
w he the r to se t id[i] = j or id[j] = i Run e mpirica l studie s to compa re this va ria nt w ith Progra m 1.3
1.21 Show tha t Prope rty 1.3 holds for the a lgorithm de scribe d in Exe rcise 1.20.
1.22 Modify Progra m 1.4 to ge ne ra te ra ndom pa irs of inte ge rs be tw e e n 0 a nd N - 1 inste a d of re a ding the m from sta nda rd
input, a nd to loop until N - 1 union ope ra tions ha ve be e n pe rforme d Run your progra m for N = 103, 104, 105, a nd 106, a nd print out
Trang 271.24 Give a n a pproxima te formula for the numbe r of ra ndom e dge s tha t a re re quire d to conne ct N obje cts, a s a function of N.
Previous Section
Top
Trang 281.4 Perspective
Ea ch of the a lgorithms tha t w e conside re d in Se ction 1.3 se e ms to be a n improve me nt ove r the pre vious in some intuitive se nse , but the proce ss
is pe rha ps a rtificia lly smooth be ca use w e ha ve the be ne fit of hindsight in looking ove r the de ve lopme nt of the a lgorithms a s the y w e re studie d
by re se a rche rs ove r the ye a rs (see reference section) The imple me nta tions a re simple a nd the proble m is w e ll spe cifie d, so w e ca n e va lua te the
va rious a lgorithms dire ctly by running e mpirica l studie s Furthe rmore , w e ca n va lida te the se studie s a nd qua ntify the compa ra tive pe rforma nce ofthe se a lgorithms (se e Cha pte r 2) Not a ll the proble m doma ins in this book a re a s w e ll de ve lope d a s this one , a nd w e ce rta inly ca n run intocomple x a lgorithms tha t a re difficult to compa re a nd ma the ma tica l proble ms tha t a re difficult to solve We strive to ma ke obje ctive scie ntificjudge me nts a bout the a lgorithms tha t w e use , w hile ga ining e xpe rie nce le a rning the prope rtie s of imple me nta tions running on a ctua l da ta from
a pplica tions or ra ndom te st da ta
The proce ss is prototypica l of the w a y tha t w e conside r va rious a lgorithms for funda me nta l proble ms throughout the book W he n possible , w efollow the sa me ba sic ste ps tha t w e took for union–find a lgorithms in Se ction 1.2, some of w hich a re highlighte d in this list:
De cide on a comple te a nd spe cific proble m sta te me nt, including ide ntifying funda me nta l a bstra ct ope ra tions tha t a re intrinsic to theproble m
Ca re fully de ve lop a succinct imple me nta tion for a stra ightforw a rd a lgorithm
De ve lop improve d imple me nta tions through a proce ss of ste pw ise re fine me nt, va lida ting the e ffica cy of ide a s for improve me nt through
e mpirica l a na lysis, ma the ma tica l a na lysis, or both
Find high-le ve l a bstra ct re pre se nta tions of da ta structure s or a lgorithms in ope ra tion tha t e na ble e ffe ctive high-le ve l de sign of improve d
ve rsions
Strive for w orst-ca se pe rforma nce gua ra nte e s w he n possible , but a cce pt good pe rforma nce on a ctua l da ta w he n a va ila ble
The pote ntia l for spe cta cula r pe rforma nce improve me nts for pra ctica l proble ms such a s those tha t w e sa w in Se ction 1.2 ma ke s a lgorithm de sign
a compe lling fie ld of study; fe w othe r de sign a ctivitie s hold the pote ntia l to re a p sa vings fa ctors of millions or billions, or more
More importa nt, a s the sca le of our computa tiona l pow e r a nd our a pplica tions incre a se s, the ga p be tw e e n a fa st a lgorithm a nd a slow onegrow s A ne w compute r might be 10 time s fa ste r a nd be a ble to proce ss 10 time s a s much da ta a s a n old one , but if w e a re using a qua dra tic
a lgorithm such a s quick find, the ne w compute r w ill ta ke 10 time s a s long on the ne w job a s the old one took to finish the old job! This sta te me nt
se e ms counte rintuitive a t first, but it is e a sily ve rifie d by the simple ide ntity (10N)2/10 = 10N2, a s w e sha ll se e in Cha pte r 2 As computa tiona lpow e r incre a se s to a llow us to ta ke on la rge r a nd la rge r proble ms, the importa nce of ha ving e fficie nt a lgorithms incre a se s a s w e ll
De ve loping a n e fficie nt a lgorithm is a n inte lle ctua lly sa tisfying a ctivity tha t ca n ha ve dire ct pra ctica l pa yoff As the conne ctivity proble m indica te s,
a simply sta te d proble m ca n le a d us to study nume rous a lgorithms tha t a re not only both use ful a nd inte re sting, but a lso intrica te a ndcha lle nging to unde rsta nd We sha ll e ncounte r ma ny inge nious a lgorithms tha t ha ve be e n de ve lope d ove r the ye a rs for a host of pra ctica lproble ms As the scope of a pplica bility of computa tiona l solutions to scie ntific a nd comme rcia l proble ms w ide ns, so a lso grow s the importa nce of
be ing a ble to a pply e fficie nt a lgorithms to solve know n proble ms a nd of be ing a ble to de ve lop e fficie nt solutions to ne w proble ms
Exercises
1.25 Suppose tha t w e use w e ighte d quick union to proce ss 10 time s a s ma ny conne ctions on a ne w compute r tha t is 10 time s a s
fa st a s a n old one How much longe r w ould it ta ke the ne w compute r to finish the ne w job tha n it took the old one to finish the oldjob?
1.26 Answ e r Exe rcise 1.25 for the ca se w he re w e use a n a lgorithm tha t re quire s N3 instructions
Previous Section
Top
Trang 291.5 Summary of Topics
This se ction comprise s brie f de scriptions of the ma jor pa rts of the book, giving spe cific topics cove re d a nd a n indica tion of our ge ne ra l orie nta tiontow a rd the ma te ria l This se t of topics is inte nde d to touch on a s ma ny funda me nta l a lgorithms a s possible Some of the a re a s cove re d a re corecompute r-scie nce a re a s tha t w e study in de pth to le a rn ba sic a lgorithms of w ide a pplica bility Othe r a lgorithms tha t w e discuss a re from
a dva nce d fie lds of study w ithin compute r scie nce a nd re la te d fie lds, such a s nume rica l a na lysis a nd ope ra tions re se a rch—in the se ca se s, ourtre a tme nt se rve s a s a n introduction to the se fie lds through e xa mina tion of ba sic me thods
The first four pa rts of the book, w hich a re conta ine d in this volume , cove r the most w ide ly use d se t of a lgorithms a nd da ta structure s, a first le ve l
of a bstra ction for colle ctions of obje cts w ith ke ys tha t ca n support a broa d va rie ty of importa nt funda me nta l a lgorithms The a lgorithms tha t w econside r a re the products of de ca de s of re se a rch a nd de ve lopme nt a nd continue to pla y a n e sse ntia l role in the e ve r-e xpa nding a pplica tions ofcomputa tion
Funda me nta ls (Pa rt I) in the conte xt of this book a re the ba sic principle s a nd me thodology tha t w e use to imple me nt, a na lyze , a nd compa re
a lgorithms The ma te ria l in Cha pte r 1 motiva te s our study of a lgorithm de sign a nd a na lysis; in Cha pte r 2, w e conside r ba sic me thods of obta iningqua ntita tive informa tion a bout the pe rforma nce of a lgorithms
Da ta Structure s (Pa rt II) go ha nd-in-ha nd w ith a lgorithms: We sha ll de ve lop a thorough unde rsta nding of da ta re pre se nta tion me thods for usethroughout the re st of the book We be gin w ith a n introduction to ba sic concre te da ta structure s in Cha pte r 3 , including a rra ys, linke d lists, a ndstrings In Cha pte r 4, w e conside r funda me nta l a bstra ct da ta type s (ADTs) such a s sta cks a nd que ue s, including imple me nta tions using
e le me nta ry da ta structure s The n in Cha pte r 5 w e conside r re cursive progra ms a nd da ta structure s, in pa rticula r tre e s a nd a lgorithms for
ma nipula ting the m
Sorting a lgorithms (Pa rt III) for re a rra nging file s into orde r a re of funda me nta l importa nce We conside r a va rie ty of a lgorithms in conside ra ble
de pth, including she llsort, quicksort, me rge sort, he a psort, a nd ra dix sorts We sha ll e ncounte r a lgorithms for se ve ra l re la te d proble ms, includingpriority que ue s, se le ction, a nd me rging Ma ny of the se a lgorithms w ill find a pplica tion a s the ba sis for othe r a lgorithms la te r in the book
Se a rching a lgorithms (Pa rt IV) for finding spe cific ite ms a mong la rge colle ctions of ite ms a re a lso of funda me nta l importa nce We discuss ba sic
a nd a dva nce d me thods for se a rching using tre e s a nd digita l ke y tra nsforma tions, including bina ry se a rch tre e s, ba la nce d tre e s, ha shing, digita l
se a rch tre e s a nd trie s, a nd me thods a ppropria te for huge file s We note re la tionships a mong the se me thods, compa ra tive pe rforma ncesta tistics, a nd corre sponde nce s to sorting me thods
Pa rts 5 through 8, w hich a re conta ine d in tw o se pa ra te volume s (one for Pa rt 5, a nothe r for Pa rts 6 through 8), cove r a dva nce d a pplica tions ofthe a lgorithms de scribe d he re for a dive rse se t of a pplica tions—a se cond le ve l of a bstra ctions spe cific to a numbe r of importa nt a pplica tions
a re a s We a lso de lve more de e ply into te chnique s of a lgorithm de sign a nd a na lysis Ma ny of the proble ms tha t w e touch on a re the subje ct ofongoing re se a rch
Graph Algorithms (Pa rt 5) a re use ful for a va rie ty of difficult a nd importa nt proble ms A ge ne ra l stra te gy for se a rching in gra phs is de ve lope d
a nd a pplie d to funda me nta l conne ctivity proble ms, including shorte st pa th, minimum spa nning tre e , ne tw ork flow , a nd ma tching A unifie dtre a tme nt of the se a lgorithms show s tha t the y a re a ll ba se d on the sa me proce dure (w hich de pe nds on the ba sic priority que ue ADT) We a lsoshow the broa d a pplica bility of gra ph-proce ssing a lgorithms by conside ring ge ne ra l proble m-solving mode ls such a s the mincost flow proble m
a nd the conce pt of re ducing one proble m to a nothe r
String Processing a lgorithms (Pa rt 6) include a ra nge of me thods for proce ssing (long) se que nce s of cha ra cte rs String se a rching le a ds to
pa tte rn ma tching, w hich le a ds to pa rsing File -compre ssion te chnique s a re a lso conside re d Aga in, a n introduction to a dva nce d topics is give nthrough tre a tme nt of some e le me nta ry proble ms tha t a re importa nt in the ir ow n right
Geometric Algorithms (Pa rt 7) a re me thods for solving proble ms involving points a nd line s (a nd othe r simple ge ome tric obje cts) tha t ha ve found
a multitude of a pplica tions We conside r a lgorithms for finding the conve x hull of a se t of points, for finding inte rse ctions a mong ge ome tricobje cts, for solving close st-point proble ms, a nd for multidime nsiona l se a rching Ma ny of the se me thods nice ly comple me nt the more e le me nta rysorting a nd se a rching me thods
Advanced Topics (Pa rt 8) a re discusse d for the purpose of re la ting the ma te ria l in the book to se ve ra l othe r a dva nce d fie lds of study We be gin
w ith ma jor a pproa che s to the de sign a nd a na lysis of a lgorithms, including divide -a nd-conque r, dyna mic progra mming, ra ndomiza tion, a nd
a mortiza tion We surve y line a r progra mming, the fa st Fourie r tra nsform, NP-comple te ne ss, a nd othe r a dva nce d topics from a n introductoryvie w point to ga in a ppre cia tion for the inte re sting a dva nce d fie lds of study sugge ste d by the e le me nta ry proble ms confronte d in this book.The study of a lgorithms is inte re sting be ca use it is a ne w fie ld (a lmost a ll the a lgorithms tha t w e study a re le ss tha n 50 ye a rs old, a nd some
w e re just re ce ntly discove re d) w ith a rich tra dition (a fe w a lgorithms ha ve be e n know n for thousa nds of ye a rs) Ne w discove rie s a re consta ntly
be ing ma de , but fe w a lgorithms a re comple te ly unde rstood In this book w e sha ll conside r intrica te , complica te d, a nd difficult a lgorithms a s w e ll
a s e le ga nt, simple , a nd e a sy a lgorithms Our cha lle nge is to unde rsta nd the forme r a nd to a ppre cia te the la tte r in the conte xt of ma ny diffe re ntpote ntia l a pplica tions In doing so, w e sha ll e xplore a va rie ty of use ful tools a nd de ve lop a style of a lgorithmic thinking tha t w ill se rve us w e ll incomputa tiona l cha lle nge s to come
Previous Section
Top
Trang 30Chapter 2 Principles of Algorithm Analysis
Ana lysis is the ke y to be ing a ble to unde rsta nd a lgorithms sufficie ntly w e ll tha t w e ca n a pply the m e ffe ctive ly to pra ctica l proble ms Although w e
ca nnot do e xte nsive e xpe rime nta tion a nd de e p ma the ma tica l a na lysis on e a ch a nd e ve ry progra m tha t w e run, w e ca n w ork w ithin a ba sicfra me w ork involving both e mpirica l te sting a nd a pproxima te a na lysis tha t ca n he lp us to know the importa nt fa cts a bout the pe rforma ncecha ra cte ristics of our a lgorithms so tha t w e ma y compa re those a lgorithms a nd ca n a pply the m to pra ctica l proble ms
The ve ry ide a of de scribing the pe rforma nce of a comple x a lgorithm a ccura te ly w ith a ma the ma tica l a na lysis se e ms a da unting prospe ct a t first,
a nd w e do ofte n ca ll on the re se a rch lite ra ture for re sults ba se d on de ta ile d ma the ma tica l study Although it is not our purpose in this book tocove r me thods of a na lysis or e ve n to summa rize the se re sults, it is importa nt for us to be a w a re a t the outse t tha t w e a re on firm scie ntificground w he n w e w a nt to compa re diffe re nt me thods More ove r, a gre a t de a l of de ta ile d informa tion is a va ila ble a bout ma ny of our mostimporta nt a lgorithms through ca re ful a pplica tion of re la tive ly fe w e le me nta ry te chnique s We do highlight ba sic a na lytic re sults a nd me thods of
a na lysis throughout the book, pa rticula rly w he n such a ctivitie s he lp us to unde rsta nd the inne r w orkings of funda me nta l a lgorithms Our prima rygoa l in this cha pte r is to provide the conte xt a nd the tools tha t w e ne e d to w ork inte llige ntly w ith the a lgorithms the mse lve s
The e xa mple in Cha pte r 1 provide s a conte xt tha t illustra te s ma ny of the ba sic conce pts of a lgorithm a na lysis, so w e fre que ntly re fe r ba ck to the
pe rforma nce of union–find a lgorithms to ma ke pa rticula r points concre te W e a lso conside r a de ta ile d pa ir of ne w e xa mple s in Se ction 2.6.Ana lysis pla ys a role a t e ve ry point in the proce ss of de signing a nd imple me nting a lgorithms At first, a s w e sa w , w e ca n sa ve fa ctors ofthousa nds or millions in the running time w ith a ppropria te a lgorithm de sign choice s As w e conside r more e fficie nt a lgorithms, w e find it more of a
cha lle nge to choose a mong the m, so w e ne e d to study the ir prope rtie s in more de ta il In pursuit of the best (in some pre cise te chnica l se nse )
a lgorithm, w e find both a lgorithms tha t a re use ful in pra ctice a nd the ore tica l que stions tha t a re cha lle nging to re solve
Comple te cove ra ge of me thods for the a na lysis of a lgorithms is the subje ct of a book in itse lf (see reference section), but it is w orthw hile for us to
conside r the ba sics he re so tha t w e ca n
Illustra te the proce ss
De scribe in one pla ce the ma the ma tica l conve ntions tha t w e use
Provide a ba sis for discussion of highe r-le ve l issue s
De ve lop a n a ppre cia tion for scie ntific unde rpinnings of the conclusions tha t w e dra w w he n compa ring a lgorithms
Most importa nt, a lgorithms a nd the ir a na lyse s a re ofte n inte rtw ine d In this book, w e do not de lve into de e p a nd difficult ma the ma tica l
de riva tions, but w e do use sufficie nt ma the ma tics to be a ble to unde rsta nd w ha t our a lgorithms a re a nd how w e ca n use the m e ffe ctive ly
Previous Section
Top
Trang 312.1 Implementation and Empirical Analysis
We de sign a nd de ve lop a lgorithms by la ye ring a bstra ct ope ra tions tha t he lp us to unde rsta nd the e sse ntia l na ture of the computa tiona lproble ms tha t w e w a nt to solve In the ore tica l studie s, this proce ss, a lthough va lua ble , ca n ta ke us fa r a fie ld from the re a l-w orld proble ms tha t
w e ne e d to conside r Thus, in this book, w e ke e p our fe e t on the ground by e xpre ssing a ll the a lgorithms tha t w e conside r in a n a ctua lprogra mming la ngua ge : Ja va This a pproa ch some time s le a ve s us w ith a blurre d distinction be tw e e n a n a lgorithm a nd its imple me nta tion, buttha t is sma ll price to pa y for the a bility to w ork w ith a nd to le a rn from a concre te imple me nta tion
Inde e d, ca re fully constructe d progra ms in a n a ctua l progra mming la ngua ge provide a n e ffe ctive me a ns of e xpre ssing our a lgorithms In thisbook, w e conside r a la rge numbe r of importa nt a nd e fficie nt a lgorithms tha t w e de scribe in imple me nta tions tha t a re both concise a nd pre cise in
Ja va English-la ngua ge de scriptions or a bstra ct high-le ve l re pre se nta tions of a lgorithms a re a ll too ofte n va gue or incomple te ; a ctua limple me nta tions force us to discove r e conomica l re pre se nta tions to a void be ing inunda te d in de ta il
We e xpre ss our a lgorithms in Ja va , but this book is a bout a lgorithms, ra the r tha n a bout Ja va progra mming Ce rta inly, w e conside r Ja vaimple me nta tions for ma ny importa nt ta sks, a nd w he n the re is a pa rticula rly conve nie nt or e fficie nt w a y to do a ta sk in Ja va , w e w ill ta ke
a dva nta ge of it But the va st ma jority of the imple me nta tion de cisions tha t w e ma ke a re w orth conside ring in a ny mode rn progra mming
e nvironme nt Tra nsla ting the progra ms in Cha pte r 1, a nd most of the othe r progra ms in this book, to a nothe r mode rn progra mming la ngua ge is astra ightforw a rd ta sk On occa sion, w e a lso note w he n some othe r la ngua ge provide s a pa rticula rly e ffe ctive me cha nism suite d to the ta sk a t
ha nd Our goa l is to use Ja va a s a ve hicle for e xpre ssing the a lgorithms tha t w e conside r, ra the r tha n to dw e ll on imple me nta tion issue s spe cific
to Ja va
If a n a lgorithm is to be imple me nte d a s pa rt of a la rge syste m, w e use a bstra ct da ta type s or a simila r me cha nism to ma ke it possible to cha nge
a lgorithms or imple me nta tions a fte r w e de te rmine w ha t pa rt of the syste m de se rve s the most a tte ntion From the sta rt, how e ve r, w e ne e d to
ha ve a n unde rsta nding of e a ch a lgorithm's pe rforma nce cha ra cte ristics, be ca use de sign re quire me nts of the syste m ma y ha ve a ma jor influe nce
on a lgorithm pe rforma nce Such initia l de sign de cisions must be ma de w ith ca re , be ca use it ofte n doe s turn out, in the e nd, tha t the pe rforma nce
of the w hole syste m de pe nds on the pe rforma nce of some ba sic a lgorithm, such a s those discusse d in this book
Imple me nta tions of the a lgorithms in this book ha ve be e n put to e ffe ctive use in a w ide va rie ty of la rge progra ms, ope ra ting syste ms, a nd
a pplica tions syste ms Our inte ntion is to de scribe the a lgorithms a nd to e ncoura ge a focus on the ir dyna mic prope rtie s through e xpe rime nta tion
w ith the imple me nta tions give n For some a pplica tions, the imple me nta tions ma y be quite use ful e xa ctly a s give n; for othe r a pplica tions,how e ve r, more w ork ma y be re quire d For e xa mple , using a more de fe nsive progra mming style tha n the one tha t w e use in this book is justifie d
w he n w e a re building re a l syste ms Error conditions must be che cke d a nd re porte d, a nd progra ms must be imple me nte d such tha t the y ca n becha nge d e a sily, re a d a nd unde rstood quickly by othe r progra mme rs, inte rfa ce w e ll w ith othe r pa rts of the syste m, a nd be a me na ble to be ingmove d to othe r e nvironme nts
Notw ithsta nding a ll the se comme nts, w e ta ke the position w he n a na lyzing e a ch a lgorithm tha t pe rforma nce is of critica l importa nce so tha t w efocus our a tte ntion on the a lgorithm's e sse ntia l pe rforma nce cha ra cte ristics We a ssume tha t w e a re a lw a ys inte re ste d in know ing a bout
a lgorithms w ith substa ntia lly be tte r pe rforma nce , pa rticula rly if the y a re simple r
To use a n a lgorithm e ffe ctive ly, w he the r our goa l is to solve a huge proble m tha t could not othe rw ise be solve d, or w he the r our goa l is toprovide a n e fficie nt imple me nta tion of a critica l pa rt of a syste m, w e ne e d to ha ve a n unde rsta nding of its pe rforma nce cha ra cte ristics
De ve loping such a n unde rsta nding is the goa l of a lgorithmic a na lysis
One of the first ste ps tha t w e ta ke to unde rsta nd the pe rforma nce of a lgorithms is to do empirical analysis Give n tw o a lgorithms to solve the
sa me proble m, the re is no myste ry in the me thod: We run the m both to se e w hich one ta ke s longe r! This conce pt might se e m too obvious to
me ntion, but it is a n a ll-too-common omission in the compa ra tive study of a lgorithms The fa ct tha t one a lgorithm is 10 time s fa ste r tha n a nothe r
is unlike ly to e sca pe the notice of some one w ho w a its 3 se conds for one to finish a nd 30 se conds for the othe r to finish, but it is e a sy to ove rlook
a s a sma ll consta nt ove rhe a d fa ctor in a ma the ma tica l a na lysis W he n w e monitor the pe rforma nce of ca re ful imple me nta tions on typica l input,
w e ge t pe rforma nce re sults tha t not only give us a dire ct indica tor of e fficie ncy but a lso provide us w ith the informa tion tha t w e ne e d to compa re
a lgorithms a nd to va lida te a ny ma the ma tica l a na lyse s tha t ma y a pply (se e , for e xa mple , Ta ble 1.1 ) W he n e mpirica l studie s sta rt to consume asignifica nt a mount of time , ma the ma tica l a na lysis is ca lle d for Wa iting a n hour or a da y for a progra m to finish is ha rdly a productive w a y to findout tha t it is slow , pa rticula rly w he n a stra ightforw a rd a na lysis ca n give us the sa me informa tion
The first cha lle nge tha t w e fa ce in e mpirica l a na lysis is to de ve lop a corre ct a nd comple te imple me nta tion For some comple x a lgorithms, thischa lle nge ma y pre se nt a significa nt obsta cle Accordingly, w e typica lly w a nt to ha ve , through a na lysis or through e xpe rie nce w ith simila rprogra ms, some indica tion of how e fficie nt a progra m might be be fore w e inve st too much e ffort in ge tting it to w ork
The se cond cha lle nge tha t w e fa ce in e mpirica l a na lysis is to de te rmine the na ture of the input da ta a nd othe r fa ctors tha t ha ve dire ct influe nce
on the e xpe rime nts to be pe rforme d Typica lly, w e ha ve thre e ba sic choice s: use actual da ta , random da ta , or perverse da ta Actua l da ta e na ble
us truly to me a sure the cost of the progra m in use ; ra ndom da ta a ssure us tha t our e xpe rime nts te st the a lgorithm, not the da ta ; a nd pe rve rse
da ta a ssure us tha t our progra ms ca n ha ndle a ny input pre se nte d the m For e xa mple , w he n w e te st sorting a lgorithms, w e run the m on da ta
such a s the w ords in Moby Dick, on ra ndomly ge ne ra te d inte ge rs, a nd on file s of numbe rs tha t a re a ll the sa me va lue This proble m of
de te rmining w hich input da ta to use to compa re a lgorithms a lso a rise s w he n w e a na lyze the a lgorithms
It is e a sy to ma ke mista ke s w he n w e compa re imple me nta tions, pa rticula rly if diffe ring ma chine s, compile rs, or syste ms a re involve d, or if hugeprogra ms w ith ill-spe cifie d inputs a re be ing compa re d The principa l da nge r in compa ring progra ms e mpirica lly is tha t one imple me nta tion ma y becode d more ca re fully tha n the othe r The inve ntor of a propose d ne w a lgorithm is like ly to pa y ca re ful a tte ntion to e ve ry a spe ct of itsimple me nta tion a nd not to e xpe nd so much e ffort on the de ta ils of imple me nting a cla ssica l compe ting a lgorithm To be confide nt of the a ccura cy
of a n e mpirica l study compa ring a lgorithms, w e must be sure to give the sa me a tte ntion to e a ch imple me nta tion
One a pproa ch tha t w e ofte n use in this book, a s w e sa w in Cha pte r 1, is to de rive a lgorithms by ma king re la tive ly minor modifica tions to othe r
a lgorithms for the sa me proble m so tha t compa ra tive studie s re a lly a re va lid More ge ne ra lly, w e strive to ide ntify e sse ntia l a bstra ct ope ra tions
a nd sta rt by compa ring a lgorithms on the ba sis of the ir use of such ope ra tions For e xa mple , the compa ra tive e mpirica l re sults tha t w e e xa mine d
in Ta ble 1.1 a re like ly to be robust a cross progra mming la ngua ge s a nd e nvironme nts, a s the y involve progra ms tha t a re simila r a nd tha t ma keuse of the sa me se t of ba sic ope ra tions For a pa rticula r progra mming e nvironme nt, w e ca n e a sily re la te the se numbe rs to a ctua l running time s.Most ofte n, w e simply w a nt to know w hich of tw o progra ms is like ly to be fa ste r, or to w ha t e xte nt a ce rta in cha nge w ill improve the time orspa ce re quire me nts of a ce rta in progra m
Pe rha ps the most common mista ke ma de in se le cting a n a lgorithm is to ignore pe rforma nce cha ra cte ristics Fa ste r a lgorithms a re ofte n morecomplica te d tha n brute -force solutions, a nd imple me ntors a re ofte n w illing to a cce pt a slow e r a lgorithm to a void ha ving to de a l w ith a dde dcomple xity As w e sa w w ith union–find a lgorithms, how e ve r, w e ca n some time s re a p huge sa vings w ith just a fe w line s of code Use rs of a
surprising numbe r of compute r syste ms lose substa ntia l time w a iting for simple qua dra tic a lgorithms to finish solving a proble m, e ve n though N
lo g N or line a r a lgorithms a re a va ila ble tha t a re only slightly more complica te d a nd could the re fore solve the proble m in a fra ction of the time
W he n w e a re de a ling w ith huge proble m size s, w e ha ve no choice but to se e k a be tte r a lgorithm, a s w e sha ll se e
Pe rha ps the se cond most common mista ke ma de in se le cting a n a lgorithm is to pa y too much a tte ntion to pe rforma nce cha ra cte ristics Improvingthe running time of a progra m by a fa ctor of 10 is inconse que ntia l if the progra m ta ke s only a fe w microse conds Eve n if a progra m ta ke s a fe wminute s, it ma y not be w orth the time a nd e ffort re quire d to ma ke it run 10 time s fa ste r, pa rticula rly if w e e xpe ct to use the progra m only a fe w
Trang 32ide a s tha t should improve a progra m but a ctua lly do not do so.
We ca nnot run e mpirica l te sts for a progra m tha t is not ye t w ritte n, but w e ca n a na lyze prope rtie s of the progra m a nd e stima te the pote ntia l
e ffe ctive ne ss of a propose d improve me nt Not a ll puta tive improve me nts a ctua lly re sult in pe rforma nce ga ins, a nd w e ne e d to unde rsta nd the
e xte nt of the sa vings re a lize d a t e a ch ste p More ove r, w e ca n include pa ra me te rs in our imple me nta tions a nd use a na lysis to he lp us se t the
pa ra me te rs Most importa nt, by unde rsta nding the funda me nta l prope rtie s of our progra ms a nd the ba sic na ture of the progra ms' re sourceusa ge , w e ha ve the pote ntia l to e va lua te the ir e ffe ctive ne ss on compute rs not ye t built a nd to compa re the m a ga inst ne w a lgorithms not ye t
de signe d In Se ction 2.2, w e outline our me thodology for de ve loping a ba sic unde rsta nding of a lgorithm pe rforma nce
Exercises
2.1 Tra nsla te the progra ms in Cha pte r 1 to a nothe r progra mming la ngua ge , a nd a nsw e r Exe rcise 1.22 for your imple me nta tions.
2.2 How long doe s it ta ke to count to 1 billion (ignoring ove rflow )? De te rmine the a mount of time it ta ke s the progra m
to comple te in your progra mming e nvironme nt, for N = 10, 100, a nd 1000 If your compile r ha s optimiza tion fe a ture s tha t a re
suppose d to ma ke progra ms more e fficie nt, che ck w he the r or not the y do so for this progra m
Previous Section
Top
Trang 332.2 Analysis of Algorithms
In this se ction, w e outline the fra me w ork w ithin w hich ma the ma tica l a na lysis ca n pla y a role in the proce ss of compa ring the pe rforma nce of
a lgorithms a nd thus la y a founda tion for us to be a ble to conside r ba sic a na lytic re sults a s the y a pply to the funda me nta l a lgorithms tha t w econside r throughout the book We sha ll conside r the ba sic ma the ma tica l tools tha t a re use d in the a na lysis of a lgorithms, both to a llow us tostudy cla ssica l a na lyse s of funda me nta l a lgorithms a nd to ma ke use of re sults from the re se a rch lite ra ture tha t he lp us unde rsta nd the
pe rforma nce cha ra cte ristics of our a lgorithms
The follow ing a re a mong the re a sons tha t w e pe rform ma the ma tica l a na lysis of a lgorithms:
To compa re diffe re nt a lgorithms for the sa me ta sk
To pre dict pe rforma nce in a ne w e nvironme nt
To se t va lue s of a lgorithm pa ra me te rs
We sha ll se e ma ny e xa mple s of e a ch of the se re a sons throughout the book Empirica l a na lysis might suffice for some of the se ta sks, but
ma the ma tica l a na lysis ca n be more informa tive (a nd le ss e xpe nsive !), a s w e sha ll se e
The a na lysis of a lgorithms ca n be cha lle nging inde e d Some of the a lgorithms in this book a re w e ll unde rstood, to the point tha t a ccura te
ma the ma tica l formula s a re know n tha t ca n be use d to pre dict running time in pra ctica l situa tions Pe ople de ve lop such formula s by ca re fullystudying the progra m to find the running time in te rms of funda me nta l ma the ma tica l qua ntitie s a nd the n doing a ma the ma tica l a na lysis of thequa ntitie s involve d On the othe r ha nd, the pe rforma nce prope rtie s of othe r a lgorithms in this book a re not fully unde rstood—pe rha ps the ir
a na lysis le a ds to unsolve d ma the ma tica l que stions, or pe rha ps know n imple me nta tions a re too comple x for a de ta ile d a na lysis to be re a sona ble ,
or (most like ly) pe rha ps the type s of input tha t the y e ncounte r ca nnot be cha ra cte rize d a ccura te ly
Se ve ra l importa nt fa ctors in a pre cise a na lysis a re usua lly outside a give n progra mme r's doma in of influe nce First, Ja va progra ms a re tra nsla te dinto byte code , a nd the byte code is inte rpre te d or tra nsla te d into runtime code on a virtua l ma chine (VM) The compile r, tra nsla tor, a nd VMimple me nta tions a ll ha ve a n e ffe ct on w hich instructions on a n a ctua l ma chine a re e xe cute d, so it ca n be a cha lle nging ta sk to figure out e xa ctlyhow long e ve n one Ja va sta te me nt might ta ke to e xe cute In a n e nvironme nt w he re re source s a re be ing sha re d, e ve n the sa me progra m ca n
ha ve va rying pe rforma nce cha ra cte ristics a t tw o diffe re nt time s Se cond, ma ny progra ms a re e xtre me ly se nsitive to the ir input da ta , a nd
pe rforma nce might fluctua te w ildly de pe nding on the input Third, ma ny progra ms of inte re st a re not w e ll unde rstood, a nd spe cific ma the ma tica l
re sults ma y not be a va ila ble Fina lly, tw o progra ms might not be compa ra ble a t a ll: one ma y run much more e fficie ntly on one pa rticula r kind ofinput a nd the se cond ma y run e fficie ntly unde r othe r circumsta nce s
All the se fa ctors notw ithsta nding, it is ofte n possible to pre dict pre cise ly how long a pa rticula r progra m w ill ta ke , or to know tha t one progra m
w ill do be tte r tha n a nothe r in pa rticula r situa tions More ove r, w e ca n ofte n a cquire such know le dge by using one of a re la tive ly sma ll se t of
ma the ma tica l tools It is the ta sk of the a lgorithm a na lyst to discove r a s much informa tion a s possible a bout the pe rforma nce of a lgorithms; it isthe ta sk of the progra mme r to a pply such informa tion in se le cting a lgorithms for pa rticula r a pplica tions In this a nd the ne xt se ve ra l se ctions, w econce ntra te on the ide a lize d w orld of the a na lyst To ma ke e ffe ctive use of our be st a lgorithms, w e ne e d to be a ble to ste p into this w orld, onocca sion
The first ste p in the a na lysis of a n a lgorithm is to ide ntify the a bstra ct ope ra tions on w hich the a lgorithm is ba se d in orde r to se pa ra te the
a na lysis from the imple me nta tion Thus, for e xa mple , w e se pa ra te the study of how ma ny time s one of our union–find imple -me nta tions e xe cute s
the code fra gme nt i = a[i] from the a na lysis of how ma ny na nose conds might be re quire d to e xe cute tha t pa rticula r code fra gme nt on ourcompute r We ne e d both the se e le me nts to de te rmine the a ctua l running time of the progra m on a pa rticula r compute r The forme r is de te rmine d
by prope rtie s of the a lgorithm; the la tte r by prope rtie s of the compute r This se pa ra tion ofte n a llow s us to compa re a lgorithms in a w a y tha t isinde pe nde nt of pa rticula r imple me nta tions or of pa rticula r compute rs
Although the numbe r of a bstra ct ope ra tions involve d ca n be la rge , in principle , the pe rforma nce of a n a lgorithm typica lly de pe nds on only a fe wqua ntitie s, a nd typica lly the most importa nt qua ntitie s to a na lyze a re e a sy to ide ntify One w a y to ide ntify the m is to use a profiling me cha nism(a me cha nism a va ila ble in ma ny Ja va imple me nta tions tha t give s instruction-fre que ncy counts) to de te rmine the most fre que ntly e xe cute d pa rts
of the progra m for some sa mple runs Or, like the union–find a lgorithms of Se ction 1.3 , our imple me nta tion might be built on a fe w a bstra ctope ra tions In e ithe r ca se , the a na lysis a mounts to de te rmining the fre que ncy of e xe cution of a fe w funda me nta l ope ra tions Our modusope ra ndi w ill be to look for rough e stima te s of the se qua ntitie s, se cure in the know le dge tha t w e ca n unde rta ke a fulle r a na lysis for importa ntprogra ms w he n ne ce ssa ry More ove r, a s w e sha ll se e , w e ca n ofte n use a pproxima te a na lytic re sults in conjunction w ith e mpirica l studie s topre dict pe rforma nce a ccura te ly
We a lso ha ve to study the da ta a nd to mode l the input tha t might be pre se nte d to the a lgorithm Most ofte n, w e conside r one of tw o
a pproa che s to the a na lysis: w e e ithe r a ssume tha t the input is ra ndom a nd study the average-case pe rforma nce of the progra m, or w e look for
pe rve rse input a nd study the worst-case pe rforma nce of the progra m The proce ss of cha ra cte rizing ra ndom inputs is difficult for ma ny a lgorithms,
but for ma ny othe r a lgorithms it is stra ightforw a rd a nd le a ds to a na lytic re sults tha t provide use ful informa tion The a ve ra ge ca se might be a
ma the ma tica l fiction tha t is not re pre se nta tive of the da ta on w hich the progra m is be ing use d, a nd the w orst ca se might be a biza rreconstruction tha t w ould ne ve r occur in pra ctice , but the se a na lyse s give use ful informa tion on pe rforma nce in most ca se s For e xa mple , w e ca n
te st a na lytic re sults a ga inst e mpirica l re sults (se e Se ction 2.1) If the y ma tch, w e ha ve incre a se d confide nce in both; if the y do not ma tch, w e ca n
le a rn a bout the a lgorithm a nd the mode l by studying the discre pa ncie s
In the ne xt thre e se ctions, w e brie fly surve y the ma the ma tica l tools tha t w e sha ll be using throughout the book This ma te ria l is outside ourprima ry na rra tive thrust, a nd re a de rs w ith a strong ba ckground in ma the ma tics or re a de rs w ho a re not pla nning to che ck our ma the ma tica lsta te me nts on the pe rforma nce of a lgorithms in de ta il ma y w ish to skip to Se ction 2.6 a nd to re fe r ba ck to this ma te ria l w he n w a rra nte d la te r inthe book The ma the ma tica l unde rpinnings tha t w e conside r, how e ve r, a re ge ne ra lly not difficult to compre he nd, a nd the y a re too close to coreissue s of a lgorithm de sign to be ignore d by a nyone w ishing to use a compute r e ffe ctive ly
First, in Se ction 2.3, w e conside r the ma the ma tica l functions tha t w e commonly ne e d to de scribe the pe rforma nce cha ra cte ristics of a lgorithms
Ne xt, in Se ction 2.4, w e conside r the O-notation, a nd the notion of is proportional to, w hich a llow us to suppre ss de ta il in our ma the ma tica l
a na lyse s The n, in Se ction 2.5, w e conside r recurrence relations, the ba sic a na lytic tool tha t w e use to ca pture the pe rforma nce cha ra cte ristics of
a n a lgorithm in a ma the ma tica l e qua tion Follow ing this surve y, w e conside r e xa mple s w he re w e use the ba sic tools to a na lyze spe cific
a lgorithms, in Se ction 2.6
Exercises
2.3 De ve lop a n e xpre ssion of the form c0 + c1N + c2N2 + c3N3 tha t a ccura te ly de scribe s the running time of your progra m from
Exe rcise 2.2 Compa re the time s pre dicte d by this e xpre ssion w ith a ctua l time s, for N = 10, 100, a nd 1000.
Trang 34Top
Trang 352.3 Growth of Functions
Most a lgorithms ha ve a primary parameter N tha t a ffe cts the running time most significa ntly The pa ra me te r N might be the de gre e of a
polynomia l, the size of a file to be sorte d or se a rche d, the numbe r of cha ra cte rs in a te xt string, or some othe r a bstra ct me a sure of the size ofthe proble m be ing conside re d: it is most ofte n dire ctly proportiona l to the size of the da ta se t be ing proce sse d W he n the re is more tha n one
such pa ra me te r (for e xa mple , M a nd N in the union–find a lgorithms tha t w e discusse d in Se ction 1.3), w e ofte n re duce the a na lysis to just one
pa ra me te r by e xpre ssing one of the pa ra me te rs a s a function of the othe r or by conside ring one pa ra me te r a t a time (holding the othe r
consta nt) so tha t w e ca n re strict ourse lve s to conside ring a single pa ra me te r N w ithout loss of ge ne ra lity Our goa l is to e xpre ss the re source
re quire me nts of our progra ms (most ofte n running time ) in te rms of N, using ma the ma tica l formula s tha t a re a s simple a s possible a nd tha t a re
a ccura te for la rge va lue s of the pa ra me te rs The a lgorithms in this book typica lly ha ve running time s proportiona l to one of the follow ingfunctions
1 Most instructions of most progra ms a re e xe cute d once or a t most only a fe w time s If a ll the instructions of a progra m ha ve this prope rty, w e
sa y tha t the progra m's running time is constant.
log
N
W he n the running time of a progra m is logarithmic, the progra m ge ts slightly slow e r a s N grow s This running time commonly occurs in
progra ms tha t solve a big proble m by tra nsforma tion into a se rie s of sma lle r proble ms, cutting the proble m size by some consta nt fra ction a t
e a ch ste p For our ra nge of inte re st, w e ca n conside r the running time to be le ss tha n a la rge consta nt The ba se of the loga rithm cha nge s
the consta nt, but not by much: W he n N is 1 thousa nd, log N is 3 if the ba se is 10, or is a bout 10 if the ba se is 2; w he n N is 1 million, log N is only double the se va lue s W he ne ve r N double s, log N incre a se s by a consta nt, but log N doe s not double until N incre a se s to N2
N W he n the running time of a progra m is linear, it is ge ne ra lly the ca se tha t a sma ll a mount of proce ssing is done on e a ch input e le me nt W he n
N is 1 million, the n so is the running time W he ne ve r N double s, the n so doe s the running time This situa tion is optima l for a n a lgorithm tha t must proce ss N inputs (or produce N outputs).
N
log
N
The N log N running time a rise s w he n a lgorithms solve a proble m by bre a king it up into sma lle r subproble ms, solving the m inde pe nde ntly,
a nd the n combining the solutions For la ck of a be tte r a dje ctive (linearithmic?), w e simply sa y tha t the running time of such a n a lgorithm is N log N W he n N is 1 million, N log N is pe rha ps 20 million W he n N double s, the running time more (but not much more ) tha n double s.
N2 W he n the running time of a n a lgorithm is quadratic, tha t a lgorithm is pra ctica l for use on only re la tive ly sma ll proble ms Qua dra tic running time s typica lly a rise in a lgorithms tha t proce ss a ll pa irs of da ta ite ms (pe rha ps in a double ne ste d loop) W he n N is 1 thousa nd, the running time is 1 million W he ne ve r N double s, the running time incre a se s fourfold.
N3 Simila rly, a n a lgorithm tha t proce sse s triple s of da ta ite ms (pe rha ps in a triple -ne ste d loop) ha s a cubic running time a nd is pra ctica l for use
on only sma ll proble ms W he n N is 100, the running time is 1 million W he ne ve r N double s, the running time incre a se s e ightfold.
2N Fe w a lgorithms w ith exponential running time a re like ly to be a ppropria te for pra ctica l use , e ve n though such a lgorithms a rise na tura lly a s
brute -force solutions to proble ms W he n N is 20, the running time is 1 million W he ne ve r N double s, the running time squa re s!
The running time of a pa rticula r progra m is like ly to be some consta nt multiplie d by one of the se te rms (the leading term) plus some sma lle r
te rms The va lue s of the consta nt coe fficie nt a nd the te rms include d de pe nd on the re sults of the a na lysis a nd on imple me nta tion de ta ils.Roughly, the coe fficie nt of the le a ding te rm ha s to do w ith the numbe r of instructions in the inne r loop: At a ny le ve l of a lgorithm de sign, it is
prude nt to limit the numbe r of such instructions For la rge N, the e ffe ct of the le a ding te rm domina te s; for sma ll N or for ca re fully e ngine e re d
a lgorithms, more te rms ma y contribute a nd compa risons of a lgorithms a re more difficult In most ca se s, w e w ill re fe r to the running time of
progra ms simply a s "line a r," "N log N," "cubic," a nd so forth W e conside r the justifica tion for doing so in de ta il in Se ction 2.4.
Eve ntua lly, to re duce the tota l running time of a progra m, w e focus on minimizing the numbe r of instructions in the inne r loop Ea ch instructioncome s unde r scrutiny: Is it re a lly ne ce ssa ry? Is the re a more e fficie nt w a y to a ccomplish the sa me ta sk? Some progra mme rs be lie ve tha t the
a utoma tic tools provide d by mode rn Ja va compile rs ca n produce the be st ma chine code or tha t mode rn VMs w ill optimize progra m pe rforma nce ;othe rs be lie ve tha t the be st route is to imple me nt critica l me thods in na tive C or ma chine code We norma lly stop short of conside ringoptimiza tion a t this le ve l, a lthough w e do occa siona lly ta ke note of how ma ny ma chine instructions a re re quire d for ce rta in ope ra tions in orde r to
he lp us unde rsta nd w hy one a lgorithm might be fa ste r tha n a nothe r in pra ctice
Table 2.1 Values of commonly encountered functions
This ta ble indica te s the re la tive size of some of the functions tha t w e e ncounte r in the a na lysis of a lgorithms The qua dra tic function cle a rly
domina te s, pa rticula rly for la rge N, a nd diffe re nce s a mong sma lle r functions ma y not be a s w e might e xpe ct for sma ll N For e xa mple , N3/2 should be gre a te r tha n N lg2 N for huge va lue s of N, but N lg2 N is gre a te r for the sma lle r va lue s of N tha t might occur in pra ctice A pre cise
cha ra cte riza tion of the running time of a n a lgorithm might involve line a r combina tions of the se functions W e ca n e a sily se pa ra te fa st a lgorithms
from slow one s be ca use of va st diffe re nce s be tw e e n, for e xa mple , lg N a nd N or N a nd N2, but distinguishing a mong fa st a lgorithms involve s
a lgorithms a re more like ly tha n fa st compute rs to be a ble to he lp us solve proble ms w ithout fa cing outra ge ous running time s
Figure 2.1 Seconds conversions
The vast difference between numbers such as 104 and 108 is more obvious when we consider them to measure time in seconds and convert
to familiar units of time We might let a program run for 2.8 hours, but we would be unlikely to contemplate running a program that would take at least 3.1 years to complete Because 210 is approximately 103, this table is useful for powers of 2 as well For example, 232 seconds
Trang 36Table 2.2 Time to solve huge problems
For ma ny a pplica tions, our only cha nce of be ing a ble to solve huge proble m insta nce s is to use a n e fficie nt a lgorithm This ta ble indica te s the
minimum a mount of time re quire d to solve proble ms of size 1 million a nd 1 billion, using line a r, N log N, a nd qua dra tic a lgorithms, w he n w e ca n
e xe cute 1 million, 1 billion, a nd 1 trillion instructions pe r se cond A fa st a lgorithm e na ble s us to solve a proble m on a slow ma chine , but a fa st
ma chine is no he lp w he n w e a re using a slow a lgorithm
A fe w othe r functions do a rise For e xa mple , a n a lgorithm w ith N2 inputs tha t ha s a running time proportiona l to N3 is be st thought of a s a n N3/2
a lgorithm Also, some a lgorithms ha ve tw o sta ge s of subproble m de composition, w hich le a d to running time s proportiona l to N log2 N It is
e vide nt from Ta ble 2.1 tha t both of the se functions a re much close r to N log N tha n to N2
The loga rithm function pla ys a spe cia l role in the de sign a nd a na lysis of a lgorithms, so it is w orthw hile for us to conside r it in de ta il Be ca use w e
ofte n de a l w ith a na lytic re sults only to w ithin a consta nt fa ctor, w e use the nota tion "log N" w ithout spe cifying the ba se Cha nging the ba se from
one consta nt to a nothe r cha nge s the va lue of the loga rithm by only a consta nt fa ctor, but spe cific ba se s norma lly sugge st the mse lve s in
pa rticula r conte xts In ma the ma tics, the natural logarithm (ba se e = 2.71828 ) is so importa nt tha t a spe cia l a bbre via tion is commonly use d:
loge N ln N In compute r scie nce , the binary logarithm (ba se 2) is so importa nt tha t the a bbre via tion log2 N lg N is commonly use d.
The sma lle st inte ge r la rge r tha n lg N is the numbe r of bits re quire d to re pre se nt N in bina ry, in the sa me w a y tha t the sma lle st inte ge r la rge r
tha n log10 N is the numbe r of digits re quire d to re pre se nt N in de cima l The Ja va sta te me nt
for (lgN = 0; N > 0; lgN++, N /= 2) ;
is a simple w a y to compute the sma lle st inte ge r la rge r tha n lg N A simila r me thod for computing this function is
for (lgN = 0, t = 1; t < N; lgN++, t += t) ;
This ve rsion e mpha size s tha t 2n N < 2 n+1 w he n n is the sma lle st inte ge r la rge r tha n lg N.
Occa siona lly, w e ite ra te the loga rithm: We a pply it succe ssive ly to a huge numbe r For e xa mple , lg lg 2256 = lg 256 = 8 As illustra te d by this
e xa mple , w e ge ne ra lly re ga rd log log N a s a consta nt, for pra ctica l purpose s, be ca use it is so sma ll, e ve n w he n N is huge
We a lso fre que ntly e ncounte r a numbe r of spe cia l functions a nd ma the ma tica l nota tions from cla ssica l a na lysis tha t a re use ful in providingconcise de scriptions of prope rtie s of progra ms Ta ble 2.3 summa rize s the most fa milia r of the se functions; w e brie fly discuss the m a nd some ofthe ir most importa nt prope rtie s in the follow ing pa ra gra phs
Our a lgorithms a nd a na lyse s most ofte n de a l w ith discre te units, so w e ofte n ha ve ne e d for the follow ing spe cia l functions to conve rt re a lnumbe rs to inte ge rs:
x : la rge st inte ge r le ss tha n or e qua l to x
x : sma lle st inte ge r gre a te r tha n or e qua l to x.
For e xa mple , p a nd e a re both e qua l to 3, a nd lg(N +1) is the numbe r of bits in the bina ry re pre se nta tion of N Anothe r importa nt use of the se functions a rise s w he n w e w a nt to divide a se t of N obje cts in ha lf W e ca nnot do so e xa ctly if N is odd, so, to be pre cise , w e divide into one subse t w ith N/2 obje cts a nd a nothe r subse t w ith N/2 obje cts If N is e ve n, the tw o subse ts a re e qua l in size ( N/2 = N/2 ); if N is odd, the y diffe r in size by 1 ( N/2 + 1 = N/2 ) In Ja va , w e ca n compute the se functions dire ctly w he n w e a re ope ra ting on inte ge rs (for e xa mple , if
N 0, the n N/2 is N/2 a nd N - (N/2) is N/2 ), a nd w e ca n use floor a nd ceil from the java.lang.Math pa cka ge to compute the m w he n
w e a re ope ra ting on floa ting point numbe rs
A discre tize d ve rsion of the na tura l loga rithm function ca lle d the harmonic numbers ofte n a rise s in the a na lysis of a lgorithms The Nth ha rmonic
numbe r is de fine d by the e qua tion
gra phics/02icon02.gif
The na tura l loga rithm ln N is the a re a unde r the curve 1/x be tw e e n 1 a nd N; the ha rmonic numbe r H N is the a re a unde r the ste p function tha t w e
de fine by e va lua ting 1/x a t the inte ge rs be tw e e n 1 a nd N This re la tionship is illustra te d in Figure 2.2 The formula
gra phics/02icon03.gif
w he re g = 0.57721 (this consta nt is know n a s Euler's constant) give s a n e xce lle nt a pproxima tion to H N By contra st w ith lg N a nd lg N , it is
be tte r to use the log me thod of java.lang.Math to compute H N tha n to do so dire ctly from the de finition
Figure 2.2 Harmonic numbers
The harmonic numbers are an approximation to the area under the curve y = 1/x The constant g accounts for the difference between H and
Trang 37gra phics/02fig02.gif
The se que nce of numbe rs
tha t a re de fine d by the formula
gra phics/02icon05.gif
a re know n a s the Fibonacci numbers, a nd the y ha ve ma ny inte re sting prope rtie s For e xa mple , the ra tio of tw o succe ssive te rms a pproa che s the golden ratio gra phics/02icon06.gif More de ta ile d a na lysis show s tha t rounde d to the ne a re st inte ge r
We a lso ha ve occa sion to ma nipula te the fa milia r factorial function N! Like the e xpone ntia l function, the fa ctoria l a rise s in the brute -force solution
to proble ms a nd grow s much too fa st for such solutions to be of pra ctica l inte re st It a lso a rise s in the a na lysis of a lgorithms be ca use it
re pre se nts a ll the w a ys to a rra nge N obje cts To a pproxima te N!, w e use Stirling's formula:
gra phics/02icon08.gif
For e xa mple , Stirling's formula te lls us tha t the numbe r of bits in the bina ry re pre se nta tion of N! is a bout N lg N.
Table 2.3 Special functions and constants
This ta ble summa rize s the ma the ma tica l nota tion tha t w e use for functions a nd consta nts tha t a rise in formula s de scribing the pe rforma nce of
a lgorithms The formula s for the a pproxima te va lue s e xte nd to provide much more a ccura cy, if de sire d (see reference section).
Most of the formula s tha t w e conside r in this book a re e xpre sse d in te rms of the fe w functions tha t w e ha ve de scribe d in this se ction, w hich a re
summa rize d in Ta ble 2.3 Ma ny othe r spe cia l functions ca n a rise in the a na lysis of a lgorithms For e xa mple , the cla ssica l binomial distribution a nd
re la te d Poisson approximation pla y a n importa nt role in the de sign a nd a na lysis of some of the funda me nta l se a rch a lgorithms tha t w e conside r in
Cha pte rs 14 a nd 15 W e discuss functions not liste d he re w he n w e e ncounte r the m
Exercises
2.5 For w ha t va lue s of N is 10N lg N > 2N2?
2.6 For w ha t va lue s of N is N3/2 be tw e e n N(lg N)2/2 a nd 2N(lg N)2?
2.7 For w ha t va lue s of N is 2NH N - N < N lg N +10N?
2.8 W ha t is the sma lle st va lue of N for w hich log10 log10 N > 8?
2.9 Prove tha t lg N + 1 is the numbe r of bits re quire d to re pre se nt N in bina ry.
2.10 Add columns to Ta ble 2.2 for N(lg N)2 a nd N3/2
2.11 Add row s to Ta ble 2.2 for 107 a nd 108 instructions pe r se cond
2.12 W rite a Ja va me thod tha t compute s H N, using the log me thod of java.lang.Math
2.13 W rite a n e fficie nt Ja va function tha t compute s lg lg N Do not use a libra ry function.
2.14 How ma ny digits a re the re in the de cima l re pre se nta tion of 1 million fa ctoria l?
2.15 How ma ny bits a re the re in the bina ry re pre se nta tion of lg(N!)?
2.16 How ma ny bits a re the re in the bina ry re pre se nta tion of H N?
2.17 Give a simple e xpre ssion for lg F
Trang 382.19 Give the la rge st va lue of N for w hich you ca n solve a proble m tha t re quire s a t le a st f(N) instructions on a ma chine tha t ca n
e xe cute 109 instructions pe r se cond, for the follow ing functions f(N): N3/2, N5/4, 2NH N , N lg N lg lg N, a nd N2 lg N.
Previous Section
Top
Trang 392.4 Big-Oh Notation
The ma the ma tica l a rtifa ct tha t a llow s us to suppre ss de ta il w he n w e a re a na lyzing a lgorithms is ca lle d the O-notation, or "big-Oh nota tion," w hich
is de fine d a s follow s
Definition 2.1 A function g(N) is said to be O(f(N)) if there exist constants c0 and N0 such that g(N) < c0f(N) for all N > N0.
W e use the O-nota tion for thre e distinct purpose s:
To bound the e rror tha t w e ma ke w he n w e ignore sma ll te rms in ma the ma tica l formula s
To bound the e rror tha t w e ma ke w he n w e ignore pa rts of a progra m tha t contribute a sma ll a mount to the tota l be ing a na lyze d
To a llow us to cla ssify a lgorithms a ccording to uppe r bounds on the ir tota l running time s
W e conside r the third use in Se ction 2.7 a nd discuss brie fly the othe r tw o he re
The consta nts c0 a nd N0 implicit in the O-nota tion ofte n hide imple me nta tion de ta ils tha t a re importa nt in pra ctice Obviously, sa ying tha t a n
a lgorithm ha s running time O(f(N)) sa ys nothing a bout the running time if N ha ppe ns to be le ss tha n N0, a nd c0 might be hiding a la rge a mount of
ove rhe a d de signe d to a void a ba d w orst ca se W e w ould pre fe r a n a lgorithm using N2 na nose conds ove r one using log N ce nturie s, but w e could not ma ke this choice on the ba sis of the O-nota tion.
Ofte n, the re sults of a ma the ma tica l a na lysis a re not e xa ct but ra the r a re a pproxima te in a pre cise te chnica l se nse : The re sult might be a n
e xpre ssion consisting of a se que nce of de cre a sing te rms Just a s w e a re most conce rne d w ith the inne r loop of a progra m, w e a re most
conce rne d w ith the leading terms (the la rge st te rms) of a ma the ma tica l e xpre ssion The O-nota tion a llow s us to ke e p tra ck of the le a ding te rms
w hile ignoring sma lle r te rms w he n ma nipula ting a pproxima te ma the ma tica l e xpre ssions a nd ultima te ly a llow s us to ma ke concise sta te me ntstha t give a ccura te a pproxima tions to the qua ntitie s tha t w e a na lyze
Some of the ba sic ma nipula tions tha t w e use w he n w orking w ith e xpre ssions conta ining the O-nota tion a re the subje ct of Exe rcise s 2.20 through
2.25 Ma ny of the se ma nipula tions a re intuitive , but ma the ma tica lly incline d re a de rs ma y be inte re ste d in w orking Exe rcise 2.21 to prove the
va lidity of the ba sic ope ra tions from the de finition Esse ntia lly, the se e xe rcise s sa y tha t w e ca n e xpa nd a lge bra ic e xpre ssions using the nota tion a s though the O w e re not the re , the n drop a ll but the la rge st te rm For e xa mple , if w e e xpa nd the e xpre ssion
O-gra phics/10icon04.gif
w e ge t six te rms
gra phics/02icon12.gif
but ca n drop a ll but the la rge st O-te rm, le a ving the a pproxima tion
Tha t is, N2 is a good a pproxima tion to this e xpre ssion w he n N is la rge The se ma nipula tions a re intuitive , but the O-nota tion a llow s us to e xpre ss the m ma the ma tica lly w ith rigor a nd pre cision W e re fe r to a formula w ith one O-te rm a s a n asymptotic expression.
For a more re le va nt e xa mple , suppose tha t (a fte r some ma the ma tica l a na lysis) w e de te rmine tha t a pa rticula r a lgorithm ha s a n inne r loop tha t is
ite ra te d 2NH N time s on the a ve ra ge , a n oute r se ction tha t is ite ra te d N time s, a nd some initia liza tion code tha t is e xe cute d once Suppose furthe r tha t w e de te rmine (a fte r ca re ful scrutiny of the imple me nta tion) tha t e a ch ite ra tion of the inne r loop re quire s a0 na nose conds, the oute r
se ction re quire s a1 na nose conds, a nd the initia liza tion pa rt a2 na nose conds The n w e know tha t the a ve ra ge running time of the progra m (in
na nose conds) is
But it is a lso true tha t the running time is
This simple r form is significa nt be ca use it sa ys tha t, for la rge N, w e ma y not ne e d to find the va lue s of a1 or a2 to a pproxima te the running time
In ge ne ra l, the re could w e ll be ma ny othe r te rms in the ma the ma tica l e xpre ssion for the e xa ct running time , some of w hich ma y be difficult to
a na lyze The O-nota tion provide s us w ith a w a y to ge t a n a pproxima te a nsw e r for la rge N w ithout bothe ring w ith such te rms.
Continuing this e xa mple , w e a lso ca n use the O-nota tion to e xpre ss running time in te rms of a fa milia r function, ln N In te rms of the O-nota tion, the a pproxima tion in Ta ble 2.3 is e xpre sse d a s H N = ln N + O(1) Thus, 2a0N ln N + O(N) is a n a symptotic e xpre ssion for the tota l running time of our a lgorithm We e xpe ct the running time to be close to the e a sily compute d va lue 2 a0 N ln N for la rge N The consta nt fa ctor a0 de pe nds on thetime ta ke n by the instructions in the inne r loop
Furthe rmore , w e do not ne e d to know the va lue of a0 to pre dict tha t the running time for input of size 2N w ill be a bout tw ice the running time for input of size N for huge N be ca use
gra phics/02icon16.gif
Tha t is, the a symptotic formula a llow s us to ma ke a ccura te pre dictions w ithout conce rning ourse lve s w ith de ta ils of e ithe r the imple me nta tion or
Trang 40We a re so ofte n in the position of counting the numbe r of time s tha t fixe d-cost ope ra tions a re pe rforme d a nd w a nting to use the le a ding te rm to
e stima te the re sult tha t w e norma lly ke e p tra ck of only the le a ding te rm, a ssuming implicitly tha t a pre cise a na lysis like the one just give n could
be pe rforme d, if ne ce ssa ry
W he n a function f(N) is a symptotica lly la rge compa re d to a nothe r function g(N) (tha t is, g(N)/f(N) 0 a s N ), w e some time s use in this book the (de cide dly nonte chnica l) te rminology about f(N) to me a n f(N) + O(g(N)) W ha t w e se e m to lose in ma the ma tica l pre cision w e ga in in cla rity, for w e
a re more inte re ste d in the pe rforma nce of a lgorithms tha n in ma the ma tica l de ta ils In such ca se s, w e ca n re st a ssure d tha t, for la rge N (if not for
a ll N), the qua ntity in que stion w ill be close to f(N) For e xa mple , e ve n if w e know tha t a qua ntity is N(N - 1)/2, w e ma y re fe r to it a s be ing a bout
N2/2 This w a y of e xpre ssing the re sult is more quickly unde rstood tha n the more de ta ile d e xa ct re sult a nd, for e xa mple , de via te s from the truth only by 0.1 pe rce nt for N = 1000 The pre cision lost in such ca se s pa le s by compa rison w ith the pre cision lost in the more common usa ge O(f(N)).
Our goa l is to be both pre cise a nd concise w he n de scribing the pe rforma nce of a lgorithms
In a simila r ve in, w e some time s sa y tha t the running time of a n a lgorithm is proportional to f(N) w he n w e ca n prove tha t it is e qua l to cf(N) + g(N)
w ith g(N) a symptotica lly sma lle r tha n f(N) W he n this kind of bound holds, w e ca n proje ct the running time for, sa y, 2 N from our obse rve d running time for N, a s in the e xa mple just discusse d Figure 2.3 give s the fa ctors tha t w e ca n use for such proje ction for functions tha t commonly a rise in
the a na lysis of a lgorithms Couple d w ith e mpirica l studie s (se e Se ction 2.1), this a pproa ch fre e s us from the ta sk of de te rmining imple me nta
tion-de pe ntion-de nt consta nts in tion-de ta il Or, w orking ba ckw a rd, w e ofte n ca n e a sily tion-de ve lop a n hypothe sis a bout the functiona l grow th of the running time
of a progra m by de te rmining the e ffe ct of doubling N on running time
Figure 2.3 Effect of doubling problem size on running time
Predicting the effect of doubling the problem size on the running time is a simple task when the running time is proportional to certain simple functions, as indicated in this table In theory, we cannot depend on this effect unless N is huge, but this method is surprisingly effective Conversely, a quick method for determining the functional growth of the running time of a program is to run that program empirically,
doubling the input size for N as large as possible, then work backward from this table.
gra phics/02fig03.gif
The distinctions a mong O-bo unds , is proportional to, a nd about a re illustra te d in Figure s 2.4 a nd 2.5 We use O-nota tion prima rily to le a rn the funda me nta l a symptotic be ha vior of a n a lgorithm; is proportional to w he n w e w a nt to pre dict pe rforma nce by e xtra pola tion from e mpirica l studie s;
a nd about w he n w e w a nt to compa re pe rforma nce or to ma ke a bsolute pe rforma nce pre dictions.
Figure 2.4 Bounding a function with an O-approximation
In this schematic diagram, the oscillating curve represents a function, g(N),which we are trying to approximate; the black smooth curve represents another function, f(N), which we are trying to use for the approximation; and the gray smooth curve represents cf(N) for some unspecified constant c The vertical line represents a value N0, indicating that the approximation is to hold for N > N0 When we say that g(N)
= O(f(N)), we expect only that the value of g(N) falls below some curve the shape of f(N) to the right of some vertical line The behavior of
f(N) could otherwise be erratic (for example, it need not even be continuous).
gra phics/02fig04.gif
Figure 2.5 Functional approximations
When we say that g(N) is proportional to f(N) (top), we expect that it eventually grows like f(N) does, but perhaps offset by an unknown constant Given some value of g(N), this knowledge allows us to estimate it for larger N When we say that g(N) is about f(N) (bottom), we
expect that we can eventually use f to estimate the value of g accurately.
gra phics/02fig05.gif
Exercises
2.20 Prove tha t O(1) is the sa me a s O(2).