Using smt solver and symbolic execution to generate test inputs for c programs

End if End ifUntil all branches are reached or solving is not successful;... 5th joint meeting of the European Software.. Communications of the ACM.. Lecture Notes in Computer Science...

A thesis s u b m itte d in fulfillment of th e req u irem en ts for the degree of

M aster of C o m p u ter Science

Ju n e, 2010

T a b l e o f C o n t e n t s

1.1 M o t i v a t i o n a n d c o n t r i b u t i o n 1

1 2 B a c k g r o u n d 3

1 2 1 S M T s o l v e r 3

1 2 2 S y m b o l i c e x e c u t i o n 5

1 2 3 U s i n g S M T s o l v e r s a n d S y m b o l i c E x e c u t i o n t o G e n e r a t e T e s t I n p u t s 8

1.3 T e x t o v e r v i e w 8

1.1 R e l a t e d w o r k 9

2 C R E S T 1 2 2.1 A r c h i t e c t u r e o f C R E S T 12

2 1 1 I n s t r u m e n t a t i o n t o o l 13

2 1 2 C i I l i b r a r y f o r p e r f o r m i n g c o n c o l i c e x e c u t i o n 17

2 1 3 S e a r c h s t r a t e g i e s m o d u l e 19

2 2 L i m i t a t i o n s o f C R E S T 2 0

3 r e a l C R E S T : A n e w t e s t g e n e r a t i o n t o o l b a s e d o n C R E S T 2 3 3.1 M a i n i m p r o v e m e n t s c o m p a r e d t o C R E S T 2 3

3 1 1 H a n d l i n g f l o a t i n g - p o i n t v a r i a b l e s .2 3

3 1 2 P r o c e s s i n g d i v i s i o n o p e r a t o r 2 1

3 1 3 C o m b i n i n g m u l t i p l e S M T s o l v e r s t o e n h a n c e t e s t i n p u t g e n e r ­

3 2 r e a l C R E S T s a r c h i t e c t u r e 2 7

L i s t o f F i g u r e s

1.1 P r o g r a m ' s s e g m e n t u s e d t o e x c h a n g e t w o v a r i a b l e ’s v a l u e s a n d c o r r e l a ­

t i v e s y m b o l i c e x e c u t i o n t r e e E a c h s t a t e ' s t r a n s f o r m a t i o n is n u m b e r e d

b y t h e n u m b e r o f s t a t e m e n t i n p r o g r a m s 6

1 2 S y m b o l i c t e s t i n g t e c h n i q u e 8

1 3 T e s t g e n e r a t i o n life c y c l e 9

2 1 C o - o p e r a t i o n o f t h r e e m a i n p a r t s i n C R E S T 13

2 2 “c r e s t c ” s c r i p t file t o i n s t r u m e n t s o u r c e c o d e 14

2 3 A c t i v i t y o f i n s t r u m e n t a t i o n t o o l 15

2 4 O v e r v i e w o f C + 4 - l i b r a r y f o r p e r f o r m i n g s y m b o l i c e x e c u t i o n 17

2 5 P r o t o t y p e o f c l a s s S y m b o l i c E x p r 18

2 6 P r o t o t y p e o f c l a s s S y m b o l i c P r c d 19

2 7 P r o t o t y p e o f c l a s s S v m b o l i c P a t h 2 0 2 8 P r o t o t y p e o f c l a s s S y m b o l i c E x e c u t i o n 21

2 9 P r o t o t y p e o f c l a s s S y m b o l i c l n t e r p r e t e r 21

2 1 0 P r o t o t y p e o f c l a s s Y i c e s S o l v e r 2 2 2 1 1 O v e r v i e w o f s e a r c h s t r a t e g i e s m o d u l e 2 2 3.1 C o m b i n i n g m u l t i s o l v e r s a l g o r i t h m 2 6 3 2 O v e r v i e w o f r e a l C R E S T 2 8 1.1 T i m e c o s t s 31

vi

L i s t o f T a b l e s

4 1 B ra n c h es coverage

1.1 M o t i v a t i o n a n d c o n t r i b u t i o n 2

T h e r e a r e m a n y s t a t e - o f - t h e - a r t s o l v e r s s u c h a s Z 3 ( M a u r a & B j o r n 2 0 0 8 ) C Y C 3 ( B a r r e t t i c T i n e l l i 2 0 0 7 ) V i c e s ( S R I , 2 0 0 8 ) , B a r c e l o g i c ( M i q u e l B o f i l l k R u b i o

1.2 B a c k g r o u n d 3

S M T s o l v e r is t h e n e x t g e n e r a t i o n o f S A T s o l v e r ( A d i t y a P a r a m e s w a r a n , 2 0 0 5 ) ( G u r e v i c h , 1 9 8 9 ) A n S A T s o l v e r is a t o o l f o r s o l v i n g t h e B o o l e a n S a t i s f i a b i l i t y

b y b o t h s y m b o l i n p u t s a n d c o n c r e t e i n p u t s In t h e e x e c u t i o n p r o c e s s , e v e r y i n p u t

c o n s t r a i n t t h a t d r i v e e x e c u t i o n f o l l o w a n u n r e a c h e d b r a n c h a r e g a t h e r e d T h i s c o n ­

s t r a i n t is s o l v e d t o c r e a t e n e w i n p u t D y n a m i c S y m b o l i c E x e c u t i o n is u s e d in P e x ( N i k o l a i T i l l m a n n , 2 0 0 8 ) a u n i t t e s t t o o l in N E T f r a m e w o r k , t h a t u s i n g S M T s o l v e r

L o o p - E x t e n d e d S v m b o l i c E x e c u t i o n ( L E S E ) ( P r a t e c k S a x c n a , 2 0 0 9 ) : I t is a n i m ­

-1.3 T e x t o v e r v i e w 8

Create initial test values,Repeat

Perform symbolic execution along with concrete execution,

If unreached branch exists thenBuild a constraint system to reach that branch Solve the constraint by SMT Solver:

If solving is successful thenGenerate test values from concrete model

End if End ifUntil (all branches are reached) or (solving is not successful);

E x ecu tio n path

2.1 A r c h i t e c t u r e of C R E S T 13

C source codes

Instrumentation toolv _J

\ /Instrumented codes, path

2.1 A r c h i t e c t u r e o f C R E S T

4 ! bi r i / 1h h

\'t I H= : i r HrlT.ri $0 / C! ! I Y = $ {H ; k } / c i 1 ‘ i r; / c. ! 1 1 yi■ ak r= e x p r 51 : ' \\.* \ ) \ c ’r-T, - r u]■ :0 t- r i L m t , CO U n! f ¡ jVi 00 u r11 or9 furic 'ririp o f g

EXTERN void C r e s t St o re ( C R EST _ID _CREST_ADDR) „ S K I P :

EXTERN void C r e s t C l e a r S t ack ( „ C R E S T ID ) S K IP :

E X T E R N v o id C r e s t B r a n c h ( C R E S T I D C R E S T J 3 R A N C I L I D C R E S T J 3 0 0 L ) S K I P :

E X T E R N v o id C r e s t C a l l ( C R E S T I D _ C R E S T F U N C T I O N I D ) S K I P :

E X T E R N v o id C r c s t l l e t u r n ( C R E S T I D ) „ S K I P :

E X T E R N v o id C r e s t H a n d le R e t u r n ( C R E S T I D , _ C R E S T _ V A L U E ) „ S K I P :

T h e se calls (loosely) c o rre sp o n d to an e x e cu tio n of th e p ro g ra m u n d e r test by

a sta c k m achine It is in te n d e d t h a t th ese calls be used to sy m b o lically e x e cu te

th e p ro g ra m u n d e r te s t, by m a in ta in in g a sym bolic s ta c k (alo n g w ith a sym bolic

m em o ry m a p ) Specifically:

A C ex p ressio n (w ith no side effects) g e n e ra te s a series of L oad a n d A p p ly calls

c o rre s p o n d in g to th e “postfix"' e v a lu a tio n of th e ex p ressio n , u sin g a s ta c k (i.e a L oad

in d ic a te s t h a t a value is p u sh e d o n to th e sta c k , a n d u n a ry a n d b in a ry o p e ra tio n s are a p p lie d to o n e /tw o values p o p p e d off th e sta c k ) For e x a m p le, th e expression:

”a * b > 3 + c "

w ould g e n e ra te th e in s tru m e n ta tio n :

L o a d (& a , a)

2.1 A r c h i t e c t u r e o f C R E S T 15

C source code

1Crestlnstrument ml

2.1 A r c h i t e c t u r e of C R E S T Hi

T h e i n s t r u m e n t a t i o n f o r f u n c t i o n c a l l s is s o m e w h a t c o m p l i c a t e d , b e c a u s e w e h a v e

t o h a n d l e t h e c a s e w h e r e a n i n s t r u m e n t e d c o d e c a l l s a n u n - i n s t r u m e n t e d f u n c t i o n ( C R E S T c u r r e n t l y f o r b i d u n - i n s t r u m e n t e d c o d e f r o m c a l l i n g b a c k i n t o i n s t r u m e n t e d

2.1 A r c h i t e c t u r e of C R E S T 18

Sym bolicExpr

- c o n s t j value J : -co e ff_: m a p < v a r_ t,v a lu e j:>

< < create > >-SymbolicExpr()

< C cre ate > >-5ym bolicExpr(c: value J : )

<< create> > -Sym bolicExpr(c; v a lu e j:, v: v a r_ t)

< < create > >-Sym bolicExpr(e: SymbolicExpr)

< <destroy > >-5ymbolicExpr() + N eg ate(): void

+ IsC o n cre te (): bool + Size(): s i z e j + Append V ars(vars: s e t< v a r_t> ): void + D epen dsO n (vars: m a p < v a r_t,ty p e _t> ): bool +AppendToString(s: string): vdd

+Serialize(s: string): void + P a rse (s: ¡stream): bool

< < C pp O perator> > -h+ = (e: SyrrbolicExpr): SymbolicExpr

< < CppO perator> > -h-= (e: SymbolicExpr): SymbolicExpr

2.1 A r c h i t e c t u r e o f C R E S T 19

Sym b olicP red

-o p_: com pare_op _t -expr j SymbolicExpr

< c c re a te > >-SymbolicPred()

< <create>>-Sym bolicPred(op: co m pare_o pJ:, expr: SymbolicExpr)

< <destroy > >-5ymbolicPred() + N egate(): void

+AppendToString(s: string): void + Serialize(s: string): void + P a rse (s: ¡stream ): bool +Equal(p: SymbolicPred): bool

■f A ppendV ars(vars: se t< v a r _t> ): void + D epen d sO n (vars: map < var _ t , typ e_t >): bool + o p(): com pare_op_t

2.2 L i m i t a t i o n s o f C R E S T 2 0

S ym b olicP ath

- b r a n c h e s j vector < b ra n ch JdJ >

-constraints J d x j vector <size_t >

-constraints j vector <5ymbolicPred* >

< C cre a te > >-Sy mbolicPath()

< < create > >-Sym bolicPath(pre_allocate: bool)

< « d estro y > >-5ymbolicPath() + Sw ap (sp: SymbolicPath): void +Push(bid: b ran ch _id _t): void +Push(bid: branch J d _ t , constraint: SymbolicPred): void +Serialize(s: string): void

+ P a rse (s: ¡stream ): bool + b ra n ch e s(): vector cb ran ch _ d _ t >

+constraints(): vecto r <Sym bdicPred*>

+co n strain ts_idx(): vector <size_t >

+ P a rse (s: istream ): bool + v a rs(): m a p < v a r_t,ty p e _t>

+inputs(): vector < valu e_t>

+ path(): SymbolicPath + m utable_vars(): m a p < v a r_t,ty p e _t>

+ m u tab leJn p u ts(): vector C valu e_t >

+Load(id: id_t, addr: a d d r_t, value: v alu e_t): void

+5tore(id: id_t, addr: ad d r_t): void

+ A pplyllnaryO p(id: id _ t, op: unary_cp t, value: v a lu e _t): void

+ Apply Binary Op(id: id _tJ op: binary _cp t, value: v a lu e _t): void

+ApplyCom pareOp(id: id _t, op: compare j d p J , value: value J :) : void

+Call(id: id_t., fid: f unction Jd t): void

+Return(id: id _t): void

+HandleReturn(id: id _t, value: value J ) : void

+Branch(id: id_t, bid: b r a n c h J d J :, pred_value: bool): void

+ N ew lnput(type: type t, addr: a d d r j) : valu e_t

+ e xe cu tio n (): SymbolicExecution

+DumpMernory(): void

+DumpPath(): void

-PushConcrete('value: v a lu e _t): void

-PushSymbolic(expr: SymbolicExpr^ value: v alu e_t): void

-C learP red icateR egister(): void

F i g u r e 2 9 : P r o t o t y p e o f c l a s s S y m b o l i c l n t c r p r e t e r

2.2 L i m i t a t i o n s o f C R E S T 22

YícesSolver+Incnerrental5olveioki soln: vector<value t>, vara: rrap<var t,type t >, constraints: vectoKconst 5vrnbolicRied*>, soln: rrap<var t,vabe t>): bool +5olveivars: map<var Uvpe t>, constraints: vector<const 5vrrftolicFYed*>, soln: rnap<var t.vale t»:bool

4-ReadSolutionFrorrFieOrDi&rfile: string, soln: map<var Lvalue t>): bool

F i g u r e 2 1 0 : P r o t o t y p e o f c l a s s Y i c e s S o l v e r

F i g u r e 2 1 1 : O v e r v i e w o f s e a r c h s t r a t e g i e s m o d u l e

3.2 r e a l C R E S T ’s a r c h i t e c t u r e 28

F i g u r e 3 2 : O v e r v i e w o f r e a l C R E S T

Deep First Line Search Heuristic Search Search Random

strategy

F i g u r e 4 1 : T i m e c o s t s

G N e c u l a , S M c P e a k , S K k W e i m e r \ V ( 2 0 0 2 ) C i l : I n t e r m e d i a t e l a n g u a g e a n d

t o o l s f o r a n a l y s i s a n d t r a n s f o r m a t i o n o f e p r o g r a m s Proceedings of Conference on Compiler Construction

G u r e v i c h , Y ( 1 9 8 9 ) T h e l o g i c i n c o m p u t e r s c i e n c e c o l u m n Bulletin of the eatcs,

38. 9 3 - 1 0 0

J a c o b B u r n i m K S ( 2 0 0 8 ) H e u r i s t i c s f o r s c a l a b l e d y n a m i c t e s t g e n e r a t i o n No

U C B/EECS-2008-123.

K S e n , D M , k A g h a , G ( 2 0 0 5 ) C u t e : A c o n c o l i c u n i t t e s t i n g e n g i n e f o r c 5th joint meeting of the European Software.

K i n g , J C ( 1 9 7 6 ) S y m b o l i c e x e c u t i o n a n d p r o g r a m t e s t i n g Communications of the ACM 19, 3 8 5 3 9 4

M a u r a , L D , k B j o r n , N ( 2 0 0 8 ) Z 3 : A n e f f i c i e n t s m t s o l v e r Lecture Notes in Computer Science Jt963, 3 3 7 - 3 4 0

M i q u e l B o f i l l , R o b e r t N i e u w e n h u i s , A O E R - C , k R u b i o A ( 2 0 0 8 ) T h e b a r c e -

l o g i c s m t s o l v e r ( t o o l p a p e r ) Lecture Notes in Computer Science, 5123, 2 9 4 - 2 9 8

N i k o l a i T i l l m a n n , J d H ( 2 0 0 8 ) P e x w h i t e b o x t e s t g e n e r a t i o n f o r n e t Tests and Proofs, ^966. 1 3 4 - 1 5 3

B b i g a h 12

T a o X i e , N i k o l a i T i l l m a n n P d H k S c h u l t e \Y ( L i s b o n P o r t u g a l J u n e - J u l y

2 0 0 9 ) F i t n e s s - g u i d e d p a t h e x p l o r a t i o n i n d y n a m i c s y m b o l i c e x e c u t i o n Pro­ ceedings of the 39th Annual IE E E /IF IP International Conference on Dependable Systems and Networks.

T h o m a s S t t z l e , F a c h g e b i e t I n t e l l e k t i k M C I D ( 2 0 0 7 ) Stochastic local search algorithms for the graph colouring p r o b l e mv o l H a n d b o o k o n A p p r o x i m a t i o n

A l g o r i t h m s a n d M e t a h e u r i s t i c s , c h a p t e r 6 3

Y a m i n i K a n n a n K S ( J u l y 2 0 - 2 4 , 2 0 0 8 ) U n i v e r s a l s y m b o l i c e x e c u t i o n a n d i t s

a p p l i c a t i o n t o l i k e l y d a t a s t r u c t u r e i n v a r i a n t g e n e r a t i o n Proceedings of the 2008 international symposium on Software testing and analysis.