Báo cáo khoa học: "THE SYNTAX AND SEMANTICS OF USER-DEFINED MODIFIERS IN A TRANSPORTABLE NATURAL LANGUAGE PROCESSOR" pot

The User-Phrase portion of L D C resembles familiar natural language database query systems such as INTELLECT, JETS.. For example, students w h o take a certain course are precisely tho

IN A

T R A N S P O R T A B L E N A T U R A L L A N G U A G E P R O C E S S O R

B r u c e W Ballard Dept of C o m p u t e r Science

D u k e University

D u r h a m , N.C 27708

A B S T R A C T

The Layered D o m a i n Class system (LDC) is an

experimental natural language processor being

developed at D u k e University which reached the

prototype stage in M a y of 1983 Its primary goals are

(I) to provide English-language retrieval capabilities

for structured but u n n o r m a U z e d data files created b y

the user, (2) to allow very c o m p l e x semantics, in terms

of the information directly available f r o m the physical

data file; a n d (3) to enable users to customize the

system to operate with n e w types of data In this paper

w e shall discuss (a) the types of modifiers L D C provides

for; (b) h o w information about the syntax a n d

semantics of modifmrs is obtained f r o m users; a n d (c)

h o w this information is used to process English inputs

I I N T R O D U C T I O N

The Layered D o m a i n Class s y s t e m (LDC) is an

experimental natural language processor being

developed at D u k e .University In this paper w e

concentrate on the typ.~s of modifiers provided by L D C

a n d the m e t h o d s by which the system acquires

information about the syntax a n d semantics of user-

defined modifiers A m o r e complete description is

available in [4,5], a n d further details on matters not

discussed in this paper can be found in [1,2,6,8,9]

The L D C system is m a d e up of two primary

c o m p o n e n t s First, t h e Ic'nowledge aeTui.~i2ion

c o m p o n e n t , w h o s e job is to find o u t a b o u t t h e

v o c a b u l a r y a n d s e m a n t i c s of t h e l a n g u a g e to be u s e d

f o r a n e w d o m a i n , t h e n i n q u i r e a b o u t t h e c o m p o s i t i o n

of t h e u n d e r l y i n g i n p u t file S e c o n d , t h e U s e r - P h a s e

Processor, w h i c h e n a b l e s a u s e r to o b t a i n s t a t i s t i c a l

reductions on his or her data by typed English inputs

The top-level design of the User-Phase processor

involves a linear sequence of modules for scavtvtir~g the

input a n d looking up each token in the dictionary;

pars/rig the s c a n n e d input to determine its syntactic

structure; translatiort of the parsed input into an

appropriate formal query; and finally query processing

This research has b e e n supported in part by the

National Science Foundation, Grants MCS-81-16607 a n d

IST-83-01994; in part by the National Library of

Medicine, Grant LM-07003; and in part by the Air Force

Office of S c i e n t i f i c R e s e a r c h , G r a n t 81-0221

The User-Phrase portion of L D C resembles familiar natural language database query systems such as INTELLECT, JETS L A D D E R , L U N A R PHLIQA, PLANES, REL,

R E N D E Z V O U S , TQA, a n d U S L (see [10-23]) while the overall L D C s y s t e m is similar in its objectives to m o r e recent systems s u c h as ASK, C O N S U L , IRUS, a n d T E A M (see [24-319

At the time of this writing, L D C has been

c o m p l e t e l y c u s t o m i z e d f o r two f a i r l y c o m p l e x d o m a i n s

f r o m w h i c h e x a m p l e s a r e d r a w n in t h e r e m a i n d e r of t h e

p a p e r , a n d s e v e r a l s i m p l e r o n e s T h e c o m p l e x d o m a i n s

a r e a 2 ~ a l gTz, d e s d o m a i n , giving c o u r s e g r a d e s for

s t u d e n t s in a n a c a d e m i c d e p a r t m e n t , a n d a bu~di~tg

~rgsvtizatiovt d o m a i n , c o n t a i n i n g i n f o r m a t i o n o n t h e floors, wings, c o r r i d o r s , o c c u p a n t s , a n d so f o r t h f o r o n e

o r m o r e b u i l d i n g s A m o n g t h e s i m p l e r d o m a i n s LDC h a s

b e e n c u s t o m i z e d f o r a r e files giving e m p l o y e e

i n f o r m a t i o n a n d s t o c k m a r k e t q u o t a t i o n s

II M O D I F I E R T Y P E S P R O V I D E D F O R

As s h o w n in [4] L D C handles inputs about as complicated as

students w h o were given a passing grade by an instructor Jim took a graduate course f r o m

As suggested here, m o s t of the syntactic a n d semantic sophistication of inputs to L D C are due to n o u n phrase modifiers, including a fairly broad coverage of relative clauses For example, if L D C is told that "students take courses from instructors", it will accept such relative clause forms as

students w h o took a graduate course from Trivedi courses Sarah took f r o m Rogers

instructors Jim took a graduate course f r o m courses that were taken by Jim

students w h o did not take a course from Rosenberg

W e s u m m a r i z e the modifier types distinguished by L D C

in Table i which is divided into four parts roughly corresponding to pre-norninal, nominal, post-nominal,

a n d negating modifiers W e have included several modifier types, m o s t of t h e m anaphorie, which are processed syntactically, a n d m e t h o d s for w h o s e semantic processing are being implemented along the lines suggested in [7]

e x p l a n a t o r y , b u t t h e r e a d e r will n o t i c e t h a t we h a v e

c h o s e n t o c a t e g o r i z e v e r b s , b a s e d u p o n t h e i r

semantics, as tr~Isial verbs, irrtplied p a r a ~ t e r verbs;

a n d operational verbs "Trivial" verbs, which involve n o

semantics to speak of, c a n be roughly p a r a p h r a s e d as

"be associated with" For example, students w h o take a

certain course are precisely those students associated

~ith the database records related to the course

"Implied parameter" verbs can be p a r a p h r a s e d as a

longer "trivial" verb phrase by adding a p a r a m e t e r a n d

requisite noise words for syntactic acceptability For

example, students w h o fai/a course are those students

w h o rrmlce a grade of F in the course Finally,

"operational" verbs require an operation to be performed on one or m o r e of its n o u n phrase arguments, rather than simply asking for a c o m p a r i s o n

of its n o u n phrase referent(s) against values in specified fields of the physical data file For example, the students w h o oz~tscure Jim are precisely those students w h o Trtake a grade h~gher than the grade of

Jirm At present, prepositions are treated semantically

as trivial verbs, so that "students in AI" is interpreted

as "students associated with records related to the AI course"

T a b l e 1 - M o d i f i e r T y p e s A v a i l a b l e in LDC

M o d i f i e r T y p e E x a m p l e Usage

Syntax

I m p l e m e n t e d

Semantics

I m p l e m e n t e d

Anaphoric

Anaphoric

Argument-Taking N o u n classmates of Jim

Anaphoric

Implied-Parameter

Operational

(of m a n y sorts) offices not adjacent to X-23B

etc

The job of t h e k n o w l e d g e a c q u i s i t i o n m o d u l e

of LDC, c a l l e d " P r e p " in F i g u r e 1, is t o ' f i n d o u t a b o u t

(a) t h e v o c a b u l a r y of t h e n e w d o m a i n a n d (b) t h e

c o m p o s i t i o n of t h e p h y s i c a l d a t a file T h i s p a p e r is

c o n c e r n e d o n l y with v o c a b u l a r y a c q u i s i t i o n , w h i c h

o c c u r s in t h r e e s t a g e s In S t a g e 1, P r e p a s k s t h e u s e r

to n a m e e a c h ent~.ty, o r c o n c e p t u a l d a t a i t e m , of t h e

d o m a i n As e a c h e n t i t y n a m e is g i v e n , P r e p a s k s f o r

s e v e r a l s i m p l e k i n d s of i n f o r m a t i o n , a s in

E N T I T Y N A M E ? section

S Y N O N Y M S : class

T Y P E (PERSON, N U M B E R , LIST, P A T T E R N , N O N E ) ?

p a t t e r n

GIVE 2 OR 3 EXAMPLE NAMES: e p s S l 1 2 , e e 3 4 1

NOUN SUBTYPES: n o n e

ADJECTIVES: l a r g e , s m a l l

NOUN MODIFIERS: n o n e

HIGHER LEVEL ENTITIES: c l a s s

LOWER LEVEL ENTITIES: s t u d e n t , i n s t r u c t o r

MULTIPLE ENTITY? y e s

O R D E R E D ENTITY? yes

P r e p n e x t d e t e r m i n e s t h e c a s e s t r u c t u r e of v e r b s

h a v i n g t h e given e n t i t y a s s u r f a c e s u b j e c t , a s in

A C Q U I R I N G V E R B S F O R S T U D E N T :

A S T U D E N T C A N pass a course

fail a course take a course f r o m a n instructor

m a k e a grade f r o m a n instructor

m a k e a grade in a course

In Stage 2, Prep learns the rnorhological variants of

words not k n o w n to it, e.g plurals for nouns,

comparative a n d superlative forms for adjectives, a n d

past tense a n d participle forms for verbs For example,

P A S T - T E N S E V E R B ACQUISITION

P L E A S E GIVE C O R R E C T E D F O R M S , O R HIT R E T U R N

FAIL FAILED >

BITE BITED > bit

T R Y TRIED >

In S t a g e 3, P r e p a c q u i r e s t h e semantics of a d j e c t i v e s ,

v e r b s , a n d o t h e r m o d i f i e r t y p e s , b a s e d u p o n t h e

following p r i n c i p l e s

1 S y s t e m s w h i c h a t t e m p t to a c q u i r e complex

s e m a n t i c s f r o m relatively untrained u s e r s h a d

b e t t e r r e s t r i c t t h e c l a s s of t h e d o m a i n s t h e y s e e k

to p r o v i d e a n i n t e r f a c e to

F o r t h i s r e a s o n , LDC r e s t r i c t s i t s e l f to a c l a s s of

d o m a i n s [1] in w h i c h t h e i m p o r t a n t r e l a t i o n s h i p s

a m o n g d o m a i n e n t i t i e s involve h i e r a r c h i c a l

d e c o m p o s i t i o n s

2 T h e r e n e e d n o t be a n y c o r r e l a t i o n b e t w e e n t h e type

of m o d i f i e r b e i n g d e f i n e d a n d t h e way in w h i c h its

rr~eaTt/rtg r e l a t e s to t h e u n d e r l y i n g d a t a file

For t h i s r e a s o n , P r e p a c q u i r e s t h e m e a n i n g s of all

u s e r - d e f i n e d m o d i f i e r s in t h e s a m e m a n n e r b y

p r o v i d i n g s u c h p r i m i t i v e s as id, t h e i d e n t i t y f u n c t i o n ;

w h i c h r e t u r n s t h e size of i t s a r g u m e n t , w h i c h is

a s s u m e d to be a set; s u m , w h i c h r e t u r n s t h e s u m of '.'-s list of i n p u t s ; aug, w h i c h r e t u r n s t h e a v e r a g e of i t s list

of i n p u t s ; a n d pct, w h i c h r e t u r n s t h e p e r c e n t a g e of its list of b o o l e a n a r g u m e n t s w h i c h a r e t r u e O t h e r u s e r -

d e f i n e d a d j e c t i v e s m a y also be u s e d T h u s , a " d e s i r a b l e

i n s t r u c t o r " m i g h t be d e f i n e d a s a n i n s t r u c t o r w h o g a v e

a g o o d g r a d e to m o r e t h a n h a l f h i s s t u d e n t s , w h e r e a

" g o o d g r a d e " is d e f i n e d a s a g r a d e of B o r a b o v e T h e s e two a d j e c t i v e s m a y b e s p e c i f i e d a s s h o w n below

A C Q U I R I N G S E M A N T I C S F O R D E S I R A B L E I N S T R U C T O R

P R I M A R Y ? section

T A R G E T ? grade

P A T H IS: G R A D E / S T U D E N T / S E C T I O N -

F U N C T I O N S ? g o o d /id /pet

P R E D I C A T E ? > 50

A C Q U I R I N G S E M A N T I C S F O R G O O D G R A D E

P R I M A R Y ? grade

T A R G E T ? grade

P A T H IS: G R A D E

F U N C T I O N S ? val

P R E D I C A T E ? > = B

As s h o w n h e r e , P r e p r e q u e s t s t h r e e p i e c e s of

i n f o r m a t i o n for e a c h a d j e c t i v e - e n t i t y pair, n a m e l y (1)

t h e pv-/.rn.ary ( h i g h e s t - l e v e l ) a n d ~c~rget [ l o w e s t - l e v e l )

e n t i t i e s n e e d e d t o s p e c i f y t h e d e s i r e d a d j e c t i v e

m e a n i n g ; (2) a list of furtcticvts c o r r e s p o n d i n g to t h e

a r c s on t h e p a t h f r o m t h e p r i m a r y to t h e t a r g e t n o d e s ;

a n d f i n a l l y (3) a p r e d / c a t e to be a p p l i e d to t h e

n u m e r i c a l v a l u e o b t a i n e d f r o m t h e s e r i e s of f u n c t i o n

c a l l s j u s t a c q u i r e d

IV UTILIZATION O F T H E I N F O R M A T I O N A C Q U I R E D

D U R I N G P R E P R O C E S S I N G

As s h o w n in Figure i, the English-language processor of L D C achieves d o m a i n i n d e p e n d e n c e b y restricting itself to (a) a domain-independent linguistically-motivated phrase-structure g r a m m a r [6]

a n d (b) a n d the domain-specific files p r o d u c e d by the

k n o w l e d g e acquisition module

T h e simplest file is the pattern file, which captures the m o r p h o l o g y of domain-specific proper nouns, e.g the entity type "room" m a y have values

s u c h as X-238 a n d A-22, or "letter, dash digits" This information frees us f r o m having to store all possible field values in the dictionary, as s o m e systems do, or to

m a k e reference to the physical data file w h e n n e w data values are typed by the user, as other systems do

T h e domain-specific d/ctlon~ry file contains

s o m e standard terms (articles, ordinals, etc.) a n d also both root words a n d inflections for terms acquired

f r o m the user T h e sample dictionary entry ( l o n g e s t S u p e r l long ( n t m e e t i n g week)) says that " l o n g e s t " is the s u p e r l a t i v e f o r m of the adjective "long", a n d m a y occur in n o u n phrases w h o s e ' h e a d n o u n refers to entities of type meeting or week

B y having this information in the dictionary, the parser can p e r f o r m "local" compatibility checks to assure the

I User

U s e r ., > P R E P

/ /

S C A N N E R ~I P A R S E R

F i l e

f

-*1 TRANSLATOR

Augmented Phrase-Structured Grammar

Macro File

\

) RETRIEVAL i

T

Text-Edited Data

File

Figure 1 - Overview of LDC

i n t e g r i t y of a n o u n p h r a s e being built up, i.e to a s s u r e

all w o r d s in t h e p h r a s e c a n go t o g e t h e r on n o n -

s y n t a c t i c g r o u n d s This aids in d i s a m b i g u a t i o n , y e t

avoids e x p e n s i v e i n t e r a c t i o n with a s u b s e q u e n t

s e m a n t i c s module

r e l a t e d to n e g a t i o n I n t e r e s t i n g l y , m o s t m e a n i n g f u l

i n t e r p r e t a t i o n s of p h r a s e s c o n t a i n i n g " n o n " or "not"

c a n be o b t a i n e d b y i n s e r t i n g t h e r e t r i e v a l r2.odule's Not

c o m m a n d a t a n a p p r o p r i a t e p o i n t in t h e m a c r o b o d y

f o r t h e m o d i f i e r in q u e s t i o n For e x a m p l e ,

An o p p o r t u n i t y to p e r f o r m " n o n - l o c a l "

c o m p a t i b i l i t y c h e c k i n g is p r o v i d e d for by t h e eompat

file, w h i c h tells (a) t h e c a s e s t r u c t u r e of e a c h verb, i.e

which p r e p o s i t i o n s may o c c u r a n d which e n t i t y t y p e s

may fill e a c h n o u n p h r a s e "slot", a n d (b) which p a i r s of

e n t i t y t y p e s m a y be linked by e a c h p r e p o s i t i o n The

f o r m e r i n f o r m a t i o n will have b e e n a c q u i r e d d i r e c t l y

f r o m t h e u s e r , while t h e l a t t e r is p r e d i c t e d by

h e u r i s t i c s b a s e d u p o n t h e s o r t s of c o n c e p t u a l

r e l a t i o n s h i p s t h a t c a n o c c u r in t h e " l a y e r e d " d o m a i n s

of i n t e r e s t [1]

Finally, t h e m a c r o file c o n t a i n s t h e m e a n i n g s

of modifiers, r o u g h l y in t h e f o r m in which t h e y w e r e

a c q u i r e d using t h e s p e c i f i c a t i o n l a n g u a g e d i s c u s s e d in

t h e p r e v i o u s s e c t i o n Although t h i s r e q u i r e d u s to

f o r m u l a t e o u r own r e t r i e v a l q u e r y l a n g u a g e [3], having

c o m p l e x m o d i f i e r m e a n i n g s d i r e c t l y e x c e u t a b l e by t h e

r e t r i e v a l m o d u l e e n a b l e s us to avoid m a n y of t h e

p r o b l e m s t y p i c a l l y arising in t h e t r a n s l a t i o n f r o m p a r s e

s t r u c t u r e s to f o r m a l r e t r i e v a l queries• F u r t h e r m o r e ,

s o m e m o d i f i e r m e a n i n g s c a n b e derived by t h e s y s t e m

f r o m t h e m e a n i n g s of o t h e r modifiers, r a t h e r t h a n

s e p a r a t e l y a c q u i r e d f r o m t h e user• For example, if t h e

m e a n i n g of t h e a d j e c t i v e "large" h a s b e e n given by t h e

u s e r , t h e s y s t e m a u t o m a t i c a l l y p r o c e s s e s " l a r g e s t " a n d

" l a r g e r t h a n ." by a p p r o p r i a t e l y i n t e r p r e t i n g t h e

m a c r o b o d y for "large"

A p a r t i a l l y u n s o l v e d p r o b l e m in m a c r o

p r o c e s s i n g involves t h e r e s o l u t i o n of s c o p e ambiguities

s t u d e n t s who w e r e n o t failed b y R o s e n b e r g

m i g h t or m i g h t n o t be i n t e n d e d to i n c l u d e s t u d e n t s who did n o t t a k e a c o u r s e f r o m R o s e n b e r g The

r e t r i e v a l q u e r y c o m m a n d s g e n e r a t e d by t h e positive

u s a g e of "fail", as in students that R o s e n b e r g failed

w o u l d be the s e q u e n c e

i n s t r u c t o r R o s e n b e r g ;

s t u d e n t -> fail

so t h e q u e s t i o n is w h e t h e r to i n t r o d u c e "not" a t t h e

p h r a s e level

n o t i i n s t r u c t o r = R o s e n b e r g ;

s t u d e n t -> fail~

or instead at the verb level instructor = Rosenberg;

not ~student -> fail]

Our c u r r e n t s y s t e m t a k e s t h e l i t e r a l r e a d i n g , a n d t h u s

g e n e r a t e s t h e f i r s t i n t e r p r e t a t i o n given• The e x a m p l e

p o i n t s o u t t h e c l o s e r e l a t i o n s h i p b e t w e e n n e g a t i o n

s c o p e a n d t h e i m p o r t a n t p r o b l e m of " p r e s u p p o s i t i o n " ,

in t h a t t h e u s e r m a y be i n t e r e s t e d only in s t u d e n t s who

h a d a c h a n c e t o b e failed•

I BaUard, B A "Domain Class" approach to transportable

natural language processing Cogn~tio~ g~td /Yrczin

Theory, 5 (1982), 3, pp 269-287

Ballard, B a n d Lusth, J An E n g l i s h - l a n g u a g e p r o c e s s i n g

system that "learns" about n e w domains AF~PS N¢~on~

Gomputer Conference, 1983 pp 39-46

Ballard, B and Lusth, J The design of DOMINO: a

knowledge-based information retrieval processor for

office enviroments Tech Report CS-1984-2, Dept of

Computer Science, Duke University, February 1984

Ballard, B., Lusth, J and Tinkham, N LDC-I: a

transportable, knowledge-based natural language

processor for office environments A C M Tt'~ns o~ Off~ce

/ ~ - m a h ~ ~ystoma, 2 (1984), 1, pp 1-25

BaUard, B., Lusth, J and Tinkham, N Transportable

English language processing for office environments

A F ~ ' Nat~mw~ O~m~uter Conference, 1984, to appear in

the proceedings

Ballard, B and Tinkham, N A phrase-structured

grammatical formalism for transportable natural

language processing, llm~r J Cow~p~t~zt~na~ L~n~ist~cs,

to appear

Biermann, A and Ballard, B Toward natural language

computation Am~r ~ Com~ut=~mu=l ~g=iet~cs, 6

(1980), 2, pp 71-86

Lusth, J Conceptual Information Retrieval for Improved

Natural Language Processing (Master's Thesis) Dept of

Computer Science, Duke University, February 1984

Lusth, J and Ballard, B Knowledge acquisition for a

natural language processor Cue,'ere*we o~ 4~t~-ieJ

.~tetH@e~ws, Oakland University, Rochester, Michigan,

April 1983, to appear in the proceedings

I0 Bronnenberg, W., Landsbergen, S., Scha, R.,

Schoenmakers, W and van Utteren, E pHLIQA-1, a

question-answering system for data-base consultation in

natural English / W t ~ s tecA, Roy 38 (1978-79), pp

229-239 a n d 269-284

11 Codd, T Seven s t e p s to RENDEZVOUS with t h e c a s u a l

user [n Do2~ Base M¢m,o, gem, en¢, J Kimbie a n d K

Koffeman (Eds.), North-Holland, 1974

12 Codd, T R E N D E Z V O U S Version I: Aa experimental

English-language query formulation system for casual

users of relational data bases IBM Research Report

RJ2144, San Jose, Ca., 1978

13 Finin, T., Goodman, B and Tennant, H JETS: achieving

completeness through coverage and closure Int J Conf

on A r t ~ j ~ / n ~ e / ~ i g e n c e , 1979, pp 275-281

14 Harris, L U s e r - o r i e n t e d d a t a b a s e q u e r y w i t h t h e Robot

natural language system Int J M~n-M~ch~ne ~ d i e s , 9

(1977), pp 697-713

15 Harris, L The R O B O T system: natural language

processing applied to data base query A C M Nct~ion~t

C~rnference, 1978, pp 165-172

16 Hendrix, G H u m a n engineering for applied natural

language processing /n~ $ Co~f o~ 4~t~j~c~a~

~¢tott@jev~e, 1977, pp 183-191

2

3

4

5

8

7

8

9

Developing a natural language interface to complex data

ACM Tr(uts on D=t~bsse ~l/stsrrts, 3 (1978), 2, pp 105-147

18 Lehmann, H Interpretation of natural language in an

information system I B M $ _N~s Des 22 (1978), 5, pp

560-571

19 Plath, W R E Q U E S T : a natural language question-

answering system I B M J: ~ s Deo., 20 (1976), 4, pp 326-

335

20 Thompson, F and Thompson, B Practical natural language processing: the g E L system as prototype In

A d ~ v t c e s ~t C o m ~ t e r s , Vol 3, M Rubinoff a n d M Yovits, Eds., A c a d e m i c P r e s s , 1975

21 Waltz, D A n English language question answering system

for a large relational database Cowzm A C M 21 (1978), 7,

pp 526-539

22 Woods, W Semantics and quantification in natural language question answering In Advances ~,n Computers,

Vol 17, M Yovits, Ed., Academic Press, 1978

23 Woods, W., Kaplan, R and Nash-Webber, B The Lunar 3L'iencos Natural Lar~w, ge ~tfov~rn~t~n ~Jstsm:

]~¢rrt Report 2378, Bolt, Beranek and N e w m a n , Cambridge, Mass., 1972

24 Ginsparg, J A robust portable natural language data

b a s e i n t e r f a c e Cmlf on Ap'1)lied Nc~t~ral L~znguage

P r o c e s s i n g , S a n t a Munica, Ca., 1983, pp 25-30

25 Grosz, B TEAM: A transportable natural language interface system Omf o~ ~ p l i e d Nut, rat L~-tLags

Processiz~, Santa Monica, Ca., 1983, pp 39-45

28 Haas, N a n d Hendrix, G A n approach to acquiring and applying knowledge .~rst N;t Cor~ o~

.~tell~qTence, Stanford univ., Palo Alto, Ca., 1980, pp 235-

239

27 Hendrix, G a n d Lewis, W Transportable natural-language

interfaces to databases Proc 19th A ~ z ~ t M e e t ~ w of the

ACL, S t a n f o r d Univ., 1981, pp 159-165

28 Mark, W Representation and inference in the Consul system ~ t Jo'i, nt Conf on ~ct#,f~c'i~l [nteU{gence, 1981

29 Thompson, B a n d Thompson, F Introducing ASK, a

simple knowledgeable system Co~I on AppLied Natu~zt

L~tg1~zge i~rocsssing, Santa Monica, Ca., 1983, pp 17-24

30 Thompson, F and Thompson, B Shifting to a higher gear

in a natural language system Na~-na~ CornF~ter Coexistence, 1981, 657-662

31 WUczynski, D Knowledge acquisition in the Consul system Int Jo~,nt Conf on 4rt~f~c~ /ntsUwence, 1981