Báo cáo khoa học: "DEFINING NATURAL LANGUAGE GRAMMARS IN GPSG" docx

Two, to construct an explicit linguistic theory whose formal consequences are clearly and eas- ily determinable.. These 'formal consequences' include both the generative power consequenc

Eric Sven R i s t a d

M I T Artificial Intelligence L a b T h i n k i n g M a c h i n e s C o r p o r a t i o n

545 Technology S q u a r e a n d 245 F i r s t Street

C a m b r i d g e , M A 02139 C a m b r i d g e , M A 02142

1 O v e r v i e w

T h r e e c e n t r a l goals of work in t h e generalized p h r a s e struc-

t u r e g r a m m a r ( G P S G ) linguistic f r a m e w o r k , as s t a t e d in t h e

leading b o o k " G e n e r a l i z e d P h r a s e S t r u c t u r e G r a m m a r " Gaz-

d a r et al (1985) ( h e r e a f t e r G K P S ) , are: (1) to c h a r a c t e r i z e all

a n d only t h e n a t u r a l l a n g u a g e g r a m m a r s , (2) to a l g o r i t h m i c a l l y

d e t e r m i n e m e m b e r s h i p a n d g e n e r a t i v e power c o n s e q u e n c e s of

G P S G s , a n d (3) to e m b o d y t h e u n i v e r s a l i s m of n a t u r a l lan-

g u a g e entirely in t h e f o r m a l s y s t e m , r a t h e r t h a n by s t a t e m e n t s

m a d e in it 1

T h e s e p a g e s f o r m a l l y consider w h e t h e r G P S G ' s weak c o n t e x t -

free g e n e r a t i v e power (wcfgp) will allow it to achieve t h e t h r e e

goals T h e centerpiece of t h i s p a p e r is a p r o o f t h a t it is u n d e -

cidable w h e t h e r a n a r b i t r a r y G P S G g e n e r a t e s t h e n o n n a t u r a l

l a n g u a g e ~ ' O n t h e b a s i s of t h i s r e s u l t , I a r g u e t h a t G P S G

fails to define t h e n a t u r a l l a n g u a g e g r a m m a r s , a n d t h a t t h e gen-

erative power c o n s e q u e n c e s of t h e G P S G f r a m e w o r k c a n n o t be

a l g o r i t h m i c a l l y d e t e r m i n e d , c o n t r a r y to goals one a n d two 2 In

t h e process, I e x a m i n e t h e l i n g u i s t i c u n i v e r s a l i s m of t h e G P S G

f o r m a l s y s t e m a n d a r g u e t h a t G P S G s c a n describe a n infinite

class of n o n n a t u r a l c o n t e x t - f r e e l a n g u a g e s T h e p a p e r concludes

w i t h a brief d i a g n o s i s of t h e r e s u l t a n d s u g g e s t s t h a t t h e p r o b l e m

m i g h t be m e t by a b a n d o n i n g t h e weak c o n t e x t - f r e e g e n e r a t i v e

power f r a m e w o r k a n d a s s u m i n g s u b s t a n t i v e c o n s t r a i n t s

1 1 T h e S t r u c t u r e o f G P S G T h e o r y

A generalized p h r a s e s t r u c t u r e g r a m m a r c o n t a i n s five l a n g u a g e -

p a r t i c u l a r c o m p o n e n t s ( i m m e d i a t e d o m i n a n c e (ID) rules, m e t a -

rules, linear precedence (LP) s t a t e m e n t s , f e a t u r e co-occurrence

IGKPS clearly outline their goals One, uto arrive at a constrained met-

alanguage capable of defining the grammars of natural languages, but not

the grammar of, say, the set of prime numbers2(p.4) Two, to construct

an explicit linguistic theory whose formal consequences are clearly and eas-

ily determinable These 'formal consequences' include both the generative

power consequences demanded by the first goal and membership determi-

nation: GPSG regards languages "as collections whose membership is def-

initely and precisely specifiable."(p.1) Three, to define a linguistic theory

where ~lhe universalism [of natural language] is, ultimately, intended to be

entirely embodied in the formal system, not ezpressed by statements made in

it.'(p.4, my emphasis)

2The proof technique make use of invalid computations, and the actual

GPSG constructed is so simple, so similar to the GPSGs proposed for actual

natural languages, and so flexible in its exact formulation that the method of

proof suggests there may be no simple reformulations of GPSG that avoid

this problem The proof also suggests that it is impossible in principle

to algorithmically determine whether linguistic theories based on a wcfgp

framework (e.g GPSG) actually define the natural language grammars

r e s t r i c t i o n s ( F C R s ) , a n d f e a t u r e specification d e f a u l t s ( F S D s ) )

a n d four u n i v e r s a l c o m p o n e n t s : a t h e o r y of s y n t a c t i c f e a t u r e s , principles of u n i v e r s a l f e a t u r e i n s t a n t i a t i o n , principles of s e m a n - tic i n t e r p r e t a t i o n , a n d f o r m a l r e l a t i o n s h i p s a m o n g v a r i o u s c o m -

p o n e n t s of t h e g r a m m a r 3

T h e set of ID rules o b t a i n e d by t a k i n g t h e finite closure

of t h e m e t a r u l e s on t h e ID rules is m a p p e d into local p h r a s e

s t r u c t u r e trees, s u b j e c t to principles of u n i v e r s a l f e a t u r e i n s t a n -

t i a t i o n , F S D s , F C R s , a n d L P s t a t e m e n t s Finally, t h e s e local

t r e e s are a s s e m b l e d to f o r m p h r a s e s t r u c t u r e trees, w h i c h are

t e r m m a t e d by lexical e l e m e n t s

T h e essence of G P S G is t h e c o n s t r a i n e d m a p p i n g of ID rules into local trees T h e c o n s t r a i n t s of G P S G t h e o r y s u b d i v i d e

i n t o a b s o l u t e c o n s t r a i n t s on local t r e e s (due to F C R s a n d L P -

s t a t e m e n t s ) a n d relative c o n s t r a i n t s on t h e rule to local t r e e

m a p p i n g ( s t e m m i n g f r o m F S D s a n d u n i v e r s a l f e a t u r e i n s t a n -

t i a t i o n ) T h e a b s o l u t e c o n s t r a i n t s are all l a n g u a g e - p a r t i c u l a r ,

a n d c o n s e q u e n t l y n o t i n h e r e n t in t h e f o r m a l G P S G f r a m e w o r k Similarly, t h e relative c o n s t r a i n t s , of which only u n i v e r s a l in-

s t a n t i a t i o n is n o t explicitly l a n g u a g e - p a r t i c u l a r , do n o t a p p l y

to fully specified ID rules a n d c o n s e q u e n t l y are n o t s t r o n g l y in-

h e r e n t in t h e G P S G f r a m e w o r k either 4 In s u m m a r y , G P S G local trees are only as constrained as I D rules are: that is, not

a t all

T h e only c o n s t r a i n t s t r o n g l y i n h e r e n t in G P S G t h e o r y ( w h e n

c o m p a r e d to c o n t e x t - f r e e g r a m m a r s ( C F G s ) ) is finite f e a t u r e closure, w h i c h l i m i t s t h e n u m b e r of G P S G n o n t e r m i n a l s y m b o l s

to be finite a n d b o u n d e d S

1 2 A N o n n a t u r a l G P S G

C o n s i d e r t h e exceedingly s i m p l e G P S G for t h e n o n n a t u r a l lan-

g u a g e Z*, c o n s i s t i n g solely of t h e two ID rules SThis work is based on current GPSG theory as presented in GKPS The reader is urged to consult that work for a formal presentation and thorough exposition of current GPSG theory

4I use "strongly inherent" to mean ~unavoidable by virtue of the formal framework." Note that the use of problematic feature specifications in universal feature instantiation means that this constraint is dependent on other, parochial, components (e.g FCRs) Appropriate choice of FCRs

or ID rules will abrogate universal feature inetantiation, thus rendering it implicitly language particular too

5This formal constraint is extremely weak, however, since the theory

of syntactic features licenses more than 10 T M syntactic categories See Ristad, E.S (1986), ~Computational Complexity of Current GPSG Theory ~

in these proceedings for a discussion

S -* { } , H I E

This G P S G generates local trees with all possible subcategoriza-

tion specifications the SUBCAT feature m a y assume any value

in the non-head daughter of the first ID rule, and S generates

the nonnatural language ~*

This exhibit is inconclusive, however W e have only shown

that G K P S and not G P S G have failed to achieve the first

goal of G P S G theory The exhibition leaves open the possibility

of trivially reformalizing G P S G or imposing ad-hoc constraints

on the theory such that I will no longer be able to personally

construct a G P S G for Z*

2 U n d e c i d a b i l i t y a n d G e n e r a t i v e P o w e r

in G P S G

T h a t " = Z*?" is undecidable for a r b i t r a r y context-free gram-

m a r s is a well-known result in t h e formal language literature

(see H o p c r a f t and Ullman(1979:201-203)) T h e s t a n d a r d p r o o f

is to c o n s t r u c t a P D A t h a t accepts all invalid c o m p u t a t i o n s of

a T M M From this P D A an equivalent C F G G is directly con-

structible Thus, L(G) = ~ ' if and only if all computations of

M are invalid, i.e L ( M ) = 0 T h e l a t t e r p r o b l e m is undecid-

able, so t h e former must be also

No such reduction is possible for a p r o o f t h a t " ~*?" is

undecidable for a r b i t r a r y G P S G s In t h e above reduction, t h e

n u m b e r of n o n t e r m i n a l s in G is a function of t h e size of t h e

simulated T M M G P S G s , however, have a b o u n d e d n u m b e r

of n o n t e r m i n a l symbols, a n d as discussed above, t h a t is t h e

essential difference between C F G s and G P S G s

Only weak generative power is of interest for t h e follow-

ing proof, and t h e formal G P S G c o n s t r a i n t s on weak generative

power are trivially abrogated For example, exhaustive c o n s t a n t

partial ordering ( E C P O ) - - which is a c o n s t r a i n t on s t r o n g gen-

erative capacity - - can be done away w i t h for all i n t e n t s and

p u r p o s e s by n o n t e r m i n a l r e n a m i n g , and c o n s t r a i n t s arising from

principles of universal feature i n s t a n t i a t i o n d o n ' t apply to fully

i n s t a n t i a t e d ID rules

First, a proof t h a t " ~*?" is undecidable for context-free

g r a m m a r s w i t h a very small n u m b e r of t e r m i n a l and nonter-

minal s y m b o l s is sketched Following t h e p r o o f for C F G s , t h e

equivalent p r o o f for G P S G s is outlined

2 1 O u t l i n e o f a P r o o f f o r S m a l l C F G s

Let L(z,~ ) be t h e class of context-free g r a m m a r s with at least

x n o n t e r m i n a l and y t e r m i n a l symbols I now sketch a p r o o f

t h a t it is undecidable of an a r b i t r a r y C F G G c L(~,v ) w h e t h e r

L(G) = ~* for some x, y g r e a t e r t h a n fixed lower b o u n d s T h e

actual c o n s t r u c t i o n details are of no obvious m a t h e m a t i c a l or

pedagogical interest, and will n o t be included T h e idea is

to directly c o n s t r u c t a C F G to generate t h e invalid c o m p u t a -

tions of t h e Universal Turing Machine ( U T M ) This g r a m m a r

will be small if t h e U T M is small T h e "smallest U T M " of Minsky(1967:276-281) has seven s t a t e s and a four s y m b o l t a p e

a l p h a b e t , for a s t a t e - s y m b o l p r o d u c t of 28 (!) Hence, it is n o t surprising t h a t t h e "smallest GUT M" t h a t generates t h e invalid

c o m p u t a t i o n s of t h e U T M has seventeen n o n t e r m i n a l s and two terminals

Observe t h a t if a string w is an invalid c o m p u t a t i o n of t h e universal Turing machine M = (Q,]E, r , 5, q0, B, F ) on i n p u t x,

t h e n one of t h e following conditions must hold

1 w has a "syntactic error," t h a t is, w is n o t of t h e form

X l ~ g 2 ~ ' ' " ~ X m ~ , where each xi is an i n s t a n t a n e o u s de- scription (ID) of M Therefore, some xl is n o t an ID of

M

2 xl is n o t initial; t h a t is, Xl ~ q0~*

3 x,~ is n o t final; t h a t is xm ~ r * f F *

4 x~ F- M (X~+l) R is false for some o d d i

5 (xi) R ~-*M Xi+l is false for some even i

S t r a i g h t f o r w a r d c o n s t r u c t i o n of GVTM will result in a C F G containing on t h e order of twenty or thirty n o n t e r m i n a l s and

at least fifteen t e r m i n a l s (one for each U T M s t a t e a n d t a p e symbol, one for t h e b l a n k - t a p e symbol, and one for t h e instan-

t a n e o u s description s e p a r a t o r " ~ ' ) T h e n t h e s u b g r a m m a r s which ensure t h a t (xi) R ~-~'M xi+l is false for some even i and

t h a t x~ ~ ~M (xi+l) R is false for some odd i may be cleverly combined so t h a t n o n t e r m i n a l s encode more i n f o r m a t i o n , and

SO o n

T h e final trick, due to A l b e r t Meyer, reduces t h e t e r m i n a l s

to 2 at t h e cost of a lone n o n t e r m i n a l by encoding t h e n ter- minals as log n k-bit words over t h e new t e r m i n a l a l p h a b e t {0, 1}, and a d d i n g some rules to ensure t h a t t h e final g r a m m a r could generate ]E* and n o t ( ~ 4 ) T h e p r o d u c t i o n s

N4 * OL41L4 I OOL4 I 01L~ I l l L 4 I

are a d d e d to t h e converted C F G GtVTM, which generates a language of t h e form

L4 * oooo I OOOl ] OOlO I I E I L4L4

Where L4 generates all s y m b o l s of length 4, and N4 gener- ates all strings n o t of length 0 rood k, where k = 4 (i.e all strings of length 1,2,3 m o d 4) Deeper consideration of t h e ac- tual GUTM reveals t h a t t h e N4 n o n t e r m i n a l is also eliminable Note t h a t all t h e preceding efforts to reduce t h e n u m b e r of

n o n t e r m i n a l s and t e r m i n a l s increase t h e n u m b e r of context-free

p r o d u c t i o n s This s y m b o l - p r o d u c t i o n tradeoff becomes clearer when one actually c o n s t r u c t s GUTM

Suppose t h e distinguished s t a r t symbol for GVTM is SUTM

Then we form a new C F G consisting Of all p r o d u c t i o n s of t h e form

S -* {Q - q0}{E p - (M}}{N4 U L4}

and the one production

S -* SUT M

where (M} is the length p encoding of an arbitrary T M M ,

and L4, N4 are as defined above

This ensures t h a t strings whose prefix is "q0(M)" will be

generated starting from S if and only if they are generated start-

ing from SVrM: t h a t is, they are invalid computations of the

U T M on M

2 2 S o m e D e t a i l s f o r Lc~,v ) a n d G P S G

Let the nonterminal symbols F, Q, and E in the following CFG

portion generate the obvious terminal symbols corresponding to

the equivalent U T M sets B is the terminal blank symbol

Then, the following sketched CF productions generate the

IDs of M such t h a t zi ~ -~M (Xi+l) R is false for some odd i

The $4 and $5 nonterminals are used to locate the even and

odd i IDs zi of w Sok generates the language {F t_J # } *

s4 -~ r s 4 I # s 5 I #SoddSok

S5 -~ r s 5 I # s 4 I #s,.,.Sok

$odd -~ S l #

Sl ~ r s ~ r I s2 I s 6 l s7

Ss -~ r s ~ [ r s 3

s7 - , s r r I s s r

$2 * EaESzFbF

where a # b, b o t h in E

aqbSs{r s - cap} if 8(q,b) = (p,c,L)

S2 * a q B # B { r s - pca} if 8(q, B) = (p,c, R)

s3 - r s ~ r I Q B # B r r I Z B # B r

$1 and $2 must generate a false transition for odd i, while Sz

need not generate a false transition and is used to pad out the

IDs of w The nonterminals Se,S7 accept IDs with improperly

different tape lengths The first $2 production accepts transi-

tions where the tape contents differ in a bad place, the second $2

production accepts invalid transitions other t h a n at the end of

the tape, and the t h i r d $2 accepts invalid end of the tape transi-

tions Note t h a t the last two $2 productions are actually classes

of productions, one for each string in F 3 - p c a , F 3 - cap,

The G P S G for " = E*?" is constructed in a virtually iden-

tical fashion Recall t h a t the G P S G formal framework does not

bar us from constructing a g r a m m a r equivalent to the CFG j u s t

presented The ID rules used in the construction will be fully

specified so as to defeat universal feature instantiation, and the construction will use nonterminal renaming to avoid ECPO Let the G P S G category C be fully specified for all features (the actual values d o n ' t m a t t e r ) with the exception of, say, the binary features GER, NEG NULL and POSS Arrange those four features in some canonical order, and let binary strings of length four represent the values assigned to those features in a given category For example, C[0100] represents the category C with the additional specifications ([-GER], [+NEG], [-NULL], [- POSS]) We replace Soda by C[0000], S1 by C[0001], $2 by

C[0010], $3 by C[0011], $6 by C[0100], and Sr by C[0101] The nonterminal r is replaced by three symbols of the form C[1 l xx], one for each linear precedence r conforms too Similarly, Y is replaced by two symbols of the form C[100x] The ID rules, in the same order as the CF productions above (with a portion of the necessary LP s t a t e m e n t s ) are:

c[oooo] -~ c [ o o o l ] # C[0001] -* C[llO0]C[O001]C[llO1]{C[O010][C[0100]]C[OIO1]

C[OIO0] * C[llO0]C[OIO0] I C[llO0]C[O011 ]

c I o l o l ] -~ C[OlOl]C[llOlltC[oonlc[llOl]

where a ~ b, b o t h in E C[0010] ~ a q b C [ 0 0 u ] { r ~ - pca} i f 6 ( q , b ) = ( p , c , R )

C[0010] * a q B # B { r s - p c a } if 8(q, B) = (p, c, R)

a q B # B { r 3 - cap} if 8 ( q , B ) = (p,c,L)

C[0011] -~ C[1100]C[0011]C[1101] ]

QB#BC[llO0]C[ll01] I

C [1000] B # B C [1100]

C[ll00] < C[O001],C[O011],C[OIO0],C[OIO1] < C[ll01] C[I000] < a < C[1001] < C[0011] < C[1110]

While the sketched ID rules are not valid G P S G rules, j u s t

as the sketched context-free productions were not the valid com- ponents of a context-free g r a m m a r , a valid G P S G can be con- structed in a straightforward and obvious m a n n e r from the sketched ID rules There would be no metarules, FCRs or FSDs

in the actual g r a m m a r The last comment to be made is t h a t in the actual GUTM,

only the n u m b e r of productions is a function of the size of the UTM The U T M is used only as a convincing crutch - - i.e not

at all Only a small, fixed number of nonterminals are needed to

construct a CFG for the invalid computations of any arbitrary Turing Machine

3 I n t e r p r e t i n g t h e R e s u l t

The preceding pages have shown t h a t the extremely simple non-

n a t u r a l language ~* is generated by a G P S G , as is the more complex language L l c consisting of the invalid computations of

an arbitrary Turing machine on an a r b i t r a r y input Because

4 2

L l c

there is no algorithmic way of knowing whether any given G P S G

generates a natural language or an unnatural one So, for ex-

a m p l e , no a l g o r i t h m c a n tell u s w h e t h e r t h e E n g l i s h G P S G of

G K P S really g e n e r a t e s E n g l i s h or ~ *

T h e r e s u l t s u g g e s t s t h a t goals 1, 2, 3 a n d t h e c o n t e x t - f r e e

f r a m e w o r k conflict w i t h each other Weak c o n t e x t - f r e e gener-

ative power allows b o t h ~* a n d L i e , y e t by goal 1 we m u s t

exclude n o n n a t u r a l l a n g u a g e s Goal 2 d e m a n d s it be possi-

ble to a l g o r i t h m i c a l l y d e t e r m i n e w h e t h e r a given G P S G gener-

a t e s a desired l a n g u a g e or n o t , y e t t h i s c a n n o t be d o n e in t h e

c o n t e x t - f r e e f r a m e w o r k Lastly, goal 3 requires t h a t all n o n n a t -

u r a l l a n g u a g e s be excluded on t h e b a s i s of t h e f o r m a l s y s t e m

alone, b u t t h i s looks to be i m p o s s i b l e given t h e o t h e r two goals,

t h e a d o p t e d f r a m e w o r k , a n d t h e t e c h n i c a l v a g u e n e s s of " n a t u r a l

l a n g u a g e g r a m m a r "

T h e p r o b l e m c a n be m e t in p a r t by a b a n d o n i n g t h e c o n t e x t -

free f r a m e w o r k O t h e r a u t h o r s h a v e a r g u e d t h a t n a t u r a l lan-

g u a g e is n o t context-free, a n d h e r e we a r g u e t h a t t h e G P S G

t h e o r y of G K P S c a n c h a r a c t e r i z e c o n t e x t - f r e e l a n g u a g e s t h a t

are t o o simple or trivial to be n a t u r a l , e.g any finite or reg-

u l a r l a n g u a g e 6 T h e c o n t e x t - f r e e f r a m e w o r k is b o t h too weak

a n d t o o s t r o n g - - it includes n o n n a t u r a l l a n g u a g e s a n d excludes

n a t u r a l ones Moreover, C F L ' s h a v e t h e w r o n g f o r m a l p r o p e r -

ties entirely: n a t u r a l l a n g u a g e is surely n o t closed u n d e r u n i o n ,

c o n c a t e n a t i o n , Kleene closure, s u b s t i t u t i o n , or i n t e r s e c t i o n w i t h

r e g u l a r sets! 7 In s h o r t , t h e c o n t e x t - f r e e f r a m e w o r k is t h e w r o n g

i d e a completely, a n d this is to be expected: w h y s h o u l d t h e ar-

b i t r a r y g e n e r a t i v e power classifications of m a t h e m a t i c s (formal

l a n g u a g e t h e o r y ) be at all relevant to biology ( h u m a n l a n g u a g e ) ?

Goal 2, t h a t t h e n a t u r a l n e s s of g r a m m a r s p o s t u l a t e d b y

l i n g u i s t i c t h e o r y be decidable, a n d to a lesser e x t e n t goal 3,

are of d u b i o u s merit In m y view, s u b s t a n t i v e c o n s t r a i n t s aris-

ing f r o m psychology, biology or even p h y s i c s m a y be freely in-

voked, w i t h a c o r r e s p o n d i n g c h a n g e in t h e m e a n i n g of " n a t u r a l

l a n g u a g e g r a m m a r " f r o m " m e n t a l l y - r e p r e s e n t a b l e g r a m m a r " to

s o m e t h i n g like "easily l e a r n a b l e a n d s p e a k a b l e m e n t a l l y - r e p r e s e n t a b ] £

g r a m m a r " T h e r e is no a priori reason or empirical evidence to

s u g g e s t t h a t t h e class of m e n t a l l y r e p r e s e n t a b l e g r a m m a r s is n o t

f a n t a s t i c a l l y c o m p l e x , m a y b e n o t even decidable, s

O n e p r o m i s i n g r e s t r i c t i o n in t h i s r e g a r d , w h i c h if p r o p e r l y

f o r m u l a t e d would alleviate G P S G ' s a c t u a l a n d f o r m a l inability

to c h a r a c t e r i z e only t h e n a t u r a l l a n g u a g e g r a m m a r s , is s t r o n g

n a t i v i s m - - t h e restrictive t h e o r y t h a t t h e class of n a t u r a l lan-

eWhile 'natural language grammar' is not defined precisely, recent work

has demonstrated empirically that natural language is not context-free, and

therefore GPSG theory will not be able to characterize all the human lan-

guage grammars See, for example, Higglnbotham(1984), Shieber(1985),

and Culy(1985) For counterarguments, see Pullum(1985) Nash(1980),

chapter 5, discusses the impossibility of accounting for free word order lan-

guages (e.g Warlplrl) using ID/LP grammars I focus on the goal of

characterizing only the natural language grammars in this paper

VThe finite, bounded number of nonterminals allowed in GPSG theory

plays a linguistic role in this regard, because the direct consequence of finite

feature closure is that GPSG languages are not truly closed under union,

concatenation, or substitution

8See Chomsky(1980:120) for a discussion

g u a g e s is finite T h i s r e s t r i c t i o n is well m o t i v a t e d b o t h by t h e issues raised here a n d by o t h e r e m p i r i c a l c o n s i d e r a t i o n s ° T h e

r e s t r i c t i o n , w h i c h m a y be s u b s t a n t i v e or purely f o r m a l , is a for-

m a l a t t a c k on t h e h e a r t of t h e result: t h e t h e o r y of undecidabil- ity is c o n c e r n e d w i t h t h e e x i s t e n c e or n o n e x i s t e n c e of a l g o r i t h m s for s o l v i n g p r o b l e m s w i t h a n infinity of i n s t a n c e s F u r t h e r m o r e ,

t h e r e s t r i c t i o n m a y be empirically plausible, l°'xl

T h e a u t h o r does n o t h a v e a clear i d e a how G P S G m i g h t be

r e s t r i c t e d in t h i s m a n n e r , a n d m e r e l y s u g g e s t s s t r o n g n a t i v i s m

as a w e l l - m o t i v a t e d direction for f u t u r e G P S G research

A c k n o w l e d g m e n t s T h e a u t h o r is i n d e b t e d to Ed B a r t o n ,

R o b e r t Berwick, N o a m C h o m s k y , J i m H i g g i n b o t h a m , R i c h a r d

L a r s o n , A l b e r t Meyer, a n d D a v i d W a l t z for a s s i s t a n c e in writ- ing t h i s p a p e r , a n d to t h e M I T Artificial Intelligence L a b a n d

T h i n k i n g M a c h i n e s C o r p o r a t i o n for s u p p o r t i n g t h i s research

4 R e f e r e n c e s

C h o m s k y , N (1980) Rules and Representations New York:

C o l u m b i a U n i v e r s i t y Press

G a s d a r , G., E Klein, G P u l l u m , a n d I Sag (1985) General- ized Phrase Structure Grammar Oxford, E n g l a n d : Basil Blackwell

H i g g i n b o t h a m , J (1984) " E n g l i s h is n o t a C o n t e x t - F r e e Lan- guage," Linguistic Inquiry 15: 119-126

~Note that invoking finiteness here is technically different from hiding intractability with finiteness Finiteness is the correct generalization here, because we are interested in whether GPSG generates nonnatural languages

or not, and not in the computational cost of determining the generative capacity of an arbitrary GPSG A finiteness restriction for the purposes of computational complexity is invalid because it prevents us from properly using the tools of complexity theory to study the computational complexity

of a problem

l°See Osherson et el (1984) for an exposition of strong nativism and related issues The theory of strong nativism can be derived in formal learning theory from three empirically motivated axioms: (1) the ability of language learners to learn in noisy environments, (2) language learner mem- ory limitations (e.g inability to remember long-past utterances), and (3) the likelihood that language learners choose simple grammars over more complex, equivalent ones These formal results are weaker empirically than they might appear at first glance: the equivalence of Ulearned~ gram- mars is measured using only weak generative capacity, ignoring uniformity considerations

llAn alternate substantive constraint, suggested by Higginbotham (per- sonal communication) and not explored here, is to require natural language grammars to generate non-dense languages Let the density of a class of lan- guages be an upper bound (across all languages in the class) on the ratio

of grammatical utterances to grammatical and ungrammatical utterances,

in terms of utterance lengths If the density of natural languages was small

or even logarithmic in utterance length, as one might expect, and a decid- able property of the reformulated GPSG's, then undecidability of "= ]~*?n would no longer reflect on the decidability of whether the GPSG framework characterized all and only the natural language grammars The exact spec- ification of this density constraint is tricky because unit density decides

"= IE'?" , and therefore density measurements cannot be too accurate Furthermore, ~* and L i c can be buried in other languages, i.e concate- nated onto the end of an arbitrary (finite or infinite) language, weakening the accuracy and relevance of density measurements

Hopcroft, J.E., and J.D Ullman (1979) Introduction to Au-

M.A: Addiso~a- Wesley

Minsky, M (1967) Computation: Finite and Infinite Machines

Englewood Cliffs, N.J: Prentice-Hall

Nash, D (1980) "Topics in Warlpiri Grammars," M.I.T De- partment of Linguistics and Philosophy Ph.D dissertation, Cambridge

Osherson, D., M Stob, and S Weinstein (1984) "Learning The- ory and Natural Language," Cognition 17: 1-28

Pullum, G.K (1985) "On Two Recent Attempts to Show that English is Not a CFL," Computational Linguistics 10: 182-

186

Shieber, S.M (1985) "Evidence Against the Context-Freeness of Natural Language," Linguistics and Philosophy 8: 333-344

44