Chinese computational linguistics and natural language processing based on naturally annotated big data

Arabic Collocation Extraction Based on Hybrid MethodsAlaa Mamdouh Akef, Yingying Wang, and Erhong Yang& School of Information Science, Beijing Language and Culture University, Beijing 10

Maosong Sun · Xiaojie Wang

123

16th China National Conference, CCL 2017

and 5th International Symposium, NLP-NABD 2017

Nanjing, China, October 13–15, 2017, Proceedings

Chinese Computational Linguistics

and Natural Language Processing

Based on Naturally Annotated Big Data

Lecture Notes in Arti ficial Intelligence 10565

Subseries of Lecture Notes in Computer Science

LNAI Series Editors

DFKI and Saarland University, Saarbrücken, Germany

LNAI Founding Series Editor

Joerg Siekmann

DFKI and Saarland University, Saarbrücken, Germany

More information about this series at http://www.springer.com/series/1244

Maosong Sun • Xiaojie Wang

Chinese Computational Linguistics

and Natural Language Processing

Based on Naturally Annotated Big Data

16th China National Conference, CCL 2017

and 5th International Symposium, NLP-NABD 2017

Proceedings

123

ChinaDeyi XiongSoochow UniversitySuzhou

China

ISSN 0302-9743 ISSN 1611-3349 (electronic)

Lecture Notes in Artificial Intelligence

ISBN 978-3-319-69004-9 ISBN 978-3-319-69005-6 (eBook)

https://doi.org/10.1007/978-3-319-69005-6

Library of Congress Control Number: 2017956073

© Springer International Publishing AG 2017

This work is subject to copyright All rights are reserved by the Publisher, whether the whole or part of the

broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed.

The use of general descriptive names, registered names, trademarks, service marks, etc in this publication

protective laws and regulations and therefore free for general use.

The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made The publisher remains neutral with regard to jurisdictional claims in

Printed on acid-free paper

This Springer imprint is published by Springer Nature

The registered company is Springer International Publishing AG

The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Welcome to the proceedings of the 16th China National Conference on ComputationalLinguistics (16th CCL) and the 5th International Symposium on Natural LanguageProcessing Based on Naturally Annotated Big Data (5th NLP-NABD) The conferenceand symposium were hosted by Nanjing Normal University located in Nanjing City,Jiangsu Province, China

CCL is an annual conference (bi-annual before 2013) that started in 1991 It is theflagship conference of the Chinese Information Processing Society of China (CIPS),which is the largest NLP scholar and expert community in China CCL is a premiernation-wide forum for disseminating new scholarly and technological work in com-putational linguistics, with a major emphasis on computer processing of the languages

in China such as Mandarin, Tibetan, Mongolian, and Uyghur

Affiliated with the 16th CCL, the 5th International Symposium on Natural LanguageProcessing Based on Naturally Annotated Big Data (NLP-NABD) covered all the NLPtopics, with particular focus on methodologies and techniques relating to naturallyannotated big data In contrast to manually annotated data such as treebanks that areconstructed for specific NLP tasks, naturally annotated data come into existencethrough users’ normal activities, such as writing, conversation, and interactions on theWeb Although the original purposes of these data typically were unrelated to NLP,they can nonetheless be purposefully exploited by computational linguists to acquirelinguistic knowledge For example, punctuation marks in Chinese text can help wordboundaries identification, social tags in social media can provide signals for keywordextraction, and categories listed in Wikipedia can benefit text classification The naturalannotation can be explicit, as in the aforementioned examples, or implicit, as in Hearstpatterns (e.g.,“Beijing and other cities” implies “Beijing is a city”) This symposiumfocuses on numerous research challenges ranging from very-large-scale unsupervised/semi-supervised machine leaning (deep learning, for instance) of naturally annotatedbig data to integration of the learned resources and models with existing handcrafted

“core” resources and “core” language computing models NLP-NABD 2017 wassupported by the National Key Basic Research Program of China (i.e.,“973” Program)

“Theory and Methods for Cyber-Physical-Human Space Oriented Web ChineseInformation Processing” under grant no 2014CB340500 and the Major Project of theNational Social Science Foundation of China under grant no 13&ZD190

The Program Committee selected 108 papers (69 Chinese papers and 39 Englishpapers) out of 272 submissions from China, Hong Kong (region), Singapore, and theUSA for publication The acceptance rate is 39.7% The 39 English papers cover thefollowing topics:

– Fundamental Theory and Methods of Computational Linguistics (6)

– Machine Translation (2)

– Knowledge Graph and Information Extraction (9)

– Language Resource and Evaluation (3)

– Information Retrieval and Question Answering (6)

– Text Classification and Summarization (4)

– Social Computing and Sentiment Analysis (1)

– NLP Applications (4)

– Minority Language Information Processing (4)

The final program for the 16th CCL and the 5th NLP-NABD was the result of agreat deal of work by many dedicated colleagues We want to thank,first of all, theauthors who submitted their papers, and thus contributed to the creation of thehigh-quality program that allowed us to look forward to an exciting joint conference

We are deeply indebted to all the Program Committee members for providinghigh-quality and insightful reviews under a tight schedule We are extremely grateful tothe sponsors of the conference Finally, we extend a special word of thanks to all thecolleagues of the Organizing Committee and secretariat for their hard work in orga-nizing the conference, and to Springer for their assistance in publishing the proceedings

in due time

We thank the Program and Organizing Committees for helping to make the ference successful, and we hope all the participants enjoyed a memorable visit toNanjing, a historical and beautiful city in East China

Ting LiuGuodong ZhouXiaojie WangBaobao ChangBenjamin K Tsou

Ming Li

General Chairs

Nanning Zheng Xi’an Jiaotong University, China

Guangnan Ni Institute of Computing Technology,

Chinese Academy of Sciences, China

Program Committee

16th CCL Program Committee Chairs

16th CCL Program Committee Co-chairs

Xiaojie Wang Beijing University of Posts and Telecommunications, China

16th CCL and 5th NLP-NABD Program Committee Area Chairs

Linguistics and Cognitive Science

Fundamental Theory and Methods of Computational Linguistics

Information Retrieval and Question Answering

Text Classification and Summarization

Tingting He Central China Normal University, China

Knowledge Graph and Information Extraction

Kang Liu Institute of Automation, Chinese Academy of Sciences, China

Machine Translation

Adria De Gispert University of Cambridge, UK

Minority Language Information Processing

Aishan Wumaier Xinjiang University, China

Language Resource and Evaluation

Social Computing and Sentiment Analysis

NLP Applications

Ruifeng Xu Harbin Institute of Technology Shenzhen Graduate School,

ChinaYue Zhang Singapore University of Technology and Design, Singapore

16th CCL Technical Committee Members

Dongfeng Cai Shenyang Aerospace University, China

Xueqi Cheng Institute of Computing Technology, CAS, China

Alexander Gelbukh National Polytechnic Institute, Mexico

Josef van Genabith Dublin City University, Ireland

Randy Goebel University of Alberta, Canada

Tingting He Central China Normal University, China

Isahara Hitoshi Toyohashi University of Technology, Japan

Heyan Huang Beijing Polytechnic University, China

Xuanjing Huang Fudan University, China

Turgen Ibrahim Xinjiang University, China

VIII Organization

Shiyong Kang Ludong University, China

Sadao Kurohashi Kyoto University, Japan

Institute of Computing Technology, CAS, China

Wolfgang Menzel University of Hamburg, Germany

Jian-Yun Nie University of Montreal, Canada

Yanqiu Shao Beijing Language and Culture University, China

Benjamin Ka Yin

Tsou

City University of Hong Kong, SAR China

Erhong Yang Beijing Language and Culture University, China

Tianfang Yao Shanghai Jiaotong University, China

Quan Zhang Institute of Acoustics, CAS, China

5th NLP-NABD Program Committee Chairs

Benjamin K Tsou City University of Hong Kong, SAR China

5th NLP-NABD Technical Committee Members

Alexander Gelbukh National Polytechnic Institute, Mexico

Josef van Genabith Dublin City University, Ireland

Randy Goebel University of Alberta, Canada

Organization IX

Isahara Hitoshi Toyohashi University of Technology, Japan

Xuanjing Huang Fudan University, China

Sadao Kurohashi Kyoto University, Japan

Hongfei Lin Dalian Polytechnic University, China

Institute of Computing, CAS, China

Wolfgang Menzel University of Hamburg, Germany

Hwee Tou Ng National University of Singapore, Singapore

Jian-Yun Nie University of Montreal, Canada

Benjamin Ka Yin

Tsou

City University of Hong Kong, SAR China

Local Organization Committee Chair

Evaluation Chairs

Publications Chairs

Erhong Yang Beijing Language and Culture University, China

X Organization

Publicity Chairs

Tutorials Chairs

Sponsorship Chairs

Wanxiang Che Harbin Institute of Technology, China

System Demonstration Chairs

Xianpei Han Institute of Software, Chinese Academy of Sciences, China

16th CCL and 5th NLP-NABD Organizers

Chinese Information Processing Society of China

Tsinghua University

Organization XI

Nanjing Normal University

Publishers

Journal of Chinese Information Processing Science China

Lecture Notes in Artificial Intelligence

Springer

Journal of Tsinghua University(Science and Technology)XII Organization

Fundamental Theory and Methods of Computational Linguistics

Arabic Collocation Extraction Based on Hybrid Methods 3Alaa Mamdouh Akef, Yingying Wang, and Erhong Yang

Employing Auto-annotated Data for Person Name Recognition

in Judgment Documents 13Limin Wang, Qian Yan, Shoushan Li, and Guodong Zhou

Closed-Set Chinese Word Segmentation Based on Convolutional

Neural Network Model 24Zhipeng Xie

Improving Word Embeddings for Low Frequency Words

by Pseudo Contexts 37Fang Li and Xiaojie Wang

A Pipelined Pre-training Algorithm for DBNs 48Zhiqiang Ma, Tuya Li, Shuangtao Yang, and Li Zhang

Enhancing LSTM-based Word Segmentation Using Unlabeled Data 60

Bo Zheng, Wanxiang Che, Jiang Guo, and Ting Liu

Machine Translation and Multilingual Information Processing

Context Sensitive Word Deletion Model for Statistical

Machine Translation 73Qiang Li, Yaqian Han, Tong Xiao, and Jingbo Zhu

Cost-Aware Learning Rate for Neural Machine Translation 85Yang Zhao, Yining Wang, Jiajun Zhang, and Chengqing Zong

Knowledge Graph and Information Extraction

Integrating Word Sequences and Dependency Structures

for Chemical-Disease Relation Extraction 97Huiwei Zhou, Yunlong Yang, Zhuang Liu, Zhe Liu, and Yahui Men

Named Entity Recognition with Gated Convolutional Neural Networks 110Chunqi Wang, Wei Chen, and Bo Xu

Improving Event Detection via Information Sharing Among

Related Event Types 122Shulin Liu, Yubo Chen, Kang Liu, Jun Zhao, Zhunchen Luo,

and Wei Luo

Joint Extraction of Multiple Relations and Entities

by Using a Hybrid Neural Network 135Peng Zhou, Suncong Zheng, Jiaming Xu, Zhenyu Qi, Hongyun Bao,

and Bo Xu

A Fast and Effective Framework for Lifelong Topic Model

with Self-learning Knowledge 147Kang Xu, Feng Liu, Tianxing Wu, Sheng Bi, and Guilin Qi

Collective Entity Linking on Relational Graph Model with Mentions 159Jing Gong, Chong Feng, Yong Liu, Ge Shi, and Heyan Huang

XLink: An Unsupervised Bilingual Entity Linking System 172Jing Zhang, Yixin Cao, Lei Hou, Juanzi Li, and Hai-Tao Zheng

Using Cost-Sensitive Ranking Loss to Improve Distant Supervised

Relation Extraction 184Daojian Zeng, Junxin Zeng, and Yuan Dai

Multichannel LSTM-CRF for Named Entity Recognition

in Chinese Social Media 197Chuanhai Dong, Huijia Wu, Jiajun Zhang, and Chengqing Zong

Language Resource and Evaluation

Generating Chinese Classical Poems with RNN Encoder-Decoder 211Xiaoyuan Yi, Ruoyu Li, and Maosong Sun

Collaborative Recognition and Recovery of the Chinese

Intercept Abbreviation 224Jinshuo Liu, Yusen Chen, Juan Deng, Donghong Ji, and Jeff Pan

Semantic Dependency Labeling of Chinese Noun Phrases Based

on Semantic Lexicon 237Yimeng Li, Yanqiu Shao, and Hongkai Yang

Information Retrieval and Question Answering

Bi-directional Gated Memory Networks for Answer Selection 251Wei Wu, Houfeng Wang, and Sujian Li

Generating Textual Entailment Using Residual LSTMs 263Maosheng Guo, Yu Zhang, Dezhi Zhao, and Ting Liu

Unsupervised Joint Entity Linking over Question Answering Pair

with Global Knowledge 273Cao Liu, Shizhu He, Hang Yang, Kang Liu, and Jun Zhao

Hierarchical Gated Recurrent Neural Tensor Network

for Answer Triggering 287Wei Li and Yunfang Wu

Question Answering with Character-Level LSTM Encoders

and Model-Based Data Augmentation 295Run-Ze Wang, Chen-Di Zhan, and Zhen-Hua Ling

Exploiting Explicit Matching Knowledge with Long Short-Term Memory 306Xinqi Bao and Yunfang Wu

Text Classification and Summarization

Topic-Specific Image Caption Generation 321Chang Zhou, Yuzhao Mao, and Xiaojie Wang

Deep Learning Based Document Theme Analysis

for Composition Generation 333Jiahao Liu, Chengjie Sun, and Bing Qin

UIDS: A Multilingual Document Summarization Framework

Based on Summary Diversity and Hierarchical Topics 343Lei Li, Yazhao Zhang, Junqi Chi, and Zuying Huang

Conceptual Multi-layer Neural Network Model for Headline Generation 355Yidi Guo, Heyan Huang, Yang Gao, and Chi Lu

Social Computing and Sentiment Analysis

Local Community Detection Using Social Relations and Topic Features

in Social Networks 371Chengcheng Xu, Huaping Zhang, Bingbing Lu, and Songze Wu

NLP Applications

DIM Reader: Dual Interaction Model for Machine Comprehension 387Zhuang Liu, Degen Huang, Kaiyu Huang, and Jing Zhang

Contents XVII

Multi-view LSTM Language Model with Word-Synchronized Auxiliary

Feature for LVCSR 398Yue Wu, Tianxing He, Zhehuai Chen, Yanmin Qian, and Kai Yu

Memory Augmented Attention Model for Chinese Implicit Discourse

Relation Recognition 411Yang Liu, Jiajun Zhang, and Chengqing Zong

Natural Logic Inference for Emotion Detection 424Han Ren, Yafeng Ren, Xia Li, Wenhe Feng, and Maofu Liu

Minority Language Information Processing

Tibetan Syllable-Based Functional Chunk Boundary Identification 439Shumin Shi, Yujian Liu, Tianhang Wang, Congjun Long,

and Heyan Huang

Harvest Uyghur-Chinese Aligned-Sentences Bitexts from Multilingual Sites

Based on Word Embedding 449ShaoLin Zhu, Xiao Li, YaTing Yang, Lei Wang, and ChengGang Mi

Language Model for Mongolian Polyphone Proofreading 461Min Lu, Feilong Bao, and Guanglai Gao

End-to-End Neural Text Classification for Tibetan 472Nuo Qun, Xing Li, Xipeng Qiu, and Xuanjing Huang

Author Index 481XVIII Contents

Fundamental Theory and Methods

of Computational Linguistics

Arabic Collocation Extraction Based on Hybrid Methods

Alaa Mamdouh Akef, Yingying Wang, and Erhong Yang(&)

School of Information Science, Beijing Language and Culture University,

Beijing 100083, Chinaalaa_eldin_che@hotmail.com, yerhong@blcu.edu.cn

Abstract Collocation Extraction plays an important role in machine tion, information retrieval, secondary language learning, etc., and has obtainedsignificant achievements in other languages, e.g English and Chinese There aresome studies for Arabic collocation extraction using POS annotation to extractArabic collocation We used a hybrid method that included POS patterns andsyntactic dependency relations as linguistics information and statistical methodsfor extracting the collocation from Arabic corpus The experiment resultsshowed that using this hybrid method for extracting Arabic words can guarantee

transla-a higher precision rtransla-ate, which heightens even more transla-after dependency reltransla-ationsare added as linguistic rules forfiltering, having achieved 85.11% This methodalso achieved a higher precision rate rather than only resorting to syntacticdependency analysis as a collocation extraction method

Keywords: Arabic collocation extraction
Dependency relation
Hybridmethod

dis-Lexical collocation is the phenomenon of using words in accompaniment, Firthproposed the concept based on the theory of“contextual-ism” Neo-Firthians advancedwith more specific definitions for this concept Halliday (1976, p 75) defined collo-cation as“linear co-occurrence together with some measure of significant proximity”,while Sinclair (1991, p 170) came up with a more straightforward definition, statingthat“collocation is the occurrence of two or more words within a short space of eachother in a text” Theories from these Firthian schools emphasized the recurrence(co-occurrence) of collocation, but later other researchers also turned to its other

© Springer International Publishing AG 2017

M Sun et al (Eds.): CCL 2017 and NLP-NABD 2017, LNAI 10565, pp 3 –12, 2017.

https://doi.org/10.1007/978-3-319-69005-6_1

properties Benson (1990) also proposed a definition in the BBI Combinatory Dictionary

of English, stating that“A collocation is an arbitrary and recurrent word combination”,while Smadja (1993) considered collocations as“recurrent combinations of words thatco-occur more often than expected by chance and that correspond to arbitrary wordusages” Apart from stressing co-occurrence (recurrence), both of these definitions placeimportance on the“arbitrariness” of collocation According to Beson (1990), collocationbelongs to unexpected bound combination In opposition to free combinations, collo-cations have at least one word for which combination with other words is subject toconsiderable restrictions, e.g in Arabic, (breast) in (the breast of theshe-camel) can only appear in collocation with (she-camel), while (breast)cannot form a correct Arabic collocation with (cow) and (woman), etc

In BBI, based on a structuralist framework, Benson (1989) divided English location into grammatical collocation and lexical collocation, further dividing these twointo smaller categories, this emphasized that collocations are structured, with rules atthe morphological, lexical, syntactic and/or semantic levels

col-We took the three properties of word collocation mentioned above (recurrence,arbitrariness and structure) and used it as a foundation for the qualitative descriptionand quantitative calculation of collocations, and designed a method for the automaticextraction of Arabic lexical collocations

Researchers have employed various collocation extraction methods based on different

definitions and objectives In earlier stages, lexical collocation research was mainlycarried out in a purely linguistic field, with researchers making use of exhaustiveexemplification and subjective judgment to manually collect lexical collocations, forwhich the English collocations in the Oxford English Dictionary (OED) are a verytypical example Smadja (1993) points out that the OED’s accuracy rate doesn’t sur-pass 4% With the advent of computer technology, researchers started carrying outquantitative statistical analysis based on large scale data (corpora) Choueka et al.(1983) carried out one of thefirst such studies, extracting more than a thousand Englishcommon collocations from texts containing around 11,000,000 tokens from the NewYork Times However, they only took into account collocations’ property of recur-rence, without putting much thought into its arbitrariness and structure They alsoextracted only contiguous word combinations, without much regard for situations inwhich two words are separated, such as“make-decision”

Church et al (1991) defined collocation as a set of interrelated word pairs, using theinformation theory concept of“mutual information” to evaluate the association strength

of word collocation, experimenting with an AP Corpus of about 44,000,000 tokens.From then on, statistical methods started to be commonly employed for the extraction

of lexical collocations Pecina (2005) summarized 57 formulas for the calculation of theassociation strength of word collocation, but this kind of methodology can only act onthe surface linguistic features of texts, as it only takes into account the recurrence andarbitrariness of collocations, so that“many of the word combinations that are extracted

by these methodologies cannot be considered as the true collocations” (Saif2011) E.g

4 A.M Akef et al

“doctor-nurse” and “doctor-hospital” aren’t collocations Linguistic methods are alsocommonly used for collocation extraction, being based on linguistic information such

as morphological, syntactic or semantic information to generate the collocations (Attia

2006) This kind of method takes into account that collocations are structured, usinglinguistic rules to create structural restrictions for collocations, but aren’t suitable forlanguages with highflexibility, such as Arabic

Apart from the above, there are also hybrid methods, i.e the combination ofstatistical information and linguistic knowledge, with the objective of avoiding thedisadvantages of the two methods, which are not only used for extracting lexicalcollocations, but also for the creation of multi-word terminology (MWT) or expressions(MWE) For example, Frantzi et al (2000) present a hybrid method, which usespart-of-speech tagging as linguistic rules for extracting candidates for multi-word ter-minology, and calculates the C-value to ensure that the extracted candidate is a realMWT There are plenty of studies which employ hybrid methods to extract lexicalcollocations or MWT from Arabic corpora (Attia2006; Bounhas and Slimani2009)

We used a hybrid method combining statistical information with linguistic rules for theextraction of collocations from an Arabic corpus based on the three properties ofcollocation In the previous research studies, there were a variety of definitions ofcollocation, each of which can’t fully cover or be recognized by every collocationextraction method It’s hard to define collocation, while the concept of collocation isvery broad and thus vague So we just gave a definition of Arabic word collocation to

fit the hybrid method that we used in this paper

3.1 Definition of Collocation

As mentioned above, there are three properties of collocation, i.e recurrence, trariness and structure On the basis of those properties, we define word collocation ascombination of two words (bigram1) which must fulfill the three following conditions:

arbi-a One word is frequently used within a short space of the other word (node word) inone context

This condition ensures that bigram satisfies the recurrence property of word location, which is recognized on collocation research, and is also an essential pre-requisite for being collocation Only if the two words co-occur frequently andrepeatedly, they may compose a collocation On the contrary, the combination of wordsthat occur by accident is absolutely impossible to be a collocation (when the corpus islarge enough) As for how to estimate what frequency is enough to say“frequently”, itshould be higher than expected frequency calculated by statistical methods

col-1 It is worth mentioning that the present study is focused on word pairs, i.e only lexical collocations containing two words are included Situations in which the two words are separated are taken into account, but not situations with multiple words.

Arabic Collocation Extraction Based on Hybrid Methods 5

b One word must get the usage restrictions of the other word.

This condition ensures that bigram satisfies the arbitrariness property of wordcollocation, which is hard to describe accurately but is easy to distinguish by nativespeakers Some statistical methods can, to some extent, measure the degree of con-straint, which is calculated only by using frequency, not the pragmatic meaning of thewords and the combination

c A structural relationship must exist between the two words

This condition ensures that bigram satisfies the structure property of word cation The structural relationships mentioned here consist of three types on threelevels: particular part-of-speech combinations on the lexical level; dependency rela-tionships on the syntactic level, e.g modified relationship between adjective and noun

collo-or between adverb and verb; semantic relationships on the semantic level, e.g tionship between agent and patient of one act

rela-To sum up, collocation is defined in this paper as a recurrent bound bigram thatinternally exists with some structural relationships To extract collocations according tothe definition, we conducted the following hybrid method

3.2 Method for Arabic Collocation Extraction

The entire process consisted of data processing, candidate collocation extraction,candidate collocation ranking and manual tagging (Fig.1)

Fig 1 Experimentalflow chart

6 A.M Akef et al

Data processing We used the Arabic texts from the United Nations Corpus, prised of 21,090 sentences and about 870,000 tokens For data analysis and annotation,

com-we used the Stanford Natural Language Processing Group’s toolkit Data processingincluded word segmentation, POS tagging and syntactic dependency parsing.Arabic is a morphologically rich language Thus, when processing Arabic texts, thefirst step is word segmentation, including the removal of affixes, in order to make thedata conform better to automatic tagging and analysis format, e.g the word (tosupport something), after segmentation POS tagging and syntactic dependencyparsing was done with the Stanford Parser, which uses an“augmented Bies” tag set.The LDC Arabic Treebanks also uses the same tag set, but it is augmented in com-parison to the LDC English Treebanks’ POS tag set, e.g extra tags start with “DT”, andappear for all parts of speech that can be preceded by the determiner“Al” ( ) Syntacticdependency relations as tagged by the Stanford Parser, are defined as grammaticalbinary relations held between a governor (also known as a regent or a head) and adependent, including approximately 50 grammatical relations, such as “acomp”,

“agent”, etc However, when used for Arabic syntactic dependency parsing, it does nottag the specific types of relationship between word pairs It only tags word pairs fordependency with “dep(w1, w2)” We extracted 621,964 dependency relations frommore than 20,000 sentences

This process is responsible for generating, filtering and ranking candidatecollocations

Candidate collocation extracting This step is based on the data when POS tagginghas already been completed Every word was treated as a node word and every wordpair composed between them and other words in their span were extracted as collo-cations Each word pair has a POS tag, such as ((w1, p1), (w2, p2)), where w1 standsfor node word, p1 stands for the POS of w1 inside the current sentence, w2 stands forthe word in the span of w1 inside the current sentence (not including punctuation),while p2 is the actual POS for w2 A span of 10 was used, i.e the 5 words precedingand succeeding the node word are all candidate words for collocation Together withnode words, they constitute initial candidate collocations In 880,000 Arabic tokens, weobtained 3,475,526 initial candidate collocations

After constituting initial candidate collocations, taking into account that tions are structured, we used POS patterns as linguistic rules, thus creating structuralrestrictions for collocations According to Saif (2011), Arabic collocations can beclassified into six POS patterns: (1) Noun + Noun; (2) Noun + Adjective; (3) Verb +Noun; (4) Verb + Adverb; (5) Adjective + Adverb; and (6) Adjective + Noun,encompassing Noun, Verb, Adjective and Adverb, in total four parts of speech.However, in the tag set every part of speech also includes tags for time and aspect,gender, number, as well as other inflections (see to Table1for details) Afterwards, weapplied the above mentioned POS patterns for filtering the initial candidate colloca-tions, and continued treating word pairs conforming to the 6 POS patterns as candidatecollocations, discarding the others After filtering, there remained 704,077 candidatecollocations

colloca-Arabic Collocation Extraction Based on Hybrid Methods 7

Candidate collocation ranking For this step, we used statistical methods to calculatethe association strength and dependency strength for collocations, and sorting thecandidate collocations accordingly.

The calculation for word pair association strength relied on frequency of wordoccurrence and co-occurrence in the corpus, and for its representation we resorted tothe score of Point Mutual Information (PMI), i.e an improved Mutual Informationcalculation method, and also a statistical method recognized for reflecting the recurrentand arbitrary properties of collocations, and being widely employed in lexical collo-cation studies Mutual Information is used to describe the relevance between tworandom variables in information theory In language information processing, it is fre-quently used to measure correlation between two specific components, such as words,POS, sentences and texts When employed for lexical collocation research, it can beused for calculating the degree of binding between word combinations The formula is:

pmi wð 1; w2Þ ¼ log p wð 1; w2Þ

p(w1, w2) refers to the frequency of the word pair (w1, w2) in the corpus p(w1), p(w2)stands for the frequency of word occurrence of w1and w2 The higher the frequency ofco-occurrence of w1and w2, the higher p(w1, w2), and also the higher the pmi(w1, w2)score, showing that collocation (w1, w2) is more recurrent As to arbitrariness, thehigher the degree of binding for collocation (w1, w2), the lower the co-occurrencefrequency between w1 or w2 and other words, and also the lower the value of p(w1) orp(w2) This means that when the value of p(w1, w2) remains unaltered, the higher thepmi(w1, w2) score, which shows that collocation (w1, w2) is more arbitrary

The calculation of dependency strength between word pairs relies on the frequency

of dependency relation in the corpus The dependency relations tagged in the StanfordParser are grammatical relations, which means that dependency relations between wordpairs still belong to linguistic information, constituting thus structural restrictions forcollocations In this paper, we used dependency relation as another linguistic rule(exception of the POS patterns) to extract Arabic collocation Furthermore, the amount

of binding relations that a word pair can have is susceptible to statistical treatment, sothat we can utilize the formula mentioned above to calculate the Point Mutual Infor-mation score We used the score to measure the degree of binding between word pairs,but the p(w1, w2) in the formula refers to the frequency of dependency relation of (w1,

w2) in the corpus, whilst p(w1), p(w2) still stand for the frequency of word occurrence

of w1and w2 The higher the dependency relation of w1and w2, the higher the value of

Table 1 Arabic POS tag example

p(w1, w2), and also the higher the pmi(w1, w2) score, meaning that collocation (w1, w2)

is more structured

This step can be further divided into two stages First we calculated the associationscore (as) for all collocation candidates and sorted them from the highest to the lowestscore And then we traverse all ((w1, p1), (w2, p2)) collocate candidates, and if (w1, w2)possessed a dependency relation in the corpus, then we proceeded to calculate theirdependency score (ds), so that every word pair and the two scores composed aquadruple AC((w1, p1), (w2, p2), as, ds)) If (w1, w2) do not have a dependency relation

in the corpus, then ds is null After calculating the strength of dependency for all621,964 word pairs and sorting them from the highest to the lowest score, the wordpairs and strength of dependency constitute a tripe DC(w1, w2, ds)

Manual Tagging In order to evaluate the performance of the collocation extractionmethod suggested in the present paper, we extracted all collocation candidates for theArabic word execute” (AC quadruples where all w1is or its variants2) and alldependency collocations (DC triples where all w1is or its variants), obtaining atotal of 848 AC quadruples and 689 DC triples However, only word pairs in 312 of the

AC quadruples appear in these 689 DC triples This happens because the span set in themethods for collocation candidates in quadruples is 10, while analysis of the scope ofsyntactical dependency analysis comprises the whole sentence Thus, words outside ofthe span are not among the collocation candidates, but might have a dependencyrelation with node words Afterwards, each word pair in AC quadruples and DC tripleswere passed on to a human annotator for manual tagging and true or false collocation

The Tables2and3below present the proportional distribution of the results from thecollocation candidates for , as well as their precision rate.“True collocations” refer

to correct collocations selected manually, while“false collocations” refer to collocationerrors filtered manually “With Dependency relation” indicates that there exists onekind of dependency relation between word pairs, while“Without Dependency relation”indicates word pairs without dependency relation So “With Dependency relation”indicates collocations selected by the hybrid method presented in this paper, “truecollocation” and “With Dependency relation” stand for correct collocations selectedusing hybrid methods As to precision rate,“Precision with dependency relation” inTable2 represents the precision rate of the hybrid method which comprises POSpatterns, statistical calculation and dependency relations “Precision without depen-dency relation” represents the precision rate using POS patterns and statistical calcu-lation, without dependency relations “Precision with dependency relation only” inTable3 represents the precision rate of the method only using dependency relations3

2 One Arabic word could have more than one from in corpus because Arabic morphology is rich, so has 55 different variants.

3 Bigrams sorted by their dependency score (ds), which actually is the Point Mutual Information Score.

Arabic Collocation Extraction Based on Hybrid Methods 9

From the tables above, we can see that the precision of the hybrid method has beensignificantly improved compared to the precision of method without dependencyrelation and with dependency relation only More concretely, we canfind that in the set

of candidate collocations (bigrams) extracted andfiltered by POS patterns, PMI scoreand dependency relations, true collocations have much higher proportions than falsecollocations But the result is completely opposite in the set of candidate collocations(bigrams) extracted without dependency relations, i.e false collocations have muchhigher proportions than true collocations This data illustrates that the potential is verygreat for one word collocation internally exists with some kind of dependency relation,but not all collocations do Thus the results are enough to illustrate that it is reasonable

to use dependency relation as a linguistic rule to restrict collocation extraction.However, when we only use dependency relation as a linguistic rule to extract collo-cations, just as the data showed in Table3, false collocations also have much higherproportions than true collocations This data illustrates that dependency relation is notsufficient enough, and that POS patterns are also necessary to restrict collocationextraction

Table 2 The numerical details about extracted collocations using the hybrid method

as > 0 as > 1 as > 2 as > 3 as > 4 as > 5 as > 6 Percent of candidate collocation 100 78.89 77.36 45.28 30.90 17.57 9.79 6.49 Percent of true

collocations

13.68 7.31 6.72 2.71 1.77 0.83 0.71 0.35 Percent of rue

collocations

46.11 35.97 35.14 18.40 12.74 7.78 4.25 3.07 Precision with dependency relation 62.82 73.16 74.55 82.58 82.35 85.11 73.91 81.25 Precision without dependency

relation

40.21 45.44 45.88 53.39 53.05 51.01 49.40 47.27

Table 3 The numerical details about extracted collocations using dependency relation

ds > 0 ds > 1 ds > 2 ds > 3 ds > 4 ds > 5 ds > 6 ds > 7Percent of candidate

collocation

100.0 92.29 83.00 70.71 56.57 40.43 29.29 17.86 11.29Percent of true

collocations

38.14 37.57 36.00 31.86 25.71 18.57 13.00 7.29 4.71Percent of false

collocations

61.86 54.71 47.00 38.86 30.86 21.86 16.29 10.57 6.57Precision with

There is an example that illustrates the effect of dependency relation as a linguisticrule to filter the candidate collocations The bigram has a very high fre-quency in the Arabic corpus, ranking second, meaning“the level of” And whenthe two words co-occur in one sentence of the corpus, which mostly means“the level

of the executive (organ or institution)”, there is no dependency relation between thetwo words, so the bigram isfiltered out There are so many situations like this bigramthat can be successfullyfiltered out, which can significantly improve the precision rate

of the hybrid method of collocation extraction

As mentioned above, not all collocations have an internal dependency relation andnot all bigrams that have internal dependency relation are true collocations Such asbigram , which means “decide to implement”, (implementation) is theobject of (decide), there is a dependency relation between the word pair But we canannotate the bigram as a“false collocation” without hesitation These kinds of bigramsresult in the error rate of the hybrid method Beyond this, another reason for error ratecan be the incorrect dependency result analyzed by the Stanford Parser The hybridmethod of this paper only uses dependency relation as one linguistic rule, withoutbeing entirely dependent on it, so the precision of the hybrid method is much higherthan the method only using dependency relations

To sum it all up, the hybrid method presented in this paper can significantlyimprove the precision of collocation extraction

In this study, we have presented our method for collocation extraction from an Arabiccorpus This is a hybrid method that depends on both linguistic information andassociation measures Linguistic information is comprised of two rules: POS patternsand dependency relations Taking the Arabic word as an example, by using thismethod we were able to extract all the collocation candidates and collocation depen-dencies, as well as calculating its precision after manual tagging This experiment’sresults show that by using this hybrid method for extracting Arabic words, it canguarantee a higher precision rate, which heightens even more after dependency rela-tions are added as rules forfiltering, achieving 85.11% accuracy, higher than by onlyresorting to syntactic dependency analysis as a collocation extraction method

References

Attia, M.A.: Accommodating multiword expressions in an arabic LFG grammar In: Salakoski,T., Ginter, F., Pyysalo, S., Pahikkala, T (eds.) FinTAL 2006 LNCS, vol 4139, pp 87–98.Springer, Heidelberg (2006) doi:10.1007/11816508_11

Benson, M.: Collocations and general-purpose dictionaries Int J Lexicogr 3(1), 23–34 (1990)Benson, M.: The Structure of the Collocational Dictionary Int J Lexicography, 2(1) (1989)Bounhas, I., Slimani, Y.: A hybrid approach for Arabic multi-word term extraction In:International Conference on Natural Language Processing and Knowledge Engineering 2009,NLP-KE, vol 30, pp 1–8 IEEE (2009)

Arabic Collocation Extraction Based on Hybrid Methods 11

Church, K.W., Hanks, P., Hindle, D.: Using Statistics in Lexical Analysis Lexical Acquisition(1991)

Choueka, Y., Klein, T., Neuwitz, E.: Automation Retrieval of Frequent Idiomatic andCollocational Expressions in a Large Corpus J Literary Linguist Comput 4 (1983)Frantzi, K., Sophia, A., Hideki, M.: Automatic recognition of multi-word terms: theC-value/NC-value method Int J Digital Libraries 3, 115–130 (2000)

Halliday, M.A.K.: Lexical relations System and Function in Language Oxford University Press,Oxford (1976)

Pecina, P.: An extensive empirical study of collocation extraction methods ACL 2005, Meeting

of the Association for Computational Linguistics, pp 13–18, University of Michigan, USA(2005)

Saif, A.M., Aziz, M.J.A.: An automatic collocation extraction from Arabic corpus J Comput.Sci 7(1), 6 (2011)

Sinclair, J.: Corpus, Concordance, Collocation Oxford University Press, Oxford (1991)Smadja, F.: Retrieving collocations from text: extract Comput Linguist 19(19), 143–177 (1993)

12 A.M Akef et al

Employing Auto-annotated Data for Person

Name Recognition in Judgment Documents

Limin Wang, Qian Yan, Shoushan Li(&), and Guodong Zhou

Natural Language Processing Lab, School of Computer Science and Technology,

Soochow University, Suzhou, China{lmwang,qyan}@stu.suda.edu.cn, {lishoushan,

gdzhou}@suda.edu.cn

Abstract In the last decades, named entity recognition has been extensivelystudied with various supervised learning approaches depend on massive labeleddata In this paper, we focus on person name recognition in judgment docu-ments Owing to the lack of human-annotated data, we propose a joint learningapproach, namely Aux-LSTM, to use a large scale of auto-annotated data to helphuman-annotated data (in a small size) for person name recognition Specifi-cally, our approach first develops an auxiliary Long Short-Term Memory(LSTM) representation by training the auto-annotated data and then leveragesthe auxiliary LSTM representation to boost the performance of classifier trained

on the human-annotated data Empirical studies demonstrate the effectiveness ofour proposed approach to person name recognition in judgment documents withboth human-annotated and auto-annotated data

Keywords: Named entity recognition
Auto-annotated data
LSTM

Named entity recognition (NER) is a natural language processing (NLP) task and plays

a key role in many real applications, such as relation extraction [1], entity linking [2],and machine translation [3] Named entity recognition wasfirst presented as a subtask

on MUC-6 [4], which aims to find organizations, persons, locations, temporalexpressions and number expressions in text The proportion of Chinese names in theentities is large, according to statistics, in the“People’s Daily” in January 1998 corpus(2,305,896 words), specifically, the average per 100 words contains 1.192 unlistedwords (excluding time words and quantifiers), of which 48.6% of the entities areChinese names [5] In addition to the complex semantics of Chinese, the Chinese namehas a great arbitrariness, so the identification of the Chinese name is one of the mainand difficult tasks in named entity recognition

In the paper, we focus on the person name recognition in judgment documents Theratio of person name in judgment documents is very big, including not only plaintiffs,defendants, entrusted agents, but also other unrelated names, such as outsider, eye-witness, jurors, clerk and so on For instance, Fig.1shows an example of a judgmentdocument where person names exist However, in most scenarios, there is insufficient

© Springer International Publishing AG 2017

M Sun et al (Eds.): CCL 2017 and NLP-NABD 2017, LNAI 10565, pp 13 –23, 2017.

https://doi.org/10.1007/978-3-319-69005-6_2

annotated corpus data for person name recognition in judgment document and to obtainsuch corpus data is extremely costly and time-consuming.

Fortunately, wefind that the judgment documents are well-structured in some parts.For example, in Fig.1, we can see that in the front part, the word “ (Plaintiff)”often follows a person name Therefore, to tackle the difficulty of obtaininghuman-annotated data, we try to auto-annotate much judgment documents with someheuristic rules Due to the large scale of existing judgment documents, it is easy toobtain many auto-annotated sentences with person names and these sentences could beused as training data for person name recognition

E1:

TYPE=“PERSON”> </ENAMEX>

(English Translation:

Plaintiff <ENAMEX TYPE=“PERSON”> Yizi A </ENAMEX> complained, dant <ENAMEX TYPE=“PERSON”> Xianyin Ai </ENAMEX>, along with herneighbor GaoShan, had brought outsider FangLiang appearing in her rental.……)

defen-One straightforward approach to using auto-annotated data in person namerecognition is to merge them into the human-annotated data and use the merging data totrain a new model However, due to the automatic annotation, the data is noisy That is

to say, there still exist some person names are not annotated For example, in E1, thereare four person names in the sentence, but we can only annotate two person names viathe auto-annotating strategy

Fig 1 An example of a judgment document with the person names annotated in the text

14 L Wang et al

In this paper, we propose a novel approach to person name recognition by usingauto-annotated data in judgment documents with a joint learning model Our approachuses a small amount of human-annotated samples, together with a large amount ofauto-annotated sentences containing person names Instead of simply merging thehuman-annotated and auto-annotated samples, we propose a joint learning model,namely Aux-LSTM, to combine the two different resources Specifically, we firstseparate the twin person name classification task using the human-annotated data andthe auto-annotated data into a main task and an auxiliary task Then, our joint learningmodel based on neural network develops an auxiliary representation from the auxiliarytask of a shared Long Short-Term Memory (LSTM) layer and then integrates theauxiliary representation into the main task for joint learning Empirical studiesdemonstrate that the proposed joint learning approach performs much better than usingthe merging method.

The remainder of this paper is organized as follows Section 2 gives a briefoverviews of related work on name recognition Section3 introduces data collectionand annotation Section4presents some basic LSTM approaches and our joint learningapproach to name recognition Section5 evaluates the proposed approach Finally,Sect.6gives the conclusion and future work

Although the study of Chinese named entities is still in the immature stage comparedwith the English named entity recognition But there is a lot of research on Chinesenamed recognition Depending on the method used, these methods can be broadlydivided into three categories: rule method, statistical method and a combination of rulesand statistics

The rule method mainly uses two kinds of information: the name classification andthe restrictive component of the surname: that is, when mark the name with the obviouscharacter in the analysis process, the recognition process of the name is started and therelevant component, which limits the position of the name before and after

In the last decades, named entity recognition has been extensively studied withvarious supervised shallow learning approaches, such as Hidden Markow Models(HMM) [6], sequential perceptron model [7], and Conditional Random Fields(CRF) [8] Meanwhile, named entity recognition has been performed in various styles

of text, such as news [6], biomedical text [9], clinical notes [10], and tweets [11]

An important line of previous studies on named entity recognition is to improve therecognition performance by exploiting extra data resources One major kind of suchresearches is to exploit unlabeled data with various semi-supervised learning approa-ches, such as bootstrapping [12, 13], word clusters [14], and Latent Semantic Asso-ciation (LSA) [15] Another major kind of such researches is to exploit parallel corpora

to perform bilingual NER [16,17]

Recently, deep learning approaches with neural networks have been more and morepopular for NER Hammerton [18] applies a single-direction LSTM network to performNER with a combination word embedding learning approach Collobert [19] employsconvolutional neural networks (CNN) to perform NER with a sequence of word

Employing Auto-annotated Data for Person Name Recognition 15

embeddings Subsequently, recent studies perform NER with some other neural works, such as BLSTM [20], LSTM-CNNs [21], and LSTM-CRF [22].

3.1 Human-annotated Data

The data is built by ourselves and it is from a kind of law documents named judgments.Choosing this special kind of document as our experimental data is mainly due to thefact that judgments always have an invariant structure and several domain-specificregulations could be found therein, which makes it a good choice to test the effec-tiveness of our approach We obtain the Chinese judgments from the governmentpublic website (i.e., http://wenshu.court.gov.cn/) The judgments are organized invarious categories of laws and we pick the Contract Law In the category, we manuallyannotate 100 judgment documents according the annotation guideline in OntoNotes 5.0[23] Two annotators are asked to annotate the data Due to the clear annotationguideline, the annotation agreement on name recognition is very high, reaching 99.8%.3.2 Auto-annotated Data

Note that a Chinese judgment always has an invariant structure where plaintiffs anddefendants are explicitly described in two lines in the front part It is easy to capturesome entities from two textual patterns, for example, “ NAME1, (Plaintiff

“NAME2” denotes a person name if the length is less than 4 Therefore, we first matchthe name through the rules in the front part of judgment instruments Second, we onlyselected the sentences containing the person name as the auto-annotated samples fromthe entire judgment documents In this way, we could quickly obtain more than 10,000auto-annotated judgment documents

4.1 LSTM Model for Name Recognition

In this subsection, we propose the LSTM classification model Figure2 shows theframework overview of the LSTM model for name recognition

Formally, the input of the LSTM classification model is a character’s representation

xi, which consists of character unigram and bigram embeddings for representing thecurrent character, i.e.,

xi¼ vci1 vci vciþ 1  vci þ 1;ciþ 2 ð1ÞWhere vci2 Rd is a d-dimensional real-valued vector for representing the characterunigram ci and vci;ciþ 12 Rd is a d-dimensional real-valued vector for representing thecharacter bigram ci; ci þ 1.

16 L Wang et al

Through the LSTM unit, the input of a character is converted into a new sentation hi, i.e.,

The dropout layer is applied to randomly omit feature detectors from networkduring training It is used as hidden layer in our framework, i.e.,

hdi ¼ h

Where D denotes the dropout operator, p denotes a tunable hyper parameter, and hd

idenotes the output from the dropout layer

Fig 2 The framework overview of the LSTM model for character-level NER

Employing Auto-annotated Data for Person Name Recognition 17

The softmax output layer is used to get the prediction probabilities, i.e.,

Where Pi is the set of predicted probabilities of the word classification, Wd is theweight vector to be learned, and the bd is the bias term Specifically, Piconsists of theposterior probabilities of the current word belonging to each position tag, i.e.,

4.2 Joint Learning for Person Name Recognition via Aux-LSTM

In the Fig.3 delineates the overall architecture of our Aux-LSTM approach whichcontains a main task and an auxiliary task In our study, we consider the person namerecognition with the human-annotated data as the main task and the name recognitionwith auto-annotated data as the auxiliary task The approach aims to enlist the auxiliaryrepresentation to assist in the performance of the main task The main idea of ourAux-LSTM approach is that the auxiliary LSTM layer is shared by both the main and

Fig 3 Overall architecture of Aux-LSTM

18 L Wang et al

auxiliary task so as to take advantage of information from both the annotated andauto-annotated data.

(1) The Main Task:

Formally, the representation of main task is generated from both the main LSTM layerand the auxiliary LSTM layer respectively:

where hmain1 represents the output of classification model via main LSTM layer and

hmain2 represents the output of classification model via auxiliary LSTM layer.Then we concatenate the two representation as the input of the hidden layer in themain task:

hdmain¼ densemainðhmain1 hmain2Þ ð9Þwhere hdmain denotes the outputs of fully-connected layer in the main task, and denotes the concatenate operator as a‘concat’ mode

(2) The Auxiliary Task:

The auxiliary classification representation is also generated by the auxiliary LSTMlayer, which is a shared LSTM layer and is employed to bridge across the classificationmodels The shared LSTM layer encodes both the same input sequence with the sameweights and the output hauxis the representation for the classification model via sharedLSTM model

Then a fully-connected layer is utilized to obtain a feature vector for classification,which is the same as the hidden layer in the main task:

Other layers such as softmax layer, as shown in Fig.2, are the same as those whichhave been described in Sect.4.1

Finally, we define our joint cost function for Aux-LSTM as a weighted linearcombination of the cost functions of both the main task and auxiliary task as follows:

In the above equation,k is the weight parameter, lossmainand lossauxis the loss function

of main task and auxiliary task respectively We take ‘adadelta’ as the optimizingalgorithm All the matrix and vector parameters in neural network are initialized with

Employing Auto-annotated Data for Person Name Recognition 19

uniform samples in pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi6= r þ cð Þ;pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi6=ðr þ cÞ, where r and c are the numbers ofrows and columns in the matrices [24].

In this section, we have systematically evaluated our approach to person namerecognition together with both human annotated and the auto-annotated data

5.1 Experimental Settings

Data Setting: The data collection has been introduced in Sect.3.1 In the main task,

we randomly select 20 articles of human-annotated data as training data and another 50articles of human-annotated as the test data In the auxiliary task, we randomly selectthe number of training samples corresponding to the number of 5 times, 10 times, 20times, 30 times and 40 times as the training data and the test data is the same as that inthe main task

Features and Embedding: We use the current character and its surrounding acters (window size is 2), together with the character bigrams as features We useword2vec (http://word2vec.googlecode.com/) to pre-train character embeddings usingthe two data sets

char-Basic Classification Algorithms: (1) Conditional Random Fields (CRFs), one ular supervised shallow learning algorithms, is implemented with the CRF++-0.531andall the parameters are set as defaults (2) LSTM, as the basic classification algorithm inour approach, is implemented with the tool Keras2 Table1 shows the finalhyper-parameters of the LSTM algorithm

pop-Hyper-parameters: The hyper-parameter values in the LSTM and Aux-LSTM modelare tuned according to performances in the development data

Table 1 Parameter settings in LSTM

Dimension of the LSTM layer output 128Dimension of the full-connected layer output 64

Evaluation Measurement: The performance is evaluated using the standard precision(P), recall (R) and F-score.

5.2 Experimental Results

In this subsection, we compare different approaches to person name recognition withboth human-annotated and auto-annotated data The implemented approaches areillustrated as follows:

• CRF: It is a shallow-learning model which has been widely employed in namerecognition, and it simply merges the human-annotated data and the auto-annotatedsamples together as the whole training data

• LSTM: It is deep learning model which has been widely employed in the naturallanguage processing community, and the training data is the same as that in CRF

• Aux-LSTM: This is our approach which develops an auxiliary representation forjoint learning In this model, we consider two tasks: one is the name recognitionwith the human-annotated data, and the other is the name recognition with theauto-annotated data The approach aims to leverage the extra information to boostthe performance of name recognition The parameterk is set to be 0.5

Table2 shows the number of characters, sentences and person names inauto-annotated documents with different sizes From this table, we can see that, thereare a great number of person names that could be automatically recognized in judgmentdocuments When 1000 documents are auto-annotated, there are totally 79411 recog-nized person names, which make the auto-annotated data a big-size training data forperson name recognition

Table3shows the performance of different approaches to person name recognitionwhen different size of human-annotated and auto-annotated data are employed.Specifically, the first line named “0” means using only human-annotated data and thesecond line “100” means using both human-annotated data and 100 auto-annotatedjudgment documents From this table, we can see that,

• When no auto-annotated data is used, the LSTM model performs much better thanCRF, mainly due to its better performance on Recall

Table 2 The number of character, sentence and person name in different auto-annotated dataNumber of auto-annotated

documents

Number ofcharacters

Number ofsentences

Number of personnames

• When a small size of auto-annotated data is used, the LSTM model generallyperforms better than CRF in terms of F1 score But when the size of auto-annotateddata becomes larger, the LSTM model performs a bit worse than CRF in terms ofF1 score No matter the LSTM or CRF model is used, using the auto-annotated dataalways improves the person name recognition performances with a large margin.

• When the auto-annotated data is used, our approach, i.e., Aux-LSTM, performs bestamong the three approaches Especially, when the size of the auto-annotated databecomes larger, our approach performs much better than LSTM This is possiblybecause our approach is more robust for adding noisy training data

In this paper, we propose a novel approach to person name recognition with bothhuman-annotated and auto-annotated data in judgment documents Our approachleverages a small amount of human-annotated samples, together with a large amount ofauto-annotated sentences containing person names Instead of simply merging thehuman-annotated and auto-annotated samples, we propose a joint learning model,namely Aux-LSTM, to combine the two different resources Specifically, we employ anauxiliary LSTM layer to develop the auxiliary representation for the main task ofperson name recognition Empirical studies show that using the auto-annotated data isvery effective to improve the performances of person name recognition in judgmentdocuments no matter what approaches are used Furthermore, our Aux-LSTM approachconsistently outperforms using the simple merging strategy with CRF or LSTMmodels

In our future work, we would like to improve the performance of person namerecognition by exploring the more features Moreover, we would like to apply ourapproach to name entity recognition on other types of entities, such as organizationsand locations in judgment documents

Acknowledgments This research work has been partially supported by three NSFC grants,

No 61375073, No 61672366 and No 61331011

Table 3 Performance comparison of different approaches to name recognition

4 Chinchor, N.: MUC7 Named Entity Task Definition (1997)

5 Ji, N.I., Kong, F., Zhu, Q., Peifeng, L.I.: Research on chinese name recognition base ontrustworthiness J Chin Inf Process 25(3), 45–50 (2011)

6 Zhou, G., Su, J.: Named entity recognition using an Hmm-based chunk tagger In:Proceedings of ACL, pp 473–480 (2002)

7 Collins, M.: Discriminative training methods for hidden Markov models: theory andexperiments with perceptron algorithms In: Proceedings of EMNLP, pp 1–8 (2002)

8 Finkel, J.R., Grenager, T Manning, C.: Incorporating non-local information into informationextraction systems by gibbs sampling In: Proceedings of ACL, pp 363–370 (2005)

9 Yoshida, K., Tsujii, J.: Reranking for biomedical named entity recognition In: Proceedings