Computational Intelligence in Automotive Applications by Danil Prokhorov_1 pot

As if reflecting the general lack of consensus on what constitutes the field of Cl, this volume iHustrates automotive applications of not only neural and fuzzy computationst which are co

Danil Prokhorov (Ed.)

Computational Intelligence in Automotive Applications

Studies in Computational Intelligence, Volume 132

Editor-in-chief

Prof Janusz Kacprzyk

Systems Research Institute

Polish Academy of Sciences

ul Newelska 6

01-447 Warsaw

Poland

E-mail: kacprzyk@ibspan.waw.pl

Further volumes of this series can be found on our

homepage: springer.com

Vol 111 David Elmakias (Ed.)

New Computational Methods in Power System Reliability,

2008

ISBN 978-3-540-77810-3

Vol 112 Edgar N Sanchez, Alma Y Alanis and Alexander

G Loukianov

Discrete-Time High Order Neural Control: Trained with

Kalman Filtering, 2008

ISBN 978-3-540-78288-9

Vol 113 Gemma Bel-Enguix, M Dolores Jimenez-Lopez and

Carlos Martin-Vide (Eds.)

New Developments in Formal Languages and Applications,

2008

ISBN 978-3-540-78290-2

Vol 114 Christian Blum, Maria José Blesa Aguilera, Andrea

Roli and Michael Sampels (Eds.)

Hybrid Metaheuristics, 2008

ISBN 978-3-540-78294-0

Vol 115 John Fulcher and Lakhmi C Jain (Eds.)

Computational Intelligence: A Compendium, 2008

ISBN 978-3-540-78292-6

Vol 116 Ying Liu, Aixin Sun, Han Tong Loh, Wen Feng Lu

and Ee-Peng Lim (Eds.)

Advances of Computational Intelligence in Industrial

Systems, 2008

ISBN 978-3-540-78296-4

Vol 117 Da Ruan, Frank Hardeman and Klaas van der Meer

(Eds.)

Intelligent Decision and Policy Making Support Systems,

2008

ISBN 978-3-540-78306-0

Vol 118 Tsau Young Lin, Ying Xie, Anita Wasilewska

and Churn-Jung Liau (Eds.)

Data Mining: Foundations and Practice, 2008

ISBN 978-3-540-78487-6

Vol 119 Slawomir Wiak, Andrzej Krawczyk and Ivo Dolezel

(Eds.)

Intelligent Computer Techniques in Applied

Electromagnetics, 2008

ISBN 978-3-540-78489-0

Vol 120 George A Tsihrintzis and Lakhmi C Jain (Eds.)

Multimedia Interactive Services in Intelligent Environments,

2008

ISBN 978-3-540-78491-3

Vol 121 Nadia Nedjah, Leandro dos Santos Coelho and Luiza de Macedo Mourelle (Eds.)

Quantum Inspired Intelligent Systems, 2008 ISBN 978-3-540-78531-6

Vol 122 Tomasz G Smolinski, Mariofanna G Milanova

and Aboul-Ella Hassanien (Eds.) Applications of Computational Intelligence in Biology, 2008

ISBN 978-3-540-78533-0 Vol 123 Shuichi Iwata, Yukio Ohsawa, Shusaku Tsumoto,

Ning Zhong, Yong Shi and Lorenzo Magnani (Eds.) Communications and Discoveries from Multidisciplinary

Data, 2008

ISBN 978-3-540-78732-7 Vol 124 Ricardo Zavala Yoe (Ed.) Modelling and Control of Dynamical Systems: Numerical

Implementation in a Behavioral Framework, 2008 ISBN 978-3-540-78734-1

Vol 125 Larry Bull, Bernad6-Mansilla Ester and John Holmes (Eds.)

Learning Classifier Systems in Data Mining, 2008 ISBN 978-3-540-78978-9

Vol 126 Oleg Okun and Giorgio Valentini (Eds.) Supervised and Unsupervised Ensemble Methods and their Applications, 2008

ISBN 978-3-540-78980-2

Vol 127 Régie Gras, Einoshin Suzuki, Fabrice Guillet

and Filippo Spagnolo (Eds.) Statistical Implicative Analysis, 2008 ISBN 978-3-540-78982-6 Vol 128 Fatos Xhafa and Ajith Abraham (Eds.)

and Manufacturing Applications, 2008 ISBN 978-3-540-78984-0

Vol 129 Natalio Krasnogor, Giuseppe Nicosia, Mario Pavone and David Pelta (Eds.)

(NICSO 2007), 2008

ISBN 978-3-540-78986-4 Vol 130 Richi Nayak, N Ichalkaranje and Lakhmi C Jain (Eds.)

Evolution of Web in Artificial Intelligence Environments, 2008 ISBN 978-3-540-79139-3

Vol 131 Roger Lee and Haeng-Kon Kim (Eds.) Computer and Information Science, 2008 ISBN 978-3-540-79186-7

Vol 132 Danil Prokhorov (Ed.) Computational Intelligence in Automotive Applications, 2008 ISBN 978-3-540-79256-7

Danil Prokhorov

(Ed.)

Computational Intelligence

in Automotive Applications

With 157 Figures and 48 Tables

2) Springer

Danil Prokhorov

Toyota Technical Center - A Division

of Toyota Motor Engineering

and Manutacturing (TEMA)

Ann Arbor, MI 48105

USA

dvprokhorov@gmail.com

ISBN 978-3-540-79256-7 e-FSBN 978-3-540-79257-4

Studies in Computational Intelligence ISSN 1860-949X

Library of Congress Control Number: 2008925554

(€) 2008 Springer-Verlag Berlin Heidelberg

This work is subject to copyright All rights are reserved, whether the whole or part of the material is con- cerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks Duplication of this publica- tion or parts thereof is permitted only under the provisions of the German Copyright Law of September

9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag Violations are liable to prosecution under the German Copyright Law

The use of general descriptive names, registered names, trademarks, etc in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use

Cover design: Deblik, Berlin, Germany

Printed on acid-free paper

987654321

springer.com

Computational Intelligence in Automotive Applications

Preface

»r 1

What is computational intelligence (CD? Traditionally, CT is understood as a collection of methods from the fields of neural networks (NN}, fuzzy logic and evolutionary computation Various definitions and opinions exist, but what belongs to CT is still being debated; see, e.g., [1-3] More recently there has been a proposal

to define the CI not in terms of the tools but in terms of challenging problems to be solved [4]

With this edited volume i have made an attempt to give a representative sample of contemporary Cl activities in automotive applications to illustrate the state of the art While CT research and achievements in some specialized fields described (see, e.g., [5, 6]), this is the first volume of its kind dedicated to automotive technology As if reflecting the general lack of consensus on what constitutes the field of Cl, this volume iHustrates automotive applications of not only neural and fuzzy computationst which are considered to be the “standard” CI topics, but also others, such as decision trees, graphical models, Support Vector Machines (SVM}, multi-agent systems, etc

This book is neither an introductory text, nor a comprehensive overview of all CI research in this area Hopefully, as a broad and representative sample of CI activities in automotive applications, it will be worth reading for both professionals and students When the details appear insutlicient, the reader is encouraged

to consult other relevant sources provided by the chapter authors

The chapter “Learning-based driver workload estimation” discusses research on estimation of driver cognitive workload and proposes a new methodology to design driver workload estimation systems The methodology is based on decision-tree learning It derives optimized models to assess the time-varving work- load levei from data which include not only measurements from various sensors but also subjective workload level ratings

The chapter “Visual monitoring of driver inattention” introduces a prototype computer vision system for real-time detection of driver fatigue The system inciudes an image acquisition module with an infrared illuminator, pupil detection and tracking module, and algorithms for detecting appropriate visuai behaviors and monitoring six parameters which may characterize the fatigue level of a driver To increase effectiveness

of monitoring, a fuzzy classifier is implemen “Ot ed to fuse all these parameters into a single gauge of driver robustly at night

The chapter “Understanding driving activity using ensemble methods” complements the chapter “Visual monitoring of driver inattention” by discussing whether driver inattention can be detected without eye and head tracking systems Instead of limiting themselves to working with just a few signals from preselected sensors, the authors chose to operate on hundreds of signals reflecting the real-time environment both outside and inside the vehicle The discovery of relationships in the data useful for driver activity classification, as

1 € ˆ + + + i ` + 1 * >

* Another “standard” CT topic called evolutionary computation (HC) is not represented in this volume in the form

if

contributors of this volume

3

weil as ranking signals in terms of their importance for classification, is entrusted to an approach called random forest, which turned out to be more effective than either hidden Markov models or SVM

‘ne chapter “Computer vision and machine learning for enhancing pedestrian safety” overviews methods for pedestrian detection, which use information from on-board and infrastructure based-sensors Many of the discussed methods are sufficiently generic to be useful for object detection, classification and motion prediction in general

The cha pter * ‘Application of graphical models in the automotive industry” describes briefly how graphical models, such as Bayesian and Markov networks, are used at Volkswagen and Daimler Production planning

at Volkswagen and demand prediction benefit significantly f from the graphical model based system developed CS

Another data mining system is developed for Daimler to help assessing the quality of vehicles and identifying causes of troubles when the vehicles have already spent some time in service It should be noted that other automotive companies are also pursuing data mining research (see, e.g., [8])

The chapter “Extraction of maximum support rules for the root cause analysis” discusses extraction of rules from manufacturing data for root cause analysis and process optimization An alternative approach to traditional methods of root cause analysis is proposed This new approach empioys branch-and-bound princi- piles, and it associates process parameters with results of measurements, which is helpful in the identification

of the main drivers for quality variations of an automotive manufacturing process

The chapter “Neural networks in automotive applications” provides an overview of neural network tech- nology, concentrating on three main roles of neural networks: models, virtual or soft sensors and controllers Training of NN is also discussed, followed by a simple example illustrating the importance of recurrent NN The chapter “On learning machines for engine control” deals with modeling for control of turbocharged spark ignition engines with variable camshaft timing Two examples are considered: (1} estimation of the in-cylinder air mass in which open loop neural estimators are combined with a dynamic polytopic observer, and (2} modeling an in-cylinder residual gas fraction by a linear programming support vector regression method The authors argue that models based on first principles (“white boxes”) and neural or other “black box” models must be combined and utilized in the “grey box” approach to obtain results which are not just superior to any alternatives but are also more acceptable to automotive engineers,

The chapter “Recurrent neural networks for AFR estimation and control in spark ignition automotive engines” cormplements the chapter “On learning machines for engine control” by discussing specifics of the air-fuel ratio (AFR) control Recurrent NN are trained of-line and employed as both the AFR virtual sensor and the inverse model controiler The authors also provide a comparison with a conventional control strategy

on a real engine

The chapter “Intelligent vehicle power management: An overview” presents four case studies: a conven- tional vehicle power comtroller and three different approaches for a parailel HEV power controller They include controllers based on dynamic programming a and neural networks, and fuzzy logic controllers, one of

which incorporates predictions of driving environments and driving patterns

The chapter “Tategrated diagnostic process for automotive systems” provides an overview of model-based and data-driven diagnostic methods applicable to complex systems Selected methods are applied to three automotive exampies, one of them being a hardware-in-the-loop system, in which the methods are put to work together to solve diagnostic and prognostic problems It should be noted that integration of different approaches is an important theme for automotive research spanning the entire product life cycle (see, e.g.,

l9)

The chapter “Automotive manufacturing: intelligent resistance welding” introduces a real-time contro! system for resistance spot welding The control system is built on the basis of neural networks and fuzzy loại It inchides a learning vector quantization NN for assessing the quality of weld nuggets and a fuzzy logïc process controller Experimental results indicate substantial quality improvement over a conventional controller

The chapter “Intelligent contro! of mobility systems” (ICMS) overviews projects of the IOMS Program

at the National Institute of Standards and Technology (NIST) The program provides architecture, interface and data standards, performance test methods and infrastructure technology available to the manufacturing industry and government agencies in developing and applying intelligent control techncology to mobility systems A common theme among these projects is autonomy and the four dimensional/real-time contro!

ns (4D/RCS) control architecture for intelligent systems proposed and developed in the NIST Intelligent Sys ster ms Division,

Unlike the book’s index, each chapter has its own bibliography for the convenience of the reader, with little overlap among references of diferent chapters

This volume highlights important challenges facing CY in the automotive domain Better vehicle diag- nostics/vehicle system safety, improved control of vehicular systems and manufacturing processes to save resources and minimize impact on the environment, better driver state moni ors improved safety of pedes- trians, making vehicles more intelligent on the road — these are important directions where the Cl lechnology can al nd should make the impact All of these are consistent with the Toyota vision [HO]:

Toyata’s vision is te balance “Zeronize” and “Maximize” “Zeronize” symbolizes the vision and philosophy

of our persistent efforts in minimizing negative aspects vehicles have such as environmental impact, traffic

congestion and traffic accidents, while “Maximize” symbolizes the vision and philosophy of our persistent efforts in maximizing the positive aspects vehicles have such as fun, delight, exciternent and comfort, that people seek in automobiles

Tam very thankful to all the contributors of this edited volume for their willingness to participate in this project, their patience and valuable time I am also grateful to Prof Janusz Kacprzyk, the Springer Series Editor, for his encouragement to organize and edit this volume, as weil as Thomas Ditzinger, the Springer production editor for his support of this project

Ann Arbor-Canton, MI, USA, Danil V Prokhorov Jaruary 2008

References

¬ http: //en.wikipedia.org/wiki/Computational_intelligence

Intelligence: Imitating Life, pp 1-12, TEEE Press, New York, 1994

3 RS Marks IY, “Intelligence: Computational Versus Artificial,” [EEG Transactions on Neurai Networks, 4(5),

(Eds.)}, Challe enges for Computational Intelligence, Vol 63 of Studies in Computational Intelligence (J

http: //cogprints.org/5358/

5 Intelligent Control Systems Using Computational Intelligence Techniques (&E Control Series) Edited by Autonio Ruano, TEE, 2005

6 R Begg, Daniel T.-H Lai, M Palaniswami Computational inteiligence in Bion

Taylor & Francis Books, Boca Raton, Florida, 2007

7 Marco Laumanns and Nando Laumanns, “Evolutionary Multiobjective Design in Automotive Development,” Applied Intelligence, 23, 55-70, 2005

8 T.A Montgomery, “Text Mining on a Budget: Reduce, Reuse, Recycle,” Afichigan Leadership Summit on Business Intelligence and Advanced Analytics, Troy, Mi, March 8, 2007 Presentation is available on-line at

http: / /www.emurc.com/bi-PreviousEvents.btm

P Struss and C Price, “Model-Based Systems in the Automotive Industry,” Af Magazine, Vol 24, No 4,

pp 17-34, AAAT, Menlo Park, Winter 2003

10 Toyota ITS vision, http: //www-toyota.co.jp/en/tech/its/vision /

ing ORC Press,

sO

Contents

Yiu Zhang, Yuri Qwechko, and Jing Zhang 0 ttt eee BAN an ằằẮẮẶẮÁẶa- —— “(dd .ẶẼằẶằẶ

2 Existing Practice and Hs Challenges 00 tee tenets

3 The Proposed Approach: Learning-Based DWE

3.1 Learning-Based DWE Design Process

3.2 Benefits of Learning-Based DWE

Experimental Data

Experimenta! Process

Experimental Results

6.1 Driver-Independent Training

6.2 Driver-Dependent Training 000.0 nent no tee e eee es 15

Visual Monitoring of Driver Inattention

Luis M Bergasa, Jestis Nuevo, Miguel A Sotelo, Rafael Barea, and Elena Lopez i

1 Introduction

3 System Architecture 00 ee ree ee tenet tern ne tenes 21 3.4 Image Acquisition System nh rene ee tbe tenes 22 3.2 Pupil Detection and Tracking 2.000.020.0000 00 cee tte eee 24 3.3 Visuai Behaviors 00.0.0 een nee been t eee 26

A"“ nh n6 6 eẼ6ẽ8aWaH ằẰằẰaaaaaẢ 3e

AL Test Sequences e6 ce tt terete tenet eens 3e 4.2 Parameter Measurement for One of the Test Sequences .0.00.0 0.00000 ce eee eee 30

6 Conclusions and Future Work .0.00.0 00000 cc teen tent e ete n ee ees 35

References

Understanding Driving Activity Using Ensemble Methods

Kari Torkkola, Mike Gardner, Chris Schreiner, Keshu Zhang, Beb Leiwian, Harry Zhang,

2 Modeling Naturalistic Driving 0.0 00000 eet teens

4 Driving Data Classification 0.0 ee tenet tenet eens

4.38 Random Eoresta for IDriving Mlaneuver D@teelÏON cv vn nh so

5 Sensor Selection Using Random Forests 2.0.2.0 0.00 ccc ete eee

6 Driver Inattention Detection Through Intelligent Analysis of Readily Available Sensors

6.2 Inattention Data PTOC©SSINE uc HH He nen HH Ho HH HH Ki và ở

Computer Vision and Machine Learning for Enhancing Pedestrian Safety

Torak Gandhi and Mohan Manubhai Trivedi occ tee cee teenies

IlMN co on n een enn ee eee .ưr

2 Framework for Pedestrian Protection System ch ng nh ke ke ng và ky

3 ‘Techniques in Pedestrian [Ầet€CHOH cv LH HH ng ng gà Hà hà tk và và và ky xo

3.1 Candidate Generation ố n8 H.MK eee teen eens

3.2 Candidate Validation 00.0.00 000 0c cc cee ett cette eens

4 Infrastructure Based Systems nến 6a .aÁA1AgA

4.1 Background Subtraction and Shadow Suppression 0.000000 eee eee ee

4.2 Robust Multi:Camera Detection and Tracking .00.0.0 0.000000 ccc eee eee

4.3 Analysis of Object Actions and Interactions 0.0.0.0 000 0 ee eee

6 Pedestrian Path Prediction .00 0.00002 ccc cette eens

References 8E aa , á ÈỀ-

Application of Graphical Models in the Automotive Industry

Matthias Steinbrecher, Frank Riigheimer, and Rudolf Kruse 0000 ee eee

1 Introduction 2.0000 HH TA n etn ae

2 Graphical Models 0 e6eéẽra 1Ả

2.1 Bayesian Networks cu uc HH HH ett tect eens

2.2 Markov Networks 8n NN HHÁẠỤIAA a -

3 Production Planning at Volkswagen Group oo 0 ee cet eee

3.1 Data Deseription and Model Induction 0.00000 c eee teen ene

3.2 Operations on the Model 0.0 ec ett tte v ene enaee

3.3 Application 0.0.0.0 ec tenet beet ene t eee aee

4 Vehicle Data Mining at Daimler AG cuc cv kh nh no nh hà nh gà kh Xà và kà và nà

4.1 Data Description and Model Induction cv tt cu nh kg nhà và và xa

5 Conclusion 2.0.2 en (ư <d(ớ

References ằẶ etn teen t eect eee n ene

Ort er) é

¬ =› ‘ =

Op meee

7

79

72

72

78

76

( z

oO is wo