The integration of electronics and clothing is an emerging field which aims to the development of multi-functional, wearable electro-textiles for applications together with body function
Trang 2Istrate, D., Vacher, M & Serignat, J.-F (2008) Embedded implementation of distress
situa-tion identificasitua-tion through sound analysis, The Journal on Informasitua-tion Technology in
Healthcare 6(3): 204–211.
Katz, S & Akpom, C (1976) A measure of primary sociobiological functions, International
Journal of Health Services 6(3): 493–508.
Kröse, B., van Kasteren, T., Gibson, C & van den Dool, T (2008) Care: Context awareness in
residences for elderly, Int Conference of the Int Soc for Gerontechnology, Pisa, Tuscany,
Italy
Kumiko, O., Mitsuhiro, M., Atsushi, E., Shohei, S & Reiko, T (2004) Input support for elderly
people using speech recognition, IEIC Technical Report 104(139): 1–6.
LeBellego, G., Noury, N., Virone, G., Mousseau, M & Demongeot, J (2006) A model for
the measurement of patient activity in a hospital suite, IEEE Trans on Information
Technology in Biomedicine 10(1): 92–99.
Litvak, D., Zigel, Y & Gannot, I (2008) Fall detection of elderly through floor vibrations and
sound, Proc 30th Annual Int Conference of the IEEE-EMBS 2008, pp 4632–4635.
Maunder, D., Ambikairajah, E., Epps, J & Celler, B (2008) Dual-microphone sounds of daily
life classification for telemonitoring in a noisy environment, Proc 30th Annual
Inter-national Conference of the IEEE-EMBS 2008, pp 4636–4639.
Michaut, F & Bellanger, M (2005) Filtrage adaptatif : théorie et algorithmes, Lavoisier.
Moore, D & Essa, I (2002) Recognizing multitasked activities from video using
stochas-tic context-free grammar, Proc of American Association of Artificial Intelligence (AAAI)
Conference 2002, Alberta, Canada.
Niessen, M., Van Maanen, L & Andringa, T (2008) Disambiguating sounds through context,
Proc Second IEEE International Conference on Semantic Computing, pp 88–95.
Noury, N., Hadidi, T., Laila, M., Fleury, A., Villemazet, C., Rialle, V & Franco, A (2008) Level
of activity, night and day alternation, and well being measured in a smart hospital
suite, Proc 30th Annual Int Conference of the IEEE-EMBS 2008, pp 3328–3331.
Noury, N., Villemazet, C., Barralon, P & Rumeau, P (2006) Ambient multi-perceptive
sys-tem for residential health monitoring based on electronic mailings experimentation
within the AILISA project, Proc 8th Int Conference on e-Health Networking, Applications
and Services HEALTHCOM 2006, pp 95–100.
Popescu, M., Li, Y., Skubic, M & Rantz, M (2008) An acoustic fall detector system that
uses sound height information to reduce the false alarm rate, Proc 30th Annual Int.
Conference of the IEEE-EMBS 2008, pp 4628–4631.
Portet, F., Fleury, A., Vacher, M & Noury, N (2009) Determining useful sensors for
auto-matic recognition of activities of daily living in health smart home, in Intelligent Data
Analysis in Medicine and Pharmacology (IDAMAP2009), Verona, Italy
Rabiner, L & Luang, B (1996) Digital processing of speech signals, Prentice-Hall.
Renouard, S., Charbit, M & Chollet, G (2003) Vocal interface with a speech memory for
dependent people, Independent Living for Persons with Disabilities pp 15–21.
Rialle, V., Ollivet, C., Guigui, C & Hervé, C (2008) What do family caregivers of alzheimer’s
disease patients desire in health smart home technologies? contrasted results of a
wide survey, Methods of Information in Medicine 47: 63–69.
Saeys, Y., Inza, I & Larrañaga, P (2007) A review of feature selection techniques in
bioinfor-matics, Bioinformatics 23: 2507–2517.
Soo, J.-S & Pang, K (1990) Multidelay block frequency domain adaptive filter, IEEE Trans on
Acoustics, Speech and Signal Processing 38(2): 373–376.
Takahashi, S.-y., Morimoto, T., Maeda, S & Tsuruta, N (2003) Dialogue experiment for elderly
people in home health care system, Text Speech and Dialogue (TSD) 2003.
Tran, Q T & Mynatt, E D (2003) What was i cooking? towards déjà vu displays of everyday
memory, Technical report.
Vacher, M., Fleury, A., Serignat, J.-F., Noury, N & Glasson, H (2008) Preliminary
evalua-tion of speech/sound recognievalua-tion for telemedicine applicaevalua-tion in a real environment,
The 9thAnnual Conference of the International Speech Communication Association, SPEECH’08 Proceedings, Brisbane, Australia, pp 496–499.
INTER-Vacher, M., Serignat, J.-F & Chaillol, S (2007) Sound classification in a smart room
environ-ment: an approach using GMM and HMM methods, Advances in Spoken Language Technology, SPED 2007 Proceedings, Iasi, Romania, pp 135–146.
Vacher, M., Serignat, J.-F., Chaillol, S., Istrate, D & Popescu, V (2006) Speech and sound
use in a remote monitoring system for health care, Lecture Notes in Computer Science, Artificial Intelligence, Text Speech and Dialogue, vol 4188/2006, Brno, Czech Republic,
pp 711–718
Valin, J.-M (2007) On adjusting the learning rate in frequency domain echo cancellation with
double talk, IEEE Trans on Acoustics, Speech and Signal Processing 15(3): 1030–1034.
Valin, J.-M & Collings, I B (2007) A new robust frequency domain echo canceller with
closed-loop learning rate adaptation, IEEE Int Conference on Acoustics, Speech and nal Processing, ICASSP’07 Proceedings Vol 1, Honolulu, Hawaii, USA, pp 93–96 Vaseghi, S V (1996) Advanced Signal Processing and Digital Noise Reduction, 1996.
Sig-Vaufreydaz, D., Bergamini, C., Serignat, J.-F., Besacier, L & Akbar, M (2000) A new
method-ology for speech corpora definition from internet documents, LREC’2000, 2nd Int Conference on Language Ressources and Evaluation, Athens, Greece, pp 423–426.
Wang, J.-C., Lee, H.-P., Wang, J.-F & Lin, C.-B (2008) Robust environmental sound
recogni-tion for home automarecogni-tion, IEEE Trans on Automarecogni-tion Science and Engineering 5(1): 25–
31
Wilpon, J & Jacobsen, C (1996) A study of speech recognition for children and the elderly,
IEEE Int Conference on Acoustics, Speech and Signal Processing, pp 349–352.
Trang 3Complete Sound and Speech Recognition System for Health Smart Homes: Application to the Recognition of Activities of Daily Living 673
Istrate, D., Vacher, M & Serignat, J.-F (2008) Embedded implementation of distress
situa-tion identificasitua-tion through sound analysis, The Journal on Informasitua-tion Technology in
Healthcare 6(3): 204–211.
Katz, S & Akpom, C (1976) A measure of primary sociobiological functions, International
Journal of Health Services 6(3): 493–508.
Kröse, B., van Kasteren, T., Gibson, C & van den Dool, T (2008) Care: Context awareness in
residences for elderly, Int Conference of the Int Soc for Gerontechnology, Pisa, Tuscany,
Italy
Kumiko, O., Mitsuhiro, M., Atsushi, E., Shohei, S & Reiko, T (2004) Input support for elderly
people using speech recognition, IEIC Technical Report 104(139): 1–6.
LeBellego, G., Noury, N., Virone, G., Mousseau, M & Demongeot, J (2006) A model for
the measurement of patient activity in a hospital suite, IEEE Trans on Information
Technology in Biomedicine 10(1): 92–99.
Litvak, D., Zigel, Y & Gannot, I (2008) Fall detection of elderly through floor vibrations and
sound, Proc 30th Annual Int Conference of the IEEE-EMBS 2008, pp 4632–4635.
Maunder, D., Ambikairajah, E., Epps, J & Celler, B (2008) Dual-microphone sounds of daily
life classification for telemonitoring in a noisy environment, Proc 30th Annual
Inter-national Conference of the IEEE-EMBS 2008, pp 4636–4639.
Michaut, F & Bellanger, M (2005) Filtrage adaptatif : théorie et algorithmes, Lavoisier.
Moore, D & Essa, I (2002) Recognizing multitasked activities from video using
stochas-tic context-free grammar, Proc of American Association of Artificial Intelligence (AAAI)
Conference 2002, Alberta, Canada.
Niessen, M., Van Maanen, L & Andringa, T (2008) Disambiguating sounds through context,
Proc Second IEEE International Conference on Semantic Computing, pp 88–95.
Noury, N., Hadidi, T., Laila, M., Fleury, A., Villemazet, C., Rialle, V & Franco, A (2008) Level
of activity, night and day alternation, and well being measured in a smart hospital
suite, Proc 30th Annual Int Conference of the IEEE-EMBS 2008, pp 3328–3331.
Noury, N., Villemazet, C., Barralon, P & Rumeau, P (2006) Ambient multi-perceptive
sys-tem for residential health monitoring based on electronic mailings experimentation
within the AILISA project, Proc 8th Int Conference on e-Health Networking, Applications
and Services HEALTHCOM 2006, pp 95–100.
Popescu, M., Li, Y., Skubic, M & Rantz, M (2008) An acoustic fall detector system that
uses sound height information to reduce the false alarm rate, Proc 30th Annual Int.
Conference of the IEEE-EMBS 2008, pp 4628–4631.
Portet, F., Fleury, A., Vacher, M & Noury, N (2009) Determining useful sensors for
auto-matic recognition of activities of daily living in health smart home, in Intelligent Data
Analysis in Medicine and Pharmacology (IDAMAP2009), Verona, Italy
Rabiner, L & Luang, B (1996) Digital processing of speech signals, Prentice-Hall.
Renouard, S., Charbit, M & Chollet, G (2003) Vocal interface with a speech memory for
dependent people, Independent Living for Persons with Disabilities pp 15–21.
Rialle, V., Ollivet, C., Guigui, C & Hervé, C (2008) What do family caregivers of alzheimer’s
disease patients desire in health smart home technologies? contrasted results of a
wide survey, Methods of Information in Medicine 47: 63–69.
Saeys, Y., Inza, I & Larrañaga, P (2007) A review of feature selection techniques in
bioinfor-matics, Bioinformatics 23: 2507–2517.
Soo, J.-S & Pang, K (1990) Multidelay block frequency domain adaptive filter, IEEE Trans on
Acoustics, Speech and Signal Processing 38(2): 373–376.
Takahashi, S.-y., Morimoto, T., Maeda, S & Tsuruta, N (2003) Dialogue experiment for elderly
people in home health care system, Text Speech and Dialogue (TSD) 2003.
Tran, Q T & Mynatt, E D (2003) What was i cooking? towards déjà vu displays of everyday
memory, Technical report.
Vacher, M., Fleury, A., Serignat, J.-F., Noury, N & Glasson, H (2008) Preliminary
evalua-tion of speech/sound recognievalua-tion for telemedicine applicaevalua-tion in a real environment,
The 9thAnnual Conference of the International Speech Communication Association, SPEECH’08 Proceedings, Brisbane, Australia, pp 496–499.
INTER-Vacher, M., Serignat, J.-F & Chaillol, S (2007) Sound classification in a smart room
environ-ment: an approach using GMM and HMM methods, Advances in Spoken Language Technology, SPED 2007 Proceedings, Iasi, Romania, pp 135–146.
Vacher, M., Serignat, J.-F., Chaillol, S., Istrate, D & Popescu, V (2006) Speech and sound
use in a remote monitoring system for health care, Lecture Notes in Computer Science, Artificial Intelligence, Text Speech and Dialogue, vol 4188/2006, Brno, Czech Republic,
pp 711–718
Valin, J.-M (2007) On adjusting the learning rate in frequency domain echo cancellation with
double talk, IEEE Trans on Acoustics, Speech and Signal Processing 15(3): 1030–1034.
Valin, J.-M & Collings, I B (2007) A new robust frequency domain echo canceller with
closed-loop learning rate adaptation, IEEE Int Conference on Acoustics, Speech and nal Processing, ICASSP’07 Proceedings Vol 1, Honolulu, Hawaii, USA, pp 93–96 Vaseghi, S V (1996) Advanced Signal Processing and Digital Noise Reduction, 1996.
Sig-Vaufreydaz, D., Bergamini, C., Serignat, J.-F., Besacier, L & Akbar, M (2000) A new
method-ology for speech corpora definition from internet documents, LREC’2000, 2nd Int Conference on Language Ressources and Evaluation, Athens, Greece, pp 423–426.
Wang, J.-C., Lee, H.-P., Wang, J.-F & Lin, C.-B (2008) Robust environmental sound
recogni-tion for home automarecogni-tion, IEEE Trans on Automarecogni-tion Science and Engineering 5(1): 25–
31
Wilpon, J & Jacobsen, C (1996) A study of speech recognition for children and the elderly,
IEEE Int Conference on Acoustics, Speech and Signal Processing, pp 349–352.
Trang 5New emerging biomedical technologies for home-care and telemedicine applications: the Sensorwear project
Luca Piccini, Oriana Ciani and Giuseppe Andreoni
X
New emerging biomedical technologies for
home-care and telemedicine applications:
the Sensorwear project
Luca Piccini, Oriana Ciani and Giuseppe Andreoni
Politecnico di Milano, INDACO Department
Italy
1 Introduction
The Grey Booming phenomenon is one of the major issues indicated by the European Union
as a problem to be analysed and faced by the Seventh Framework Programme (FP7)
Statistics highlighted that elderly people (over 65 years old) should double in the next 40
years The medical and health care to such an ‘older’ society means growing expenditures
for the UE national health systems, which already amount to significant percentages of the
Gross Domestic Product (GDP) in the different countries The UE Healthcare Systems risk to
collapse if strong countermeasures will not be undertaken Agreeing with this assumption,
the European Commission included among its priorities the stimuli to deeply remodel the
national healthcare systems France, United Kingdom, Holland, Austria, Italy and other
countries drafted national programs in order to face this emerging problem More in detail,
cardiac and respiratory diseases have been identified as some of the most frequent causes of
hospitalization; telemedicine and home-care have been therefore selected to face the
negative evolution of these pathologies, both in clinical and economical terms, assuring
domestic assistance for older people as well as disabled or chronic patients The rationale of
this choice is the opportunity of reducing the overall costs while maintaining high quality of
care and providing an easy access to care from any place, at any time Moreover the focus of
healthcare consequently shifts from treatment to prevention and early diagnosis, thanks to
the contribution of parallel wellness programs, too
Increasing the impact of home-care solutions is a difficult challenge, since technological
issues, such as biosignals monitoring, data communications and basic automated signal
analysis coexist with the efforts to improve new technologies’ acceptability by the patients,
who need to interact with them for long time Generally these users are not technologically
skilled therefore textile sensors platforms represent an ideal way to develop the
telemedicine approach
Under these perspectives, the research and development of Wearable Health Systems
(WHS) become even relevant They are expected to play a significant role on the spreading
of ‘extra-hospital’ cares, thus improving the national health policies effectiveness and the
citizens’ quality of life, too
34
Trang 6WHS are integrated systems on body-worn platforms, such as wrist-worn devices or biomedical clothes, offering pervasive solutions for continuous health status monitoring trough non-invasive biomedical, biochemical and physical measurements (Lymberis & Gatzoulis, 2006) In other words, they provide not only a remote monitoring platform for prevention and early diagnosis, but also a valid contribution to disease management and support of elderly or people in need; in particular, they enable multi-parametric monitoring including body-kinematics, vital signs, biochemical as well as emotional and sensorial parameters in a defined social and environmental context
The integration of electronics and clothing is an emerging field which aims to the development of multi-functional, wearable electro-textiles for applications together with body functions monitoring, actuation, communication, data transfer and individual environment control Furthermore, the integration of advanced microsystems at the fibre core, in conjunction with user interfaces, power sources and embedded software, make R&D
in this field extremely challenging Moreover, current research is dealing with the development of stretchable conductive patterns and soft-touch substrates for component textile mounting and interconnection
As a matter of fact, WHS cope with a variety of challenging topics, whose complexity increases with their integration: wireless communication, power supply and management, data processing, new algorithms for biosignal analysis, connection, sensors’ cleaning and stability over time and external conditions, sensors positioning on the human body, user’s interface, garment’s elasticity and adherence to the skin and other minor themes
Surely the first issue to be managed is the technological one - current state of the art has achieved a good level of maturity to be industrialized and brought to the market - but another key factor, that is still not mature enough, is the ergonomic or human factor in terms
of device’s usability, comfort and acceptance by the end user According to the authors,
design for wearability is necessary for the real and definitive acknowledgment of WHS in
clinical applications, telemedicine and more (Andreoni, 2008) That’s the reason why, besides the main objectives of developing healthcare wearable devices, meeting the aforementioned requirements for enhanced user-friendliness, affordability and unobtrusive monitoring in several clinical applications is becoming a growing topic of worldwide research about WHS
In order to let WHS regularly break into the healthcare practice this and other issues should
be solved, for example, from the commercial and industrial point of view, the consolidation
of R&D results in different domains and their integration (David, 2007) The Sensorwear
project tries to organically coordinate the emerging technologies in the field of wearable biomedical devices, conductive yarns or garments, embedded monitoring devices, automated alarm systems and ICT channels optimizations, in order to design a complete, automated service for home and clinical cardiac monitoring applications
2 The international scenario of wearable telemonitoring systems
Wearable solutions for biophysical conditions monitoring can address many of the emerging issues previously described for a broad cross-section of user groups Elderly care and disease management are just the immediate application, in addition to wellness and sport which represent significant segments that can benefit from continuous, remote and personal monitoring solutions
Trang 7WHS are integrated systems on body-worn platforms, such as wrist-worn devices or
biomedical clothes, offering pervasive solutions for continuous health status monitoring
trough non-invasive biomedical, biochemical and physical measurements (Lymberis &
Gatzoulis, 2006) In other words, they provide not only a remote monitoring platform for
prevention and early diagnosis, but also a valid contribution to disease management and
support of elderly or people in need; in particular, they enable multi-parametric monitoring
including body-kinematics, vital signs, biochemical as well as emotional and sensorial
parameters in a defined social and environmental context
The integration of electronics and clothing is an emerging field which aims to the
development of multi-functional, wearable electro-textiles for applications together with
body functions monitoring, actuation, communication, data transfer and individual
environment control Furthermore, the integration of advanced microsystems at the fibre
core, in conjunction with user interfaces, power sources and embedded software, make R&D
in this field extremely challenging Moreover, current research is dealing with the
development of stretchable conductive patterns and soft-touch substrates for component
textile mounting and interconnection
As a matter of fact, WHS cope with a variety of challenging topics, whose complexity
increases with their integration: wireless communication, power supply and management,
data processing, new algorithms for biosignal analysis, connection, sensors’ cleaning and
stability over time and external conditions, sensors positioning on the human body, user’s
interface, garment’s elasticity and adherence to the skin and other minor themes
Surely the first issue to be managed is the technological one - current state of the art has
achieved a good level of maturity to be industrialized and brought to the market - but
another key factor, that is still not mature enough, is the ergonomic or human factor in terms
of device’s usability, comfort and acceptance by the end user According to the authors,
design for wearability is necessary for the real and definitive acknowledgment of WHS in
clinical applications, telemedicine and more (Andreoni, 2008) That’s the reason why,
besides the main objectives of developing healthcare wearable devices, meeting the
aforementioned requirements for enhanced user-friendliness, affordability and unobtrusive
monitoring in several clinical applications is becoming a growing topic of worldwide
research about WHS
In order to let WHS regularly break into the healthcare practice this and other issues should
be solved, for example, from the commercial and industrial point of view, the consolidation
of R&D results in different domains and their integration (David, 2007) The Sensorwear
project tries to organically coordinate the emerging technologies in the field of wearable
biomedical devices, conductive yarns or garments, embedded monitoring devices,
automated alarm systems and ICT channels optimizations, in order to design a complete,
automated service for home and clinical cardiac monitoring applications
2 The international scenario of wearable telemonitoring systems
Wearable solutions for biophysical conditions monitoring can address many of the
emerging issues previously described for a broad cross-section of user groups Elderly care
and disease management are just the immediate application, in addition to wellness and
sport which represent significant segments that can benefit from continuous, remote and
personal monitoring solutions
During the last years, different research projects all over the European Union were dedicated to the creation of telemonitoring systems based on wearable or standard sensors MyHeart is one of the most important and complete among them Notwithstanding the relevant efforts that have been made since 2000 by the granted projects of the Seventh Framework Program (FP7), researchers and industries are still trying to improve patients’ condition monitoring at home using unobtrusive sensors built into everyday objects able to automatically report to clinicians1 (Lymberys & De Rossi, 2004) These examples and other projects demonstrated both the importance of such applications and the technological problems related to the creation of such a systems
On the other side, pilot studies were lead in order to evaluate the potential impact of care monitoring in terms of costs through a comparison with standard instrumentation The
home-EU Commission, in fact, has underlined the economical potentialities of such solutions, but also has pointed out doubts with the achievement of the potential results and the effective introduction of these technologies in the healthcare systems (COM 689, 2008)
All the predictive models, analysis and studies confirmed the importance of the wearable telemonitoring scenario, but many problems occur if one aims to the implementation of an industrial project and not only to a research prototype (Lymberis & Paradiso, 2008)
The Sensorwear project tries to avoid the segmentation of technologies and competences, concentrating a small, skilled group of people for the creation of a wearable, unobtrusive, low cost and fully automated solution whose usability, reliability and release of brief information are the most peculiar qualities
The market analysis has shown there are no commercial solutions able to assure those requirements with a complete wearable system for daily clinical monitoring It is not uncommon reading about prototypes or finding patents about wearable systems for health care and catching poor information coming from military applications context, not accessible by definition (Pantepopulous & Bourbakis, 2008) To date, the main companies involved in the development of wearable monitoring systems such as Body Media Inc., Sensatex Inc., Textronics Inc and Vivometrics Inc., experience every day the need for more consistent and remote monitoring of individuals for a variety of purpose: from elderly care
to chronic disease management and others Their solutions are just beginning the transition from the development phase into commercialization, facing the barrier of the regulatory approval, which remains critical for many of the producers
Just to give an example, a common electrocardiograph, the instrument allowing the execution of an electrocardiogram exam, costs about 600€ in the UE market and cannot be used with wearable sensors to provide unobtrusive measures The paradigm of measures transparency requires solutions’ refinement or improvement or the design of new integrated systems when noise, artefacts or ergonomic deficiencies enlarge
The Sensorwear system points at achieving these crucial objectives
1 For more details, go to: http://heartcycle.med.auth.gr and http://www.ehealthnews.eu
Trang 83 The Sensorwear project
The Sensorwear project focuses on design and development of a low-cost, industrial solution for smart home-monitoring and hospital applications The objective is not to create
a life-support system, but a reliable, cost effective solution able to monitor biosignals detecting specific conditions requested by clinicians and to transmit them consequently through a long-range communication channel
The project is granted by the Regione Lombardia and it involves the Politecnico di Milano - INDACO Department -, three technological partners (STMicroelectronics, Microsystems and SXT-Sistemi per Telemedicina), one clothes manufacturer (MCS – Manifatture Cotoniere Settentrionali), a service provider company for the textile and clothing sector (Centro Tessile Cotoniero) and the Mater Domini Hospital in Castellanza (IT), the project’s clinical partner
We will illustrate the main aspects related to the project’s objectives, technical solutions, applications and expected results in the following paragraphs
3.1 Objectives and overall architecture
The Sensorwear project aims at developing a complete home monitoring service able to collect a set of different biosignals in a transparent way during the spontaneous activity of the subjects: this paradigm is known as unobtrusive measure An important part of the project is the creation of a Body Sensor Network (BSN) dedicated to the health state monitoring trough record, process and transmission of the biosignals and some useful parameters obtained from them BSN is mainly based on wearable sensors for the collection
of biopotentials (like the electrocardiographic signals, the ECG) and integrated and miniaturized electronic solution based on Bluetooth® technology
The detailed objectives of the project are (Fig 1):
Research, development and production of a System in Package (SIP) solution for monitoring, processing and transmission
Research, development and production of embedded sensors
Creation of fully featured t-shirts with integrated SIP devices to be tested and used both at home and in hospital
Development of software and algorithms for the processing and management of signals, data and alarm for the different applications
Development of software for remote data receiving and database integration
In order to fulfil those items, the fundamental point of the product industrialisation, which
is a peculiarity of the Sensorwear project, is continuously kept into consideration In this way, the final solution is expected to be compliant with the specifications for medical devices of class IIa Garments’ testing, which is an ongoing concern, is an unavoidable step
in order to ensure biocompatibility
The architecture of the system is essentially composed by four main systems:
1 t-shirt with embedded electrodes for the collection of bio-potentials
2 preconditioning and acquisition system
3 processing and transmission device
4 remote data management software
The second and third systems compose a body gateway able to directly control a mobile phone without requiring the user interaction Actually, the possibility to act in a fully automated way is another significant feature of the Sensorwear device
Trang 93 The Sensorwear project
The Sensorwear project focuses on design and development of a low-cost, industrial
solution for smart home-monitoring and hospital applications The objective is not to create
a life-support system, but a reliable, cost effective solution able to monitor biosignals
detecting specific conditions requested by clinicians and to transmit them consequently
through a long-range communication channel
The project is granted by the Regione Lombardia and it involves the Politecnico di Milano -
INDACO Department -, three technological partners (STMicroelectronics, Microsystems and
SXT-Sistemi per Telemedicina), one clothes manufacturer (MCS – Manifatture Cotoniere
Settentrionali), a service provider company for the textile and clothing sector (Centro Tessile
Cotoniero) and the Mater Domini Hospital in Castellanza (IT), the project’s clinical partner
We will illustrate the main aspects related to the project’s objectives, technical solutions,
applications and expected results in the following paragraphs
3.1 Objectives and overall architecture
The Sensorwear project aims at developing a complete home monitoring service able to
collect a set of different biosignals in a transparent way during the spontaneous activity of
the subjects: this paradigm is known as unobtrusive measure An important part of the
project is the creation of a Body Sensor Network (BSN) dedicated to the health state
monitoring trough record, process and transmission of the biosignals and some useful
parameters obtained from them BSN is mainly based on wearable sensors for the collection
of biopotentials (like the electrocardiographic signals, the ECG) and integrated and
miniaturized electronic solution based on Bluetooth® technology
The detailed objectives of the project are (Fig 1):
Research, development and production of a System in Package (SIP) solution for
monitoring, processing and transmission
Research, development and production of embedded sensors
Creation of fully featured t-shirts with integrated SIP devices to be tested and used
both at home and in hospital
Development of software and algorithms for the processing and management of
signals, data and alarm for the different applications
Development of software for remote data receiving and database integration
In order to fulfil those items, the fundamental point of the product industrialisation, which
is a peculiarity of the Sensorwear project, is continuously kept into consideration In this
way, the final solution is expected to be compliant with the specifications for medical
devices of class IIa Garments’ testing, which is an ongoing concern, is an unavoidable step
in order to ensure biocompatibility
The architecture of the system is essentially composed by four main systems:
1 t-shirt with embedded electrodes for the collection of bio-potentials
2 preconditioning and acquisition system
3 processing and transmission device
4 remote data management software
The second and third systems compose a body gateway able to directly control a mobile
phone without requiring the user interaction Actually, the possibility to act in a fully
automated way is another significant feature of the Sensorwear device
Fig 1 Sensorwear main activities: different tasks and their relationships
The signals identified for the specific purpose of the telecardiology application are:
Three ECG leads
Body movement
Respiratory frequency
Cardiac output monitoring
The ECG signal is the most important one allowing the device to detect useful parameters like Heart Rate (HR), arrhythmias and their classification, ST line anomalies ECG is a primary source of indications about health condition, so it receives, at least at early stages, greater attention
The body movement is recorded through a three-axial accelerometer, whose properly processed signals allow determining the number of steps and the body position with respect
to the earth gravity
Furthermore, it is possible to detect the changing of the cardiac output during the day and also the respiratory movements through impedance cardiography measure (ICG)
3.2 Technological key-point and main issues
The main objectives of this project deal with the creation of an industrial, compact, easy to use, automated solution designed with a special attention to elderly people These
INDUSTRIAL SPECIFICATIONS
& FEASIBILITY
MATERIALS &
METHODS for the WEARABLE COMPONENTS PRODUCTION
SIP DESIGN &
PROTOTYPING SERVICES & SYSTEM
Clinical Evaluation
R&D AND INDUSTRIALIZATION
VALIDATION
Trang 10demanding requirements are addressed by the different skills of the partners on yarns and textile solutions, electronic design and production, data collection and databases
WEARABLE SENSORS
The t-shirts were designed in order to facilitate the integration of sensors during the industrialization and to assure the best sensors’ positioning for ECG and ICG signals quality The design of garments is a crucial point in the field of unobtrusive measures, in fact as previous research projects and studies have evidenced, it needs configurations able to reduce the effects of movements, without impacting the comfort The testing phase for the t-shirt, their sensors and sensors’ position has started with the ECG signal check, following a specific protocol First of all the signals are recorded with the first prototype and standard electrode in the Einthoven's configuration, afterwards the device has to collect signals through the t-shirt The last scheduled test requires to connect the prototype to standard electrodes but placed in the same positions of the wearable ones Each recording is done 3 minutes at rest and 3 minutes into action
ERGONOMIC and MECHANICAL ASPECTS
As far as the design of the t-shirts and the adherence of sensors affect the quality of the ECG and ICG signals, the enclosure of the device and its connection to the sensors pathways strongly impact on both the usability and the industrial sustainability Our analysis of the production process and its related constraints evidenced the necessity to conceive custom boxes in order to create a real comfortable solution without renouncing to an appealing product The enclosure will also include the visual signalling with yellow and green leds, compliant to the specifications for Holter medical devices (Fig 2)
Moreover the custom case can be inserted in a docking station, directly sewed to the t-shirt and including the sensors connectors
At this purpose, a custom solution is not a cost-effective one, while the use of the docking station as mating support can allow the choice of stable, reliable although simple and cheaper connectors
Fig 2 The ergonomic study for the user interface and the device shape as regards wearability issues
Trang 11demanding requirements are addressed by the different skills of the partners on yarns and
textile solutions, electronic design and production, data collection and databases
WEARABLE SENSORS
The t-shirts were designed in order to facilitate the integration of sensors during the
industrialization and to assure the best sensors’ positioning for ECG and ICG signals
quality The design of garments is a crucial point in the field of unobtrusive measures, in
fact as previous research projects and studies have evidenced, it needs configurations able to
reduce the effects of movements, without impacting the comfort The testing phase for the
t-shirt, their sensors and sensors’ position has started with the ECG signal check, following a
specific protocol First of all the signals are recorded with the first prototype and standard
electrode in the Einthoven's configuration, afterwards the device has to collect signals
through the t-shirt The last scheduled test requires to connect the prototype to standard
electrodes but placed in the same positions of the wearable ones Each recording is done 3
minutes at rest and 3 minutes into action
ERGONOMIC and MECHANICAL ASPECTS
As far as the design of the t-shirts and the adherence of sensors affect the quality of the ECG
and ICG signals, the enclosure of the device and its connection to the sensors pathways
strongly impact on both the usability and the industrial sustainability Our analysis of the
production process and its related constraints evidenced the necessity to conceive custom
boxes in order to create a real comfortable solution without renouncing to an appealing
product The enclosure will also include the visual signalling with yellow and green leds,
compliant to the specifications for Holter medical devices (Fig 2)
Moreover the custom case can be inserted in a docking station, directly sewed to the t-shirt
and including the sensors connectors
At this purpose, a custom solution is not a cost-effective one, while the use of the docking
station as mating support can allow the choice of stable, reliable although simple and
For what concerns the electronic design, the two main topics are the miniaturization of the circuitry and the reduction of power consumption in order to reach at least 5 days of continuous working with small, commercial batteries Our technological partners already designed and developed products or prototypes able to collect the proposed signals through wearable sensors in a reliable way, but they need to be improved since all the solutions are not optimized in terms of power consumption and scalability The design of a SIP solution requires the refinement of previous solutions and the research of new components that can
be integrated with it The choice of new elements is one of the strategies taken to achieve a low-power design, even if it implies to test again the performance of the system in terms of signal quality, signal-to-noise ratio, drifts and all the parameters influencing the compliance with the medical specifications The final design, the list of components and the features of the system as output of the whole project will be released after the completion of the ICG tests To date, the logical structure of the prototype in use is described in Fig 3
Signals are real-time recorded and processed in order to extract parameters relevant for the clinicians who will receive them through the remote server Dialling, connection and authentication procedures are directly controlled by the wearable unit, thus excluding any user intervention The processing output is directly stored in the remote database The use
of a new generation of 32-bit microcontroller unit (MCU) allows the management of the entire process, optimizing the power consumption at the same time
Different strategies of power management are investigated for each working condition
In fact there are two different situations:
- the normal one, when only brief parameters are transmitted, once in every minute;
- the “alarm” condition, during which also the raw signals are transmitted
This latter configuration allows a prompt analysis of the ECG by the clinicians, who can decide to reject the alarm or to activate countermeasures
This fully automated model of service is based on the possibility to identify different critical conditions from the biosignals applying the rules provided by the clinicians and embedded
in the CPU After the trigger of a possible critical situation has launched, the DUN Bluetooth profile tries to directly connect itself to the remote server and to transmit the raw signals, beginning from the past last minute, until the remote operator will decide to stop the “alarm condition” Its management implies the possibility to use also GPRS data transmission, because the 3G network coverage is still not ensured in all the neighbouring areas As a consequence, we are also exploring data compression algorithms for raw signals transmission in case of poor mobile network coverage
Trang 12Fig 3 The electronic device, the SIP components are evidenced
SERVER and WEB-SERVICE
The hospital security model generally denies the possibility to directly connect a remote device to a local server, physically and logically placed inside the hospital, according to both the Italian and other countries regulation On the same purpose, it is worth noting that it is not mandatory placing the main data collector inside the hospital network, in fact through a secure web-service application the operators can access data while being in the hospital, or more precisely, in a control room where the physiological parameters and alarms can be constantly observed by them The use of different servers in order to cover the involved area
in a better way and to create a local node to distribute the collision management overhead in case of multiple alarm conditions is thus made possible The problem of priority management could not be faced at the mobile phone level because a typical mobile service provider can grant dedicated server resources only with business contracts The mobile phone is just a transparent modem connected to the server The policy in case of a coincidence of alarms is a matter of debate, but speaking from the informatics point of view,
as soon as the connection mobile-server is established through the standard port, the link should be automatically turned on a specific port, in order to set free the common resource for the next alarm
BT Module
3-AXIS Accelerometer
SIP
Trang 13Fig 3 The electronic device, the SIP components are evidenced
SERVER and WEB-SERVICE
The hospital security model generally denies the possibility to directly connect a remote
device to a local server, physically and logically placed inside the hospital, according to both
the Italian and other countries regulation On the same purpose, it is worth noting that it is
not mandatory placing the main data collector inside the hospital network, in fact through a
secure web-service application the operators can access data while being in the hospital, or
more precisely, in a control room where the physiological parameters and alarms can be
constantly observed by them The use of different servers in order to cover the involved area
in a better way and to create a local node to distribute the collision management overhead in
case of multiple alarm conditions is thus made possible The problem of priority
management could not be faced at the mobile phone level because a typical mobile service
provider can grant dedicated server resources only with business contracts The mobile
phone is just a transparent modem connected to the server The policy in case of a
coincidence of alarms is a matter of debate, but speaking from the informatics point of view,
as soon as the connection mobile-server is established through the standard port, the link
should be automatically turned on a specific port, in order to set free the common resource
for the next alarm
Bat
BT Module
3-AXIS Accelerometer
SIP
4 Preliminary results and conclusions
During one year and half of work the consortium faced the problems related to the industrialisation and certification of the product Since some partners were already involved
in such area of the market, preliminary operations like market survey or patent analysis were almost ready at the early stage of the project This way it was possible to promptly release the main specifications for the system, although the industrialisation process is still a work in progress In fact the targeted customers population, predominantly composed by not-or-less technology skilled people, requires a detailed analysis on specific components responsible for the usability and allowing a user-friendly system’s management and handling
As anticipated, we are currently addressing the testing phase on the t-shirts in terms of biocompatibility and clinical performances
Regarding the hardware, we are testing the low-scaled device The SIP will be the final result of the project because the design and production of a SIP system is an expensive and complex process, requiring a lot of efforts in order to reduce the risk of a major fault For this reason we are carrying out specific tests collecting data on performances and trying to understand potential criticism before starting the first production
Current status Industrialisation
problems specifications [%] Compliance with
validating 2 nd release Sensors embedding and connectors 70%
T-shirts 100%
consolidation Refine power management 80%
Table 1 Current project checkout list
During the last months, the firmware has been tested on the same devices (MCU and Bluetooth module) in terms of power consumption and transmission throughput Based on current measures, we forecast it will be possible to ensure a 5-days working time with a commercial Lithium-Ion or Lithium-Polymer battery with less than 500mAh of capacity with the transmission protocol already working
Although there are still some critical aspects highlighted in Table 1, several problems related
to the creation of a commercial unobtrusive and fully automated wearable monitoring solution have been solved Moreover a great boost to the project has come from the introduction of a System in Package solution, the heart of the electronic device, which could probably have a deep impact also in next products and projects
Trang 145 References
Andreoni G (2008), Sistemi di sensori indossabili per il monitoraggio: Dalla Ricerca al Mercato, In:
Bonfiglio A., Cerutti S., De Rossi D., Magenes G (eds.), Sistemi Indossabili Intelligenti per la salute e la protezione dell’uomo, Patron, 2008
COM 689 (2008) Communication from the commission to the European Parliament, the
Council, the European Economic and Social Committes and the Committee of the Regions on telemedicine for the benefit of patients, healthcare systems and society,
COMMISSION OF THE EUROPEAN COMMUNITIES, Brussels
David K (2007), Wearable Electronics Systems Global Market Demand Analysis: Health Care
Solutions, In: VDC Research Report # VDC6520
Lymberis A and De Rossi D (2004) Wearable eHealth Systems for Personalised Health
Management State of the Art and Future Challenges, IOS Press, ISBN I 58603 449 9,
Netherlands
Lymberis A., Gatzoulis L (2006), Wearable Health Systems: from smart technologies to real
applications Conf Proc IEEE Eng Med Biol Soc.: 6789-92
Lymberis A and Paradiso R (2008) Smart Fabrics and Interactive Textile Enabling
Wearable Personal Applications: R&D State of the Art and Future Challenges,
Proceedings of 30th Annual International IEEE EMBS Conference, pp 5270-5273,
Vancouver, British Columbia, August 2008, Canada
Pantepopulous A and Bourbakis N (2008) A Survey on Wearable Biosensor Systems for
Health Monitoring, Proceedings of 30th Annual International IEEE EMBS Conference,
pp 4887-4890, Vancouver, British Columbia, August 2008, Canada
Trang 15Neuro-Developmental Engineering: towards early diagnosis of neuro-developmental disorders 685
Neuro-Developmental Engineering: towards early diagnosis of developmental disorders
neuro-Domenico Campolo, Fabrizio Taffoni, Giuseppina Schiavone, Domenico Formica, Eugenio Guglielmelli and Flavio Keller
0
Neuro-Developmental Engineering: towards early
diagnosis of neuro-developmental disorders
1Domenico Campolo, Fabrizio Taffoni, Giuseppina Schiavone, Domenico Formica, Eugenio Guglielmelli and Flavio Keller
Università Campus Bio-Medico
00128 Roma - Italy
Nanyang Technological University
639798 Singapore
1 Introduction
Neuro-Developmental Engineering (NDE) is a new and emerging interdisciplinary research
area at the intersection of developmental neuroscience and bioengineering aiming at
provid-ing new methods and tools for: i) understanding neuro-biological mechanisms of human
brain development; ii)quantitative analysis and modeling of human behavior during
neuro-development; iii)assessment of neuro-developmental milestones achieved by humans from
birth onwards
Main application fields of NDE are:
- New clinical protocols and standards for early diagnosis, functional evaluation and
therapeutic treatments of neuro- developmental disorders;
- New generations of educational, interactive toys which can provide adequate stimuli
and guidance for supporting the physiological neuro-development process
This technology is expected to be also useful in the long term for developing new tools, e.g
toys, which can sustain, in ecological scenarios, the regular development of motor and
cogni-tive abilities of the child, based on a rigorous scientific approach
The long term goal is establishing standards against which development of infants at risk
for neuro-developmental disorders, particularly autism, can be measured, with the aim of
detecting early signs of disturbed development
1.1 Sensori-Motor Integration Deficits in Neurodevelopmental Disorders
Neurodevelopmental disorders such as ASD, ADHD, Tourette syndrome and others are
char-acterized by a genetic basis In this case behavioral analysis, or behavioral phenotyping, will
be instrumental for the analysis of the roles of genes in behavior (Gerlai 2002)
Autism is a behavioral disorder, with onset in childhood, which is characterized by deficits in
three basic domains: social interaction, language and communication, and pattern of interests
There is no doubt that autism has a strong genetic component, and that biological disease
mechanisms leading to autism are already active during foetal development and/or infancy,
35
Trang 16as demonstrated, for example, by the abnormal pattern of brain growth during late foetal and
early postnatal life (see (Keller and Persico 2003), for a review) Autism is usually diagnosed
at the age of 3 years, in many cases after a period of seemingly normal neurological and
behavioral development The diagnosis of autism is purely clinical, there are no laboratory
tests to confirm or disprove the diagnosis It has been recognized that, although typical autism
is not associated with major neurological deficits, autism has characteristic manifestations in
the perceptual and motor domains.
Deficits in the perceptual domain include altered processing and recognition of socially relevant
information from peopleŠs faces (see (Grelotti et al 2003), for a review), deficits in
percep-tion of mopercep-tion cues (Milne et al 2002), (Spencer et al 2000), (Bertone et al 2003), (Takerae
et al 2004), difficulty in disengaging attention (Landry and Bryson 2004) and alterations of
auditory processing (Courchesne et al 1984), (Boddaert et al 2004) Studies based on
analy-sis of home-made movies suggest that an impairment of spontaneous attention toward social
stimuli is present already at 20 months (Swettenham et al 1998), and possibly also as early as
during the first 6 months of life (Maestro et al 2002) Furthermore, an autism-like syndrome is
frequently observed in congenitally blind children (Hobson and Bishop 2003) Taken together,
these observations suggest that at least some individuals with autism are characterized by an
early deficit of ‘low-level’ perceptual processing, which jeopardizes their ability to develop
higher-level capacities, such as language and interpersonal skills
Motor impairments in autism include deficits in postural reflexes (Minshew et al 2004),
(Schmitz et al 2003), (Molloy et al 2003), repetitive, stereotyped movements and awkward
patterns of object manipulation, lack of purposeful exploratory movements (see e.g (Pierce
and Courchesne 2001)), gaze abnormalities (Sweeney et al 2004), unusual gait pattern (Hallett
et al 1993), and alterations of movement planning and execution, which express themselves
as ‘hyper- dexterity’ (Rinehart et al 2001), (Mari et al 2003) Motor abnormalities may be
observed retrospectively in infants who later develop the autistic syndrome, on the basis of
home-made movies made during the first year of life (Teitelbaum et al 1998), (Teitelbaum et
al 2004) These clinical observations are consistent with a large body of evidence of subtle
structural and functional abnormalities of cortical and subcortical neural systems involved
in movement planning and execution, such as the prefrontal cortex, the basal ganglia and the
cerebellum (see (Keller and Persico 2003), for a review)
1.2 Ecological Approach
The diagnosis of ASD is currently made at 3 years of age; Attention-Deficit Hyperactivity
Dis-order (ADHD) is always considered as an alternative diagnosis of “high functioning” autism;
Tourette syndrome is diagnosed at age 7 or later ASD is therefore a natural candidate for
demonstrating the validity of novel approaches to early diagnosis As shown in Fig 1,
in-fancy, i.e the first 2-3 years of life before language development, represents an important
temporal window for an early diagnosis of ASD
The goal of our approach is twofold On one hand, guided by neuroscientists, we develop
technological platforms and methods to extract more information on perceptual and
intersub-jective capacities of human infants than is currently possible; this information could be later
used for early diagnosis of developmental disorders On the other hand, infancy provides
us with an important window of opportunity to capture the mechanisms behind
sensorimo-tor integration as these are just developing Moreover, neurodevelopmental disorders are an
important benchmark to highlight failures within such mechanism Such a knowledge can
be useful to neuroscientists to better understand the human brain functions involved in the
age (years)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Autism ADHD Tourette Syndrome
infancy
current diagnostic tools
window of opportunity
brain development
Fig 1 Current diagnostic tools
sensorimotor integration but also to engineers, providing unique insights on how to buildcomplex and adaptable artificial systems (Metta et al 1999)
2 Technology for Assessing Movement and Gaze 2.1 Motion Tracking
Motion tracking can count on a host of different technological solutions, operating on entirelydifferent physical principles, with different performance characteristics and designed for dif-ferent purposes As shown in (Welch and Foxlin 2002), there is not a single technology thatcan fit all needs Each application defines the best technology to be implemented In order toperform a selection, the main characteristics of available technologies are briefly summarizedhereafter (see (Welch and Foxlin 2002) for details)
Mechanical sensing:
typically used for body motion capture; it uses angle and range measurements with the help
of gears and bend sensors; very accurate but bulky, often limiting mobility.
Optical sensing:
several principles are available, typical systems are camera- based ones; position of markers
in 3D space can be estimated very accurately within working volume (typically a few cubemeters, depending on the number of deployed cameras); line-of- sight issues (i.e the fact thatbody parts or other objects may occlude the visual scene of a camera, losing thus the sight
of one or more markers) is a limiting factor; very expensive; often requires highly structuredenvironments, at least when high accuracy is needed
Acoustic sensing:
typically based on time-of-flight of ultrasound pulses between emitters and receivers; sound
speed in air (about 340 m/s, resulting in sampling periods in the order of a few tens of
mil-liseconds) is slow but still acceptable for sensing human ( in particular infants) movements;line-of-sight issues are not as severe as for the optical technology; requires much less struc-tured environments than optical trackers; suitable to be used in ecological conditions (e.g.kindergartens)
Trang 17Neuro-Developmental Engineering: towards early diagnosis of neuro-developmental disorders 687
as demonstrated, for example, by the abnormal pattern of brain growth during late foetal and
early postnatal life (see (Keller and Persico 2003), for a review) Autism is usually diagnosed
at the age of 3 years, in many cases after a period of seemingly normal neurological and
behavioral development The diagnosis of autism is purely clinical, there are no laboratory
tests to confirm or disprove the diagnosis It has been recognized that, although typical autism
is not associated with major neurological deficits, autism has characteristic manifestations in
the perceptual and motor domains.
Deficits in the perceptual domain include altered processing and recognition of socially relevant
information from peopleŠs faces (see (Grelotti et al 2003), for a review), deficits in
percep-tion of mopercep-tion cues (Milne et al 2002), (Spencer et al 2000), (Bertone et al 2003), (Takerae
et al 2004), difficulty in disengaging attention (Landry and Bryson 2004) and alterations of
auditory processing (Courchesne et al 1984), (Boddaert et al 2004) Studies based on
analy-sis of home-made movies suggest that an impairment of spontaneous attention toward social
stimuli is present already at 20 months (Swettenham et al 1998), and possibly also as early as
during the first 6 months of life (Maestro et al 2002) Furthermore, an autism-like syndrome is
frequently observed in congenitally blind children (Hobson and Bishop 2003) Taken together,
these observations suggest that at least some individuals with autism are characterized by an
early deficit of ‘low-level’ perceptual processing, which jeopardizes their ability to develop
higher-level capacities, such as language and interpersonal skills
Motor impairments in autism include deficits in postural reflexes (Minshew et al 2004),
(Schmitz et al 2003), (Molloy et al 2003), repetitive, stereotyped movements and awkward
patterns of object manipulation, lack of purposeful exploratory movements (see e.g (Pierce
and Courchesne 2001)), gaze abnormalities (Sweeney et al 2004), unusual gait pattern (Hallett
et al 1993), and alterations of movement planning and execution, which express themselves
as ‘hyper- dexterity’ (Rinehart et al 2001), (Mari et al 2003) Motor abnormalities may be
observed retrospectively in infants who later develop the autistic syndrome, on the basis of
home-made movies made during the first year of life (Teitelbaum et al 1998), (Teitelbaum et
al 2004) These clinical observations are consistent with a large body of evidence of subtle
structural and functional abnormalities of cortical and subcortical neural systems involved
in movement planning and execution, such as the prefrontal cortex, the basal ganglia and the
cerebellum (see (Keller and Persico 2003), for a review)
1.2 Ecological Approach
The diagnosis of ASD is currently made at 3 years of age; Attention-Deficit Hyperactivity
Dis-order (ADHD) is always considered as an alternative diagnosis of “high functioning” autism;
Tourette syndrome is diagnosed at age 7 or later ASD is therefore a natural candidate for
demonstrating the validity of novel approaches to early diagnosis As shown in Fig 1,
in-fancy, i.e the first 2-3 years of life before language development, represents an important
temporal window for an early diagnosis of ASD
The goal of our approach is twofold On one hand, guided by neuroscientists, we develop
technological platforms and methods to extract more information on perceptual and
intersub-jective capacities of human infants than is currently possible; this information could be later
used for early diagnosis of developmental disorders On the other hand, infancy provides
us with an important window of opportunity to capture the mechanisms behind
sensorimo-tor integration as these are just developing Moreover, neurodevelopmental disorders are an
important benchmark to highlight failures within such mechanism Such a knowledge can
be useful to neuroscientists to better understand the human brain functions involved in the
age (years)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Autism ADHD Tourette Syndrome
infancy
current diagnostic tools
window of opportunity
brain development
Fig 1 Current diagnostic tools
sensorimotor integration but also to engineers, providing unique insights on how to buildcomplex and adaptable artificial systems (Metta et al 1999)
2 Technology for Assessing Movement and Gaze 2.1 Motion Tracking
Motion tracking can count on a host of different technological solutions, operating on entirelydifferent physical principles, with different performance characteristics and designed for dif-ferent purposes As shown in (Welch and Foxlin 2002), there is not a single technology thatcan fit all needs Each application defines the best technology to be implemented In order toperform a selection, the main characteristics of available technologies are briefly summarizedhereafter (see (Welch and Foxlin 2002) for details)
Mechanical sensing:
typically used for body motion capture; it uses angle and range measurements with the help
of gears and bend sensors; very accurate but bulky, often limiting mobility.
Optical sensing:
several principles are available, typical systems are camera- based ones; position of markers
in 3D space can be estimated very accurately within working volume (typically a few cubemeters, depending on the number of deployed cameras); line-of- sight issues (i.e the fact thatbody parts or other objects may occlude the visual scene of a camera, losing thus the sight
of one or more markers) is a limiting factor; very expensive; often requires highly structuredenvironments, at least when high accuracy is needed
Acoustic sensing:
typically based on time-of-flight of ultrasound pulses between emitters and receivers; sound
speed in air (about 340 m/s, resulting in sampling periods in the order of a few tens of
mil-liseconds) is slow but still acceptable for sensing human ( in particular infants) movements;line-of-sight issues are not as severe as for the optical technology; requires much less struc-tured environments than optical trackers; suitable to be used in ecological conditions (e.g.kindergartens)
Trang 18(Geo)Magnetic sensing:
a first method is based on electromagnetic coupling between a source and several trackers;
main drawbacks are that signal decays as 1/d3 (where d is the source-tracker distance) and
is affected by the geomagnetic field; these devices are quite expensive and require a certain
amount of structuring of the environment A second method is electronic compassing;
esti-mates heading and solely relies on the geomagnetic field, i.e it does not require any artificial
source and is therefore sourceless; measurements can be altered by ferromagnetic influence of
surrounding objects
Inertial sensing:
highly miniaturized accelerometers and gyroscopes are used to sense, respectively,
acceler-ation (comprising the gravitacceler-ational field) and angular velocity; used as inclinometers,
ac-celerometers can sense the gravity vector, i.e the ‘vertical’ direction, in this sense they are
also sourceless.
temperature fluctuations
position orientation
working volume cost
ferromagnetic influence acoustic noise optical noise
Inertial Geomagnetic Acoustic Optical Mechanical
+
- -
+- +-
- -
LEGEND:
tracker size / obstructive
Fig 2 Selection chart of different motion tracking technologies
In Fig 2, a selection chart for the different available technologies is provided For each
avail-able technology (columns) its suitability with respect to the performance characteristics of
interest (rows) is indicated Since our main purpose is developing technological tools that are
either wearable by infants or embeddable into toys, the highest priority is given to
technolo-gies which are unobtrusive This directly leads us to discard solutions involving mechanical
trackers
The second element considered for selection are the line-of-sight issues, since we are going to
deal with infants, it is extremely difficult to perform experiments with technologies that are
limited by the line of sight, a peculiarity of the optical technology which is only suitable to
experiments with collaborative subjects who are somehow willing to ‘act’ in front of a era Line-of-sight issues are much less severe for the acoustic technology which is thus stillappealing for movements analysis in infants
cam-The third element of the selection criterion is performance with respect to tracking accuracy.Here a distinction is made between tracking positions and tracking orientations Measure-ment principles such as the time-of-flight (typically deployed in acoustic measurements) orcamera-based tracking are inherently suitable to measure the distance of points (markers) andthe origin of the measurement system (e.g the source of acoustic waves or a camera etc ).Orientations can be inferred indirectly by estimating distances between two or more mark-ers and the source of measurement The larger the distance between two markers, the betterthe estimation of orientation As dimensions shrink, as in the case of infants, accuracy ofindirect orientation measurements also decreases (e.g accurate tracking of the orientation
of an infant’s wrist can be problematic even without considering line-of-sight issues) Othertechnologies allow a direct measurement of orientations (for example inertial sensors used
as inclinometers can sense deviations from the vertical axis while magnetic sensors used ascompasses can sense deviations from the horizontal geomagnetic north direction) without re-quiring the positioning of multiple markers
As long as orientation is concerned, inertial and magnetic technologies appear to be very pealing since: are highly unobtrusive due the availability of miniaturized off-the-shelf devices;
ap-do not suffer from line-of-sight issues; can provide high accuracy in orientation tracking are
sourceless: do not require any structuring of the environment; have virtually unlimited
work-ing volume; are low-cost
The bottom half of Fig 2 shows, for each technology, the main limiting factors to a correctoperation Besides temperature, which affects any electrical device and that can be compen-sated in most of the cases, the real limiting factor for the magnetic technology is the presence
of ferromagnetic materials Common ferromagnetic objects such as iron parts of doors, chairs,tables etc can produce local distortions of the geomagnetic field, causing thus errors in theestimations of orientations As discussed in (Kemp et al 1998), some care should be taken,when conducting experiments, to avoid large ferromagnetic objects in the surroundings Wefound that this can be easily done in environments such as day-cares where, for safety reasons,all metals are usually avoided and typical materials used with children are wood, rubber andplastic
1 Magnetic Induction Method (Search Coil)
2 Electro-Oculography (EOG)
Trang 19Neuro-Developmental Engineering: towards early diagnosis of neuro-developmental disorders 689
(Geo)Magnetic sensing:
a first method is based on electromagnetic coupling between a source and several trackers;
main drawbacks are that signal decays as 1/d3 (where d is the source-tracker distance) and
is affected by the geomagnetic field; these devices are quite expensive and require a certain
amount of structuring of the environment A second method is electronic compassing;
esti-mates heading and solely relies on the geomagnetic field, i.e it does not require any artificial
source and is therefore sourceless; measurements can be altered by ferromagnetic influence of
surrounding objects
Inertial sensing:
highly miniaturized accelerometers and gyroscopes are used to sense, respectively,
acceler-ation (comprising the gravitacceler-ational field) and angular velocity; used as inclinometers,
ac-celerometers can sense the gravity vector, i.e the ‘vertical’ direction, in this sense they are
also sourceless.
temperature fluctuations
position orientation
working volume cost
ferromagnetic influence acoustic noise
+
- -
+-
-
-
LEGEND:
tracker size / obstructive
Fig 2 Selection chart of different motion tracking technologies
In Fig 2, a selection chart for the different available technologies is provided For each
avail-able technology (columns) its suitability with respect to the performance characteristics of
interest (rows) is indicated Since our main purpose is developing technological tools that are
either wearable by infants or embeddable into toys, the highest priority is given to
technolo-gies which are unobtrusive This directly leads us to discard solutions involving mechanical
trackers
The second element considered for selection are the line-of-sight issues, since we are going to
deal with infants, it is extremely difficult to perform experiments with technologies that are
limited by the line of sight, a peculiarity of the optical technology which is only suitable to
experiments with collaborative subjects who are somehow willing to ‘act’ in front of a era Line-of-sight issues are much less severe for the acoustic technology which is thus stillappealing for movements analysis in infants
cam-The third element of the selection criterion is performance with respect to tracking accuracy.Here a distinction is made between tracking positions and tracking orientations Measure-ment principles such as the time-of-flight (typically deployed in acoustic measurements) orcamera-based tracking are inherently suitable to measure the distance of points (markers) andthe origin of the measurement system (e.g the source of acoustic waves or a camera etc ).Orientations can be inferred indirectly by estimating distances between two or more mark-ers and the source of measurement The larger the distance between two markers, the betterthe estimation of orientation As dimensions shrink, as in the case of infants, accuracy ofindirect orientation measurements also decreases (e.g accurate tracking of the orientation
of an infant’s wrist can be problematic even without considering line-of-sight issues) Othertechnologies allow a direct measurement of orientations (for example inertial sensors used
as inclinometers can sense deviations from the vertical axis while magnetic sensors used ascompasses can sense deviations from the horizontal geomagnetic north direction) without re-quiring the positioning of multiple markers
As long as orientation is concerned, inertial and magnetic technologies appear to be very pealing since: are highly unobtrusive due the availability of miniaturized off-the-shelf devices;
ap-do not suffer from line-of-sight issues; can provide high accuracy in orientation tracking are
sourceless: do not require any structuring of the environment; have virtually unlimited
work-ing volume; are low-cost
The bottom half of Fig 2 shows, for each technology, the main limiting factors to a correctoperation Besides temperature, which affects any electrical device and that can be compen-sated in most of the cases, the real limiting factor for the magnetic technology is the presence
of ferromagnetic materials Common ferromagnetic objects such as iron parts of doors, chairs,tables etc can produce local distortions of the geomagnetic field, causing thus errors in theestimations of orientations As discussed in (Kemp et al 1998), some care should be taken,when conducting experiments, to avoid large ferromagnetic objects in the surroundings Wefound that this can be easily done in environments such as day-cares where, for safety reasons,all metals are usually avoided and typical materials used with children are wood, rubber andplastic
1 Magnetic Induction Method (Search Coil)
2 Electro-Oculography (EOG)
Trang 203 Photoelectric Methods: Infra-Red (IR) Oculography
4 Video-Oculography (VOG)
Each methodology is characterized by parameters such as range of measurement, sensitivity,
linearity, accuracy, discomfort for the subject, interference with the field of view of the subject,
tolerance to head movement
Magnetic Induction Method (Search Coil):
The search coil technique has become the accepted standard for the measurement of 3D eye
movement This technique is based on the fact that a magnetic field induces a voltage in a coil
(search coil) which is attached to the eye The induced voltage has amplitude proportional to
the sine of the angle between the coil axis and the magnetic field direction The magnetic field
is provide by coils mounted at the sides of a cubic frame The dimensions of the sides of the
frame can vary from few tens of centimeters to few meters, allowing to measure also other
movements (i.e eye-hand coordination) Robinson (Robinson 1963) was the first to apply this
technique, using a coil secured to the eye by suction Nowadays the search coil is embedded
in a scleral contact lens The lens is subject to slippage if the lens covers only the cornea Eye
movement is measured in absolute spatial coordinates Head orientation can also be measured
with a search coil mounted on the forehead, and orientation and movement of the eye within
the head can be calculated from the orientation of the head and of the eye with respect to
the magnetic field (Haslwanter 1995) Currently a number of different search coil systems are
commercially available (e.g by Skalar Instruments, C-N-C Engineering, Remmel Labs, etc.)
Although the scleral search coil is the most precise eye movement measurement method (very
high temporal and spatial resolution can be obtained with accuracy to about 5-10 arc-seconds
over a limited range of about 5 deg), it is also the most intrusive method Insertion of the lens
requires care and practice and wearing the lens causes discomfort and risk of corneal abrasion
or lead breakage The requirements to stay in the center of the magnetic field precludes the
use of search coils during many natural activities Thus, this technique is mostly used for
research purposes, it is not suitable for clinical routine
Electro-Oculography:
First applications of electro-oculography are dated back to the ‘30s and are currently widely
used both for clinical and research purposes It relies on measurement of electrical potential
differences between the cornea and the retina, discovered by DuBois-Reymond in 1849 Skin
electrodes are positioned around the eye The measured potential difference is proportional to
the sine of the rotation angle of the eye For small rotation the proportionality is almost linear;
it decreases for higher angles of rotation (Byford 1963) The recorded potentials are in the
range 15-200 µV, with nominal sensitivities of order of 20 µV/deg of eye movement The eye
movement is measured in craniotopic coordinates and head movement during recording does
not affect the measurement The discomfort for the subject is limited and the measurement
range is wide both for horizontal movements (±70 deg) and for vertical movements (±30 deg),
even if the sensitivity decreases for lateral position of the eye The most important advantage
of this methodology is the possibility of recording eye movement with closed eyes, which is
relevant requirement during some experimental protocol (e.g during sleeping phases) The
main drawback of this technique are related to the nature of the potential recorded and to
the artifacts due to the electrodes properties As concern the potential, the resting
corneo-retinic potential (usually of the order of 0.4-1 mV) can be affected by lighting conditions of
the environment and by the psycho-physical condition of the subject The artifacts at the
level of the skin electrodes relies on the contact resistance electrode-skin, on the oxidation andpolarization of the electrodes
Infra-Red Oculography:
Infra-red (IR) oculography is based on the recording of the light reflected by the eye when
it is lighted with IR light beam Since IR light is not visible, it does not interfere with thesubject vision, moreover the IR detectors are not influenced by environmental lighting con-ditions There are three categories of Infra-red (IR) oculography which use respectively: thecorneal reflection, the Purkinje images and the track of the limb Due to the construction ofthe eye, when a beam of IR light points to it, four reflections are formed on the eye, calledPurkinje images (Cornsweet and Crane 1973): the first on the front surface of the cornea and
it is called corneal reflection, the second image on rear surface of the cornea, the third on thefront surface of the lens and the fourth on the rear surface of the lens By detecting the cornealreflection and the pupil center and by using an appropriate calibration procedure, it is possi-ble to measure the Point of Regard (gaze) on a planar surface on which calibration points arepositioned Two points of reference on the eye are needed to separate eye movements fromhead movements The positional difference between the pupil center and corneal reflectionchanges with pure eye rotation, but remains relatively constant with minor head movements.The corneal reflection moves in the opposite direction of the eye respect to the pupil center
In other cases both the first and the fourth Purkinje images (Dual-Purkinje images eye ers are detected Both reflections move together through exactly the same distance upon eyetranslation but they move through different distances upon eye rotation The third methodbased on photoelectric principle relies on the track of the limb (scleral-iris edge) of the eye bymeasuring the amount of scattered light Most photoelectric systems must be mounted close
track-to the eyes (i.e EL-MAR tracking device), so they may restrict the field of view, moreover fastmovements of the head can cause slippage of the device on the head leading to mis-alignment
of the eye respect to the IR emitter and detector There exist also external device and a supportfor keeping the head fixed is needed (i.e Tobii eye tracker) The range of measurement of thephotoelectric eye tracker is not higher then±30 deg in the horizontal plane and±20 deg inthe vertical plane
Video-Oculography:
Video systems for measuring ocular movements are based on the analysis of images recorded
by cameras This technique, introduced in the ‘80s, quickly improved in terms of mances and reliability thanks to the technological development of digital cameras and com-puter powerful The Video Oculography (VOG) provides directly a digital output Severalalgorithms are available for the pupil detection in an image frame and pupil centroid coor-dinates extraction, nevertheless environmental lighting conditions can affect the automaticdetection (Eizenman et al 1984) (Landau 1987) Thus, IR light is used together with videorecording, so that the pupil appears brighter This technique is called Pupil Center/CornealReflection (PC/CR) because the IR light produces also the Purkinje images, mentioned before
perfor-As in IR Oculography, also VOG can be realized both as wearable device (DiScenna 1995) andprovides measurements in craniotopic coordinates or external device and provides measure-ments in spatial coordinates Head-mounted system (i.e EyeLink) can be worn without toomuch discomfort High resolution and high frame rate CCD and CMOS cameras are used.Reduced dimensions and weight of the actual cameras allow to position them in such a waythat they interfere as less as possible with the field of view of the subject (Babcock and Pelz