Apply incremental learning for daily activity recognition using low level sensor data

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF ENGINEERING AND TECHNOLOGY TA VIET CUONG APPLY INCREMENTAL LEARNING FOR DAILY ACTIVITY RECOGNITION USING LOW LEVEL SENSOR DATA MASTER

VIETNAM NATIONAL UNIVERSITY, HANOI

UNIVERSITY OF ENGINEERING AND TECHNOLOGY

TA VIET CUONG

APPLY INCREMENTAL LEARNING FOR DAILY ACTIVITY RECOGNITION USING LOW LEVEL

SENSOR DATA

MASTER THESIS OF INFORMATION TECHNOLOGY

Ha Noi, 2012

VIETNAM NATIONAL UNIVERSITY, HANOI

UNIVERSITY OF ENGINEERING AND TECHNOLOGY

TA VIET CUONG

APPLY INCREMENTAL LEARNING FOR DAILY ACTIVITY RECOGNITION USING LOW LEVEL

SENSOR DATA

MASTER THESIS OF INFORMATION TECHNOLOGY

Ha Noi, 2012

Major: Computer Science

Code: 60.48.01

Supervised by Assoc Prof Bui The Duy

Table of Contents

1.1 Overview 2

1.2 Our Works 3

1.3 Thesis Outline 4

2 Related Work 5 2.1 Daily Activity Recognition in Smart Home System 5

2.2 Daily Activity Recognition approach 6

2.3 Incremental learning model 7

3 Framework for Activity Recognition in the Home Environment 10 3.1 Data Acquisition 10

3.2 Data annotation 12

3.3 Feature extraction 12

3.4 Segmentation 14

4 Growing Neural Gas model 15 4.1 GNG structure 15

4.1.1 Competitive Hebbian Learning 17

4.1.2 Weights Adapting 18

4.1.3 New Node Insertion 19

4.2 Using GNG network for supervised learning 23

4.2.1 Separated Weighted Euclid 24

4.2.2 Reduce Overlapping Regions by Using a Local Error Threshold 25 5 Radial Basic Function network 27 5.1 Standard Radial Basic Function 27

5.2 Incremental Radial Basic Function 29

TABLE OF CONTENTS v

7.1 Summary 35 7.2 Future works 36

Apply Incremental Learning for Daily Activity Recognition Using Low Level Sensor Data

Abstract Daily activity recognition is an important task of many applica-tions, especially in an environment like smart home The system needs

to be equipped with the recognition ability so that it can automat-ically adapt to resident’s preferences So far, a great deal number

of learning models has been proposed to classify activities However, most of them only work well in off-line manner when training data is known in advance It is known that people’s living habits change over time, therefore a learning technique that should learn new knowledge when there is new data in great demand

In this thesis, we improve an existing incremental learning model

to solve this problem The model, which is traditionally used in unsu-pervised learning, is extended for classification problems Incremental learning strategy by evaluating the error classification is applied when deciding whether a new class is generated A separated weighted Eu-clid is used to measure the distance between samples because the large variance of information contains in feature vectors To avoid constant allocation of new nodes in the overlapping region, an adaptive inser-tion constraint is added Finally, an experiment is carried to assess its performance The results show that the proposed method is better than the previous one The proposed method can be integrated into

a smart system, which then pro-actively adapts itself to the changes

of human daily activity pattern

Smart home is considered as a part of Ubiquitous Computing trend It aims

to enrich the home environment with intelligent devices with the purpose of

supporting the residents living inside the environment [4] Understanding the users’ activities is one of the key features of a smart home system Regular activities which are performed in the home can be divided into two types One type of activities is characterized as simple gestures of some part of the body such as running or walking The other one is the set of actions which become a pattern in users’ daily routines Some examples of this type are reading, cooking or watching television Recognizing these activities will provide useful information for understanding the environment context In our thesis, we focus on the problem of recognizing the second types of the activities

The most common data source in the smart home system is low-level sensor information [4] There have been many propose approaches for recog-nizing activities from low level sensory data ([20, 3, 21]) One of the proper-ties of daily activiproper-ties in the home environment is its variation through time because the user’s habits are change in real life To deal with this prob-lem, previous approach must have to train the models again when there are changes in activities patterns However, this is a resource consuming process

In our thesis, we apply incremental learning model into the problem of activity recognition to resolve the above problem The incremental learning models are based on the Growing Neural Gas (GNG) network [8] We extend the growing neural gas model for activity recognition in two ways The first approach is to use multiple GNG networks For each activity class, we train

a separated network using samples of the class Technically, the weak point

of the traditional growing neural is its network size constantly grows based

on the error evaluation for new sample insertion This learning strategy encounters the performance problem in case classes overlap We propose the use of another metric for measure distance in the GNG network and a constraint to the growing condition with the purpose to make the models more balance between each class The second approach is to create a Radial Basic Function (RBF) network [16] from the GNG network This approach

is similar to the method proposed in [7]

We carry out an experiment to compare both models Performance is evaluated on real daily activity dataset The experimental results show that the task of recognizing daily activity can achieve good result with incremental learning approach

The remaining of thesis is organized as follows In Section 2, we present the works related to activity recognition and incremental learning In Sec-tion 3, we describe more detail of activity recogniSec-tion problems including

data acquisition, data segmentation and feature extraction In Section 4, we introduce the structures of the GNG network and the way of using it for daily activity recognition problem In Section 5, the incremental version of the RBF network based on the GNG network is presented An experiment with real data set is presented in Section 6 In Section 7, we discuss the summary of our thesis and future works

In recent years, there are many researches in building a system for smart home environment In these systems, the activity recognition is considered

as a part of the context recognition module While the context recognition module is refer to understand a wide range of knowledge in the intelligent environment, the main purpose of activity recognition is to extract the in-formation of what the resident is doing The task of recognizing activities depends on which sensory data the system can perceive from the real world There are a variety of sensor types in a smart home system Sensors can be used for position measurement, detecting motion, detecting sound, reading temperature These sensors usually create many data streams [14], which have to be combined in order to produce more useful information Compar-ing to other types of receivers like cameras or microphones, usCompar-ing low level sensors offers low cost in building the sensor network and transparency The data generated by low level sensors is quite easy to collect and process in comparison to other types of device like camera

2.1 Daily Activity Recognition approach

There are many proposed models for recognizing daily activities In [20], they built an activity recognition model based on Naive Bayes classifiers In [23],

a multilayer model is used to detect walking movements from passive infrared motion detectors placed in the environment In [3], they proposed a frame-work for activity recognition including feature extraction, feature selection and predicting models The framework use the four popular machine learn-ing methods includlearn-ing Bayes Belief Networks, Artificial Neural Networks, Sequential Minimal Optimization and Boosting

One of the properties of daily activity is the change of their patterns because of the vary of users’ habits This makes the decision boundary can

change through time Proposed methods requires to train the models again when the changes occur Incremental learning approach can be used to solve this problem

2.2 Incremental learning model

The approach of incremental learning is aim to handle the problem of learning new knowledge while maintaining the previous one [11] In unsupervised approach, the well known clustering algorithm k-means [13] can be learned on-line by a weight update rule An artificial neural network, namely Self-Organized Map (SOM) [12], is presented as a unsupervised learning method using incremental approach More flexible networks with similar approach are Growing Cell Structures [7] and Growing Neural Gas [8]

In supervised learning, there are several efforts to adapt the exist off-line method to use in incremental training With the inspiring of adap-tive boost algorithm(AdaBoost) [6], [17] proposed an incremental learning method, namely Learn++ It uses the MultiLayer Perceptron network as a week learner to generate a number of weak hypotheses Radial Basic Func-tion (RBF) networks [16] combine a local representaFunc-tion of the input space and a perceptron layer to fit the input pattern with its target [9] proposed a method to insert new nodes into the RBF network Using this approach, the RBF network is capable for learning overtime Another approach is Fuzzy ARTMAP [2] which based on Adaptive Resonance Theory (ART) networks

Home Environment

In this section, we present the framework of daily activity recognition using low level sensor data in the home environment Its main purpose is to map the data from the environment to a sequence of activities We adapt the proposed framework in [3] to emphasize the incremental learning properties

of the framework Figure 1 illustrates the steps in an activity recognition module using low-level sensor data The information of the surrounding en-vironment comes under a stream of sensor events Then, the stream can be annotated and split into labelled samples The labeled samples are then pro-vided to the incremental learning model for training In online recognition, the stream is segmented into separated sequences of sensor events Each

Figure 1: Framework for activity recognition

separated sequence is then classified by the learned model By applying in-cremental learning, the model is trained continuously over the life span of the system whenever there is new labelled data

In the data acquisition phase, sensors are implemented around the home The sensors are low level, which are differ from cameras or microphones They continuously monitor the environment for some specific information There are many types of information in the home environment that can be monitored by state change sensor such as door, picking object or temperature Choosing the types of sensor are included totally depends on the system’s design

The data from each low level sensor is collected and combined into a stream of data at a central server Each sensor’s signal is considered as an event in the stream An example of event stream [5] is given in table 1 Each event has a time stamp, the name of the sensor and its value There are several platforms for collecting and processing the data stream in the smart home system such as Motes platform [1] or OSGI platform [10]

Table 1: An example of sensor events in data stream.

For the purpose of training model, the stream data needs to be segmented and labelled into separated activities In a smart home system, data can be annotated directly by using devices such as Personal Digital Assistant [20]

An alternative way is labelling the stream data indirectly through a post processing step The users can label their activities with a visualization tool [3] Because the data is captured from various sensors and span over a long time, it is difficult to determine the right starting and ending point of an activity After this step, each sensor event in the stream data is added with

an optional action label

3.3 Feature extraction

Because the daily activities in the home environment usually happen around

a specific area, the motion sensors can produce good features for differing the activity classes In [3], the activated motion sensors during the period

of an activity is included into the feature vector The feature vector also uses high contextual information such as day of week, time of day, previous activitiy, next activity and energy consumption This approach achieves a high accuracy with learning models such as multilayer perceptron or decision tree

In our approach, we use a similar method to extract feature for training with online learning model A feature vector of an activity sample includes:

• Length of the activity in second

• Part of the day when the activity happens It is divided into six sepa-rated parts: morning, noon, afternoon, evening, night, latenight,

• Day of week.

• Previous performed activity

• Number of motion sensors are activated during the activity’s period, and the number of value ”ON”

• A list of motion sensors which are activated

• Energy consumption

In a real application of activity recognition, the stream of sensor events is required to be segmented into separated subsequences before the recognition models can classify them In [18], they mine the data stream to discover the trigger patterns and use these patterns to determine the start of an activ-ity However, the trigger patterns are not always clear enough to discover because daily activities are often change and overlapped with each other An alternative approach is to use sliding time windows [19] The data stream is split into fixed-time windows and then classify them into one of the activity classes Although this method can generate a sequence of activity labels from data stream effectively, it may have difficulty when the time window overlaps with more than one activity

In this chapter, we present an incremental learning model which is improved from Growing Neural Gas (GNG) networks [8] The GNG network is an un-supervised learning and has been applied into vector quantization, clustering, and interpolation Its structure is similar to SOM [12] network and Neural Gas [15] network The network produces a local representation of the input space which is learned incrementally In the first section, we introduce the structure and learning rule of the network in unsupervised learning Then,

we extend its structure into the supervised problems with some modifications for improving the performance

4.1 GNG Network Structure

A GNG network includes nodes and connections Each node u in the network

is presented with reference vector wu in the input space V (V is a subset of

Rn( and a local counter variable eu The wu is considered as the position of node u in the vector space and eu presents the density in the region around

u The vector space is split into small regions, which takes each node of the network as a center The connections between nodes represent the structure

of the network If two nodes are connected, their correspondent regions are adjacent in V The network extends the Competitive Hebbian Learning principle [22] to learn its structure The GNG network uses its structure to adapt the nodes’ reference vector and to determine where to add new nodes When the network receives a new training sample x, the model will find the nearest node s1 to x and assign x to the region of s1 Then, the position

of s1 and all its topological neighbors n are moved toward x The moving distances are computed as follow:

The weight of s1 is adjusted by an amount of ∆s1:

For each neighbour v of s1, the weight vector wv is added by ∆v:

The network uses two constants b, n to control the adaptive of a node to a new sample

The network automatically increases its number of nodes after receiving a fixed number λ of training samples A new node will be placed into the high-est density region; i.e the region has maximum local error in the network More specifically, the adding algorithm finds a node q which eq is maximum Then it chooses f among the neighbors of q which ef is maximum New node

u is set to the middle between q and f and the local error of the two nodes

q and f are decreased due to the introducing of a new node

4.2 Apply growing neural gas model to activity

recog-nition

The GNG network can be extended for supervised learning by creating a separate network for each class In unsupervised problems, the GNG con-structs a network that represents the underlying structure of the input space