Design template in cellular neural network

Designing template is the most important stage for design and make CNN chip that gives mathematical logic architecture basing on each problem. CNN researchers have many methods to design template corresponding to solutions. This paper review some ways which is used commonly in solving PDE using CNN. The paper has 3 parts, part 1 introduction CNN technology; next part introduces some common ways to design template; part 3 give some illustrations and the last is conclusion and future trends.

DESIGN TEMPLATE IN CELLULAR NEURAL NETWORK

1 College of Information and Communication Technology - TNU 2

Ha Noi-Vieng Chan Friendship Vocational School – PDR Lao

SUMMARY

Designing template is the most important stage for design and make CNN chip that gives mathematical logic architecture basing on each problem CNN researchers have many methods to design template corresponding to solutions This paper review some ways which is used commonly in solving PDE using CNN The paper has 3 parts, part 1 introduction CNN technology; next part introduces some common ways to design template; part 3 give some illustrations and the last is conclusion and future trends

Keywords: Template Design, Partial Differential Equation, Cellular Neural Network, Lyapunov

Function, Taylor Expansion

INTRODUCTION*

The theory of CNN has been proposed by

L.O Chua an L Yang in 1988 and developed

to hardware architecture on CNN Universal

Machine (CNN-UM) by L.O Chua and R

Tamas [1,2] The CNN is the physical

paralleled computing with array of processor

called cell Depending on particular problem,

the numbers of cell can expand from 10,000

to 100,000 cells

CNN is applied in many fields like image

processing; scientific computing; robot and

economic at high speed processing

From 1988, many researchers have been

developed the CNN in theory and application

as prof the stability and condition constrains

for CNN chip

The International Conferences of CNN

applicationare organized every two years The

13th CNNA was organized from 29-31

August, at Turin, Italy with topics like:

Theoretical advances of CNNs; Sensory

integration; New spatial-temporal algorithms;

Biological relevance of CNNs; Applications

on FPGAs and GPUs;

Emerging new Cellular Wave Computing

Technologies

In Vietnam, there are some groups in IT

Institute- Vietnam Academy of Science and

Technology; Hanoi National University;

*

Tel: 0985 158998, Email: vdthai@ictu.edu.vn

Hanoi Poly technique University and especially in University of Information & Communication Technology -Thai Nguyen University (ICTU), the lecturers have taken researches in image processing; Solving PDE; CNN chaos in data encryption Up to now, they have had more than 10 papers on Vietnamese and International scientific and technology journals

Cellular neural networks is made of a massive aggregate of regularly spaced circuit clones, called cells, which communicate with each other directly only through its nearest neighbors Each cell is made of a linear capacitor, a nonlinear voltage-controlled current source, and a few resistive linear circuit elements

The basic circuit unit of a cellular neural network is called a cell It contains linear and nonlinear circuit ele-ments, which typically are linear capacitors, linearresistors, linearandnonlinear controlled sources, and independent sources The structure of cellular neuralnetworksissimilar to that found in cellular automata Each cell in a cellular neural network is connected only to its neighbor cells Adjacent cells can interact direct with each other Cells not directly connected together may affect each other indirectly because of the propagation effects of the continuoustime dynamics of the network The state equation of cell C(i,j) is given by the following equation:

( , ) ( , ) ( , ) ( , )

1

ij

x

(1) here, R, C is the linear resistor and capacitor respectively A(i,j;kl) is the feedback operator parameter; B(i,j;kl) is the control operator parameter and zij is the bias value of the cell C(i,j) On the CNN system, (A, B, z) are the local connective weight values of each cell C(i,j) to its neighbors

The output of the cell C(i,j) is modeled by:

|) 1 ) (

|

| 1 ) ( (|

2

1

)

v yij xij xij (2)

N j

M

i ;1

1

The CNN program the series of templates in

steps as design, so the templates are actual

instructions for CNN chip The programmers

find the templates then design architecture

CNN chip follow algorithm analyzed

Running CNN program is in steps as follow:

1 Set up the initial state

2 Load and run the template automatically by

the electronic operations inside circuit (do the

instructions coded)

Get the output as the result

DESIGNING CNN TEMPLATES METHODS

Using mask in image processing technique:

This method using some types of mask like in

classical process on PC to create CNN

template These masks are used for A

template for CNN chip like average,

erosion, dilation

Analyzing the dynamic of CNN chip to create templates: This method is analyze the

operating of processing into detail interactive tasks to find local rules then base on the CNN state equations and relevance between state variables and the first its derivatives on DP chart to find templates

Direct template design: This method is often

used for uncouple CNN, in which the A template has only center particle having zero off center values, but others are zero, this only applied for process binary image and simple processing

Using GA and Fuzzy: This method is new

and developing and only applying for some special CNN architectures

Eij

Ixu(ij,kl) Ixy(ij,kl) I

yx

Ry

vxij

Fig 1 Structure inside of the cell

A

+

B

z

uij

ukl

y kl

yij

x(t)

-

yij

Fig 2 The processing model of cell

Learning method: This method to find

connection weight, the input is image need to

process and desired image at output compare

between input and output one can compute

the variable values to find weight matrix then

having correspondent template

Using Taylor Difference: This method bases

on difference the differential model by Taylor

formula After differencing original equation

by refine difference grid then compare to

CNN state equation one has templates which

describe the operation of CNN chip This

method is very useful for solving DPE and

advanced image processing

Example of using Taylor Difference:

Give an assume partial differential equation follow:

(4)

with boundary and initial conditions satisfied, after differencing, one has:

We the templates for this equation like

0denote a zero synaptic weight denote a positive or zero synaptic weight

denote a negative synaptic weight

a denote any value

Fig 4 Templates with different stable state patterns of CNN chip

( , ) ( , )

( , ) ( , )

2

A

z = 0

Fig 3 Architectural design of CNN chip for Equation(4)

THE STABILITY OF CNN TEMPLATES

After finding templates, one knows the

dynamic behavior among cells, and then we

need to assure that the circuit works steadily,

mean that voltage and current are in working

ranges The designer must to demonstrate that

found templates are accepted for making

circuit

We have some ways to prove the stability of

designed diagram

Using Chua method [3,4]:

A CNN with MxN cells and a 3 x 3 A

template for arbitrary B-template, arbitrary

threshold z is completely stable of the

following three condition are satisfied:

+ The A template is sign symmetric

+ The A template possesses any one of the six

synaptic weight pattern as shown in Fig 4

Using Lyapunov function method

Complete Stability Theorem [3]:

Any MxN space-invariant CNN of arbitrary

neighborhood size with constant inputs and

constant threshold is completely stable if the

following three hypotheses are satisfied:

1 The A template is symmetric:

A(i,j;k,l)=A(k,l;i,j)

2.The nonlinear function yij = f(xij) is

differentiable, bounded, and f ′(xij) > 0, for

all -∞<xij<∞

3.All equilibrium points are isolated

Proof:

Consider the CNN stateequation, constant

input u and threshold z;

Here are nxn matrixes, whose nonzero entries

are the synaptic weights A(i,j;k,l) and

B(i,j;k,l), respectively The matrix and its

transpose are presented as:

and

(5)

Define the scalar function

where θ denotes any number such that

f(-∞)<θ<f(∞)

A scalar function V(x) is called a Lyapunov function if its time derivative along any trajectory is non-positive,

Taking the time derivative of both sides of Eq.(6) we obtain

-1

i i i=1

1

V (x)=- ( y A y+y A y

2 +( f (y y )- y B u- y z

And we can write

Substituting (5) and (8) into (7), we have:

Observe next that

Substituting (10) into (9), and D(f) is symmetric, so we have:

Prove stability of CNN chip designed for Air pollution problem

The air pollution problem is describe by equation [5,6]:

1

( )

(6)

(7)

(8)

(9)

(10)

Differencing (11), one has:

1, , , , 1, , , 1, , , , 1,

, , 1 , , , , 1 1, , 1, ,

, , 2

, 1, , 1, , , 1 , , 1

2

( 2 )

i j k i j k i j k i j k i j k i j k

i j k

i j k i j k i j k i j k i j k

i j k

i j k i j k i j k i j k

f

u

From (12), we find the templates follow:

i j k

u A

i j k

v A

i j k

A

R

zi,j,k = 0; Bi,j,k = 1;

Prove the stability of CNN chip with above

templates [4]:

Rewrite the state equation (12) with all CNN

constrains in [1]:

Choose the energy function of CNN E(t):

We compute E max(t):

Clearly that: ax | ( ) | max

t

say E(t) is bounded

Finding diffential of E(t):

, ,

, ,

, , , , , , , , , , , , ,

, , , , , ,

, , , , , , , ,

, , , , , , , , , , , , ,

, , , , ,

( ) ( )

( ) ( )

1

( ) ( )

yl m n

ul m n

y j i k x i j k

i j k l m n x i j k

y j i k x i j k

y j i k

i j k x i j k

y j i k x i j k

i j k l m n x i j k

y j i

i j k

dv dv t

dE t

dv dv t

v t

dv dv t

dv dt dv

, , ,

( )

k x i j k

x i j k

dv t

dv dt

, , , ,

( )

1

yl m n

xi j k

R

Then, we have:

Applying Lyapunov theory, we can conclude that CNN chip work stability with found templates

From these templates, we design the architecture of 3D-CNN chip having one layer but very sophisticated structure

CONCLUSION The CNN technologies have been researched for many purposes for high speed processing

in parallel and real time environment Using CNN chip to processing images and video, solve partial differential equation have achieved good results The paper introduce the methods to design templates and prove those templates could use for making CNN chip working stable From theory rules, we give an example of air pollution problem described byPED which three -variables function

, , ,

, , ,

1

x i j k

x i j k

v

2 , ,

, ,

1

yl m n

ul m n

R

ax

, , , , , , , ,

, , , ,

1

m

i j k l m n i j k l m n

i j k l m n

LMN

R

R

(12)

(13)

, ,

, ,

2 1

( ) ( )

xi j k

v

dE t

( ) 0

dE t

xi j k

v

( )

=0

dE t

xi j k

v

when when

In future, we can apply for others PDE and

make the CNN chip using FPGA technology

to simulate the computation

REFERENCES

1 Chua L O., Yang L (1988), "Cellular Neural

Networks: Theory", IEEE Transaction on Circuits

and System,35 (10), pp 1257-1272

2 Chua L.O., L Yang, (1988), "Cellular Neural

Networks: Application", IEEE Trans Circuits and

System 35, PP 1273-1290

3 Kék L.,Karacs K., Roska T (2007), Cellular

Wave Computing Library, ver 2.1 Cellular

Sensory wave computer Laboratory Hungarian

Academy of Sciences Budapest, Hungary

4 Yeniceri R.,Yalcm M E (2008), “ A Programmable Hardware for Exploring

Spatiotemporal Waves in Real-time”, Proceeding

of 11 th InternatIonal Workshop on CNN and their Applications, (CNNA2008), PP 7-9

5 Vũ Đức Thái, ”Vấn đề ổn định của mạng CNN giải phương trình thuỷ lực hai chiều trên chip”,

Tạpchí Tin học và Điều khiển, tập 26, số 3, năm

2010, Tr 278-288

6 V.D.Thai, P.T.Cat “Modelling Air pollution Problem by Cellular Neural Network” Proceeding

(ISI) of 10th Intl Conf on Control, Automation, Robotics and Vision, Hanoi, Vietnam 17-20/12/2008 Page(s):1115-1118; website: http://IEEE.explorer.com

TÓM TẮT

CÁC PHƯƠNG PHÁP THIẾT KẾ MẪU CHO MẠNG NƠ RON TẾ BÀO

Thiết kế mẫu là một bước quan trọng trong việc chế tạo chip CNN để giải quyết một bài toán tính toán khoa học trên công nghệ mạng nơron tế bào Với mỗi bài toán, ta phải thiết kế một kiến trúc tính toán riêng dựa trên các ràng buộc toán học mô tả theo mẫu (template) Các nhà nghiên cứu về CNN có nhiều phương pháp thiết kế mẫu trong đó một phương pháp quan trọng là sử dụng phương pháp sai phân Taylor Bài báo này giới thiệu về các phương pháp thiết kế mẫu và chứng minh tính ổn định của mẫu Việc áp dụng nhóm tác giả đã minh họa qua một bài toán giải phương trình đạo hàm riêng mô tả hiện tượng khuếch tán chất thải qua môi trường không khí Bài báo có 4 phần: phần Giới thiệu; Các phương pháp thiết kế mẫu CNN; Chứng minh tính ổn định của chip theo mẫu tìm được; Bài toán ứng dụng và Kết luận đưa ra hướng phát triển

Từ khóa:Thiết kế mẫu; Phương trình đạo hàm riêng, Mạng nơron tế bào, Hàm Lyapunov, Sai

phân Taylor

Ngày nhận bài:25/01/2014; Ngày phản biện:10/02/2014; Ngày duyệt đăng: 26/02/2014

Phản biện khoa học: TS Phạm Đức Long – Trường ĐH Công nghệ Thông tin & Truyền thông - ĐHTN

*

Tel: 0985 158998, Email: vdthai@ictu.edu.vn