A new fuzzy regression model based on interval valued fuzzy neural network and its applications to management

A new fuzzy regression model based on interval valued fuzzy neural network and its applications to management Accepted Manuscript A new fuzzy regression model based on interval valued fuzzy neural net[.]

Accepted Manuscript

A new fuzzy regression model based on interval-valued fuzzy neural network

and its applications to management

Somaye yeylaghi, Mahmood Otadi, Niloofar Imankhan

DOI: http://dx.doi.org/10.1016/j.bjbas.2017.01.004

To appear in: Beni-Suef University Journal of Basic and Applied

Sciences

Received Date: 19 November 2016

Revised Date: 8 January 2017

Accepted Date: 16 January 2017

Please cite this article as: S yeylaghi, M Otadi, N Imankhan, A new fuzzy regression model based on

interval-valued fuzzy neural network and its applications to management, Beni-Suef University Journal of Basic and Applied

Sciences (2017), doi: http://dx.doi.org/10.1016/j.bjbas.2017.01.004

This is a PDF file of an unedited manuscript that has been accepted for publication As a service to our customers

we are providing this early version of the manuscript The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain

ŶĞǁĨƵnjnjLJƌĞŐƌĞƐƐŝŽŶŵŽĚĞůďĂƐĞĚŽŶŝŶƚĞƌǀĂůͲǀĂůƵĞĚĨƵnjnjLJŶĞƵƌĂů

ŶĞƚǁŽƌŬĂŶĚŝƚƐĂƉƉůŝĐĂƚŝŽŶƐƚŽŵĂŶĂŐĞŵĞŶƚ

^ŽŵĂLJĞLJĞLJůĂŐŚŝ
͕DĂŚŵŽŽĚKƚĂĚŝ ͕EŝůŽŽĨĂƌ/ŵĂŶŬŚĂŶ


ĞƉĂƌƚŵĞŶƚŽĨDĂŶĂŐŵĞŶƚ͕&ŝƌŽŽnjŬŽŽŚƌĂŶĐŚ͕/ƐůĂŵŝĐnjĂĚ

hŶŝǀĞƌƐŝƚLJ͕&ŝƌŽŽnjŬŽŽŚ͕/ƌĂŶ

 ĞƉĂƌƚŵĞŶƚŽĨDĂƚŚĞŵĂƚŝĐƐ͕&ŝƌŽŽnjŬŽŽŚƌĂŶĐŚ͕/ƐůĂŵŝĐnjĂĚ

hŶŝǀĞƌƐŝƚLJ͕&ŝƌŽŽnjŬŽŽŚ͕/ƌĂŶ

ďƐƚƌĂĐƚ͗

In this paper, a novel hybrid method based on interval-valued fuzzy neural network for approximate of interval-valued fuzzy regression models, is presented The work of this paper is

an expansion of the research of real fuzzy regression models In this paper interval-valued fuzzy neural network (IVFNN) can be trained with crisp and interval-valued fuzzy data

Here a neural network is considered as a part of a large field called neural computing or soft computing Moreover, in order to find the approximate parameters, a simple algorithm from the cost function of the fuzzy neural network is proposed Finally, we illustrate our approach by some numerical examples and compare this method with existing methods



A new fuzzy regression model based on

interval-valued fuzzy neural network

Abstract

In this paper, a novel hybrid method based on interval-valued fuzzy neural network for approximate of interval-valued fuzzy regres-sion models, is presented Here a neural network is considered as a part

of a large field called neural computing or soft computing Moreover,

in order to find the approximate parameters, a simple algorithm from the cost function of the fuzzy neural network is proposed Finally, we illustrate our approach by some numerical examples and compare this method with existing methods

Keywords: Interval-valued fuzzy neural networks; Interval-valued fuzzy regression model; Feedforward neural network; Learning algo-rithm

1 Introduction

The concept of fuzzy numbers and fuzzy arithmetic operations were first introduced by Zadeh,1975, Dubois and Prade,1978 Also, fuzzy systems are used to study a variety of problems ranging from (Zhang et al.,2014a,Zhang

et al., 2014b) We refer the reader to (Kaufmann and Gupta, 1985) for more information on fuzzy numbers and fuzzy arithmetic

Regression analysis is of the most popular methods of estimation It is applied to evaluate the functional relationship between the dependent and independent variables Fuzzy regression analysis is an extension of the clas-sical regression analysis in which some elements of the model are represented

by fuzzy numbers Fuzzy regression methods have been successfully applied

to various problems such as forecasting (Chang,1997,Chen and Wang,1999, Kao,2003, Tanaka,1989,Tseng,2002) and engineering (Lai and Chang,1994) Thus, it is very important to develop numerical procedures that can appro-priately treat fuzzy regression models Sakawa and Yano (1992) proposed

a mathematical programming model to estimate the parameters of a fuzzy linear regression

Yi= A1xi1+ A2xi2+ Anxin, where xij ∈ R and A1, A2, , An, Yi are symmetric fuzzy numbers for i =

1, 2, , m, j = 1, 2, , n

Ishibuchi et al (1995) proposed a learning algorithm of fuzzy neural net-works with triangular fuzzy weights and Hayashi et al (1993) fuzzified the

delta rule Buckley and Eslami (1997) consider neural net solutions to fuzzy problems The topic of numerical solution of fuzzy polynomials by fuzzy neural network investigated by Abbasbandy et al (2006,2008), consists of finding solution to polynomials like a1x+a2x2+ .+anxn = a0where x ∈ R and a0, a1, , an are fuzzy numbers, and finding solution to systems of s fuzzy polynomial equations where x1, x2, , xn ∈ R and all coefficients are fuzzy numbers Then Mosleh and Otadi (2010,2011,2012,2014) proposed a fuzzy neural network model to estimate the parameters of a fuzzy regression models

In this paper, we first propose an architecture of interval-valued fuzzy neural network with interval-valued fuzzy weights for real input vectors and valued fuzzy targets to find approximate coefficients to interval-valued fuzzy linear regression model

˜˜

Yi= ˜˜A0+ ˜˜A1xi1+ + ˜˜Anxin, where i indexes the different observations, xi1, xi2, , xin ∈ R, all coeffi-cients and ˜˜Yi are interval-valued fuzzy numbers

2 Basic concepts of fuzzy numbers

2.1 Generalized fuzzy numbers

In this section, we briefly review basic concepts of generalized fuzzy num-bers Chen and Wei (1999,2009) represented a generalized triangular fuzzy number represented a generalized triangular fuzzy number ˜Aas ˜A= (a1, a2, a3; w) where a1, a2 and a3 are real values and 0 < w ≤ 1, as shown in Fig 1 The membership function µA˜ of a generalized fuzzy number ˜Asatisfies the fol-lowing conditions:

(1) µA˜ is a continuous mapping from the universe of discourse R to the closed interval in [0, 1];

(2) µA˜= 0, where −∞ < x ≤ a1;

(3) µA˜is monotonical increasing in [a1, a2];

(4) µA˜= w, where x = a2;

(5) µA˜is monotonical decresing in [a2, a3];

(6) µA˜= 0, where a3≤ x < +∞

If w = 1, then the generalized fuzzy number µA˜is a normal fuzzy number, denoted as µA˜= (a1, a2, a3) If a1= a2 = a3 and w = 1, then µA˜ is a crisp value

2.2 Interval-valued fuzzy numbers and their arithmetic op-erations

Gorzalczany (1987) proposed the concept of IVFS Then, Yao and Lin (2002) represented the IVFS ˜˜A= [ ˜˜AL, ˜A˜U] shown in Fig 2 where ˜˜ALdenotes the lower IVFS, ˜¯AU denotes the upper IVFS, where 0 ≤ ˜˜AL≤ ˜˜AU ≤ 1, and

˜

AL⊂ ˜˜AU.Thereby, the minimum and maximum membership value of ˜˜Aare

˜

AL and ˜˜AU, respectively We denote the set of all IVFS with EI

From Fig 3, we can see interval-valued triangular fuzzy number ˜˜A con-sists of the lower triangular fuzzy number ˜˜ALand the upper triangular fuzzy number ˜˜AU Let ˜˜AL and ˜˜AU be two generalized triangular fuzzy numbers, and let wL

˜

Aand WU

˜

A denote the heights of ˜˜ALand ˜˜AU, respectively A IVFN

A defined in the universe of discourse X is represented by the following:

˜˜

A= [(aL

1, aL2, aL3; wL

˜

A), (aU

1, aU2, aU3; wU

˜

A)]

The membership functions of ˜˜AL and ˜˜AU can be expressed as follows:

˜˜

AL=























w L

˜

A (x−a L

1 )

a L

2 −a L 1

, f or aL1 ≤ x ≤ aL

2,

wL

˜

2,

w U

˜

A (a L

3 −x)

a L

3 −a L 2

, f or aL2 ≤ x ≤ aL

3,

(1)

˜˜

AU =























w U

˜

A (x−a U

1 )

a U

2 −a U 1

, f or aU1 ≤ x ≤ aU

2,

wU

˜

2,

w U

˜

A (a U

3 −x)

a U

3 −a U 2

, f or aU2 ≤ x ≤ aU

3,

(2)

Denote the h-level sets of [ ˜˜A]h= [[ ˜˜AL]h,[ ˜˜AU]h] as follows:

[[ ˜˜AL]h,[ ˜˜AU]h] =







[[[ ˜˜AL]l

h,[ ˜˜AL]r

h], [[ ˜˜AU]l

h,[ ˜˜AU]r

h]], f or 0 < h ≤ wL

˜

A, [[ ˜˜AU]l

h,[ ˜˜AU]r

A≤ h ≤ wU

˜

A, (3) where [ ˜˜AL]l

h and [ ˜˜AU]l

h are left hand side of the h-cut, and [ ˜˜AL]r

[ ˜˜AU]r

h are right hand side of the h-cut Also, [ ˜˜AL]l

h = aL

1 + (aL2 −a L

1 )h

w L

˜ A

, [ ˜˜AL]r

h= aL

3 −(aL3 −a L

2 )h

w L

˜ A

,[ ˜˜AU]l

h= aU

1 +(aU2 −a U

1 )h

w U

˜ A

and [ ˜˜AU]r

h= aL

3 −(aU3 −a U

2 )h

w U

˜ A

Definition 1 Let ˜˜A and ˜˜B are two interval-valued fuzzy numbers We say that ˜˜Ais equal to ˜˜B if and only if ˜˜AL= ˜˜BL and ˜˜AU = ˜˜BU

Assume that there are two interval-valued triangular fuzzy numbers ˜˜A and ˜˜B, where

˜˜

A= [(aL1, aL2, aL3; wL˜

A), (aU1, aU2, aU3; wU˜

A)],

˜˜

B= [(bL1, bL2, bL3; wL˜

B), (bU1, bU2, bU3; wU˜

B)],

aL

1, aL

2, aL

3, aU

1, aU

2, aU

3, bL

1, bL

2, bL

3, bU

1, bU

2 and bU

3 are real values, 0 ≤ wL

˜

wU˜

A ≤ 1 and 0 ≤ wL

˜

˜

B ≤ 1 The arithmetic operations between the interval-valued triangular fuzzy numbers ˜˜A and ˜˜B are reviewed from (1985,1985) as follows:

(1) Interval-valued fuzzy numbers addition:

˜˜

A+ ˜˜B= [(aL

1 + bL

1, aL

2 + bL

2, aL

3 + bL

3; min{wL

˜

A, wL

˜

B}) ,(aU

1 + bU

1, aU2 + bU

2, aU3 + bU

3; min{wU

˜

A, wU˜

(2) Interval-valued fuzzy numbers subtraction:

˜˜

A− ˜˜B= [(aL

1 − bL

3, aL2 − bL

2, aL3 − bL

1; min{wL

˜

A, wL˜

B}) ,(aU

1 − bU

3, aU

2 − bU

2, aU

3 − bU

1; min{wU

˜

A, wU

˜

(3) Interval-valued fuzzy numbers multiplication by a crisp number (q is

a nonzero number):

q ˜A˜=







[(q.aL

1, q.aL

2, q.aL

3; wL

˜

A), (q.aU

1, q.aU

2, q.aU

3; wU

˜

A)], if q ≥ 0, (q.aL

3, q.aL2, q.aL1; hL

¯

A), (q.aU

3, q.aU2, q.aU1; hU

¯

A)], if q < 0

(6)

4) Interval-valued fuzzy numbers by an increasing function:

f( ˜˜N et) = f ([ ˜˜N etL, ˜N et˜ U]), (7) with the h−level sets:

[f ( ˜˜N et)]h=











[[f ([ ˜˜N etL]l

h), f ([ ˜˜N etL]r

h)], [f ([ ˜˜N etU]l

h), f ([ ˜˜N etU]r

h)]],

f or0 ≤ h ≤ wL

˜

N et, [f ([ ˜˜N etU]l

h), f ([ ˜˜N etU]r

˜

A≤ h ≤ wU

˜

A (8) Definition 2 Let I be a real interval A mapping g : I → EI is called a interval-valued fuzzy process We denote

[˜˜g(x)]h=

(

[[[˜˜gL(x)]l

h,[˜˜gL(x)]r

h], [[˜˜gU(x)]l

h,[˜˜gU(x)]r

h]], f or 0 < h ≤ wL

˜ g(x), [[˜˜gU(x)]l

h,[˜˜gU(x)]r

h], f or wL˜g(x)≤ h ≤ wU

˜ g(x), for x ∈ I

3 Regression model

We have postulated that the dependent interval-valued fuzzy variable ˜˜Y,

is a function of the independent real variables x1, x2, , xn More formally

˜˜

f : Rn −→ EI,

˜˜

Yi=f(x˜˜ i1, xi2, , xin), where i indexes the observations

The objective is to estimate a interval-valued fuzzy linear regression (IVFLR) model, express as follows:

˜˜

Yi= ˜˜A0+ ˜˜A1xi1+ + ˜˜Anxin (9) The extension principle leads to the following definition of ˜˜Yi:

˜

Yi(y) = sup{min{ ˜˜A0(τ0), ˜˜A1(τ1), , ˜˜An(τn)}|y = τ0+ τ1xi1+ + τnxin}

Whenf˜˜: R −→ EI, we might do it by eye-fitting the line that looks best

to us Unfortunately, different people will draw different lines and it would

be nice to have a formal method for finding the line that would consistently provide us with the best line possible What would a “best possible line” look like? Intuitively, it would seem to have to be a line that fit the data well That is, the distance of the line from the observations should be as small as possible Let ˜˜A0, ˜A˜1, , ˜A˜n denote the list of regression coeffi-cients (parameters) ˜˜A0 is an optional intercept parameter and ˜˜A1, , ˜A˜n are weights or regression coefficients corresponding to xi1, , xin Then interval-valued fuzzy linear regression is given by Eq (9) where i indexes the different observations and ˜˜A0, ˜A˜1, , ˜A˜nare interval-valued fuzzy num-bers We are interested in finding ˜˜A0, ˜A˜1, , ˜A˜n of interval-valued fuzzy linear regression such that ˜˜Yiapproximates Yifor all i = 1, 2, , m, closely enough according to some norm k.k, i.e.,

mink[ ˜˜Y

L

i]lh− [ ˜˜YLi]lhk, 0 < h ≤ wL˜

Yi, mink[ ˜˜Y

L

i]r

h− [ ˜˜YLi]r

hk, 0 < h ≤ wL

˜

Yi, mink[ ˜˜Y

U

i ]l

h− [ ˜˜Y

U

i ]l

hk, 0 < h ≤ wL

˜

Y i

, mink[ ˜˜Y

U

i ]r

h− [ ˜˜Y

U

i ]r

hk, 0 < h ≤ wL

˜

Yi, mink[ ˜˜Y

U

i ]l

h− [ ˜˜YUi ]l

hk, wL

˜

Yi ≤ h ≤ wU

˜

Yi, mink[ ˜˜Y

U

i ]r

h− [ ˜˜YUi ]r

hk, wL

˜

Yi ≤ h ≤ wU

˜

Yi

(10)

Then, it becomes a problem of optimization

A IV F N N4 (interval-valued fuzzy neural network with interval-valued fuzzy weights, output signals and real inputs) solution to Eq (9) is given

in Fig 4 The input neurons make no change in their inputs and the input signals interact with the weights, so the input to the output neuron is

˜˜

A0+ ˜˜A1xi1+ + ˜˜Anxin, and the output, in the output neuron, equals its input, so

˜˜

Yi= ˜˜A0+ ˜˜A1xi1+ + ˜˜Anxin How does the IV F N N4solve the interval-valued fuzzy linear regression? The training data are {(1, x11, , x1n), , (1, xm1, , xmn)} for inputs and target (desired) outputs are {Y1, , , Ym} We proposed a learn-ing algorithm from the cost function for adjustlearn-ing fuzzy number weights Following Section, we proposed a learning algorithm such that the net-work can approximate the fuzzy A0, A1, , An of Eq (9) to any degree of accuracy

4 Learning algorithm

Consider the learning algorithm of the two-layer fuzzy feedforward neural network with 2 inputs and one output as shown in Fig 4 Let the h-level sets of the target output Yi, i= 1, , m be denoted

[ ˜˜Yi]h= [[ ˜˜YiL]h,[ ˜˜YiU]h], i= 1, , n, (11) where YL

i (h) shows the left-hand side and YU

i (h) the right-hand side of the h-level sets of the desired output

A cost function to be minimized is defined for each h-level sets as follows: [E( ˜˜W0, ˜W˜1, , ˜W˜n)]h= [E( ˜˜W0, ˜W˜1, , ˜W˜n)]l

h+ [E( ˜˜W0, ˜W˜1, , ˜W˜n)]r

h, (12) where

[E( ˜˜W0, ˜W˜1, , ˜W˜n)]l

h=

m

X

i=1

([ ˜˜Y

L

i]l

h− [ ˜Y˜iL]l

h)2

([ ˜˜Y

U

i ]l

h− [ ˜˜YiU]l

h)2

for 0 < h ≤ wL

˜

Y i

,

[E( ˜˜W0, ˜W˜1, , ˜W˜n)]r

h=

m

X

i=1

([ ˜˜Y

L

i]r

h− [ ˜Y˜L

i ]r

h)2

([ ˜˜Y

U

i ]r

h− [ ˜˜YU

i ]r

h)2

for 0 < h ≤ w˜

Y i

,

[E( ˜˜W0, ˜W˜1, , ˜W˜n)]l

h=

m

X

i=1

([ ˜˜Y

U

i ]l

h− [ ˜˜YU

i ]l

h)2

for wL

˜

Yi ≤ h ≤ wU

˜

Yi,

[E( ˜˜W0, ˜W˜1, , ˜W˜n)]r

h=

m

X

i=1

([ ˜˜Y

U

i ]r

h− [ ˜˜YiU]r

h)2

for wL

˜

Yi≤ h ≤ wU

˜

Yi.The total cost function for the input-output pair (xi, ˜Y˜i)

is obtained as

h

[E(W0, W1, , Wn)]h (13)

Hence [E( ˜˜W0, ˜W˜1, , ˜W˜n)]L

hdenotes the error between the left-hand sides of the h-level sets of the desired and the computed output, and [E( ˜˜W0, ˜W˜1, , ˜W˜n)]U

h

denotes the error between the right-hand sides of the h-level sets of the de-sired and the computed output

Clearly, this is a problem of optimization of quadratic functions without constrains that can usually be solved by gradient descent algorithm In fact, denoting

[∇E( ˜˜W)]h= ([∂E( ˜˜W)

∂ ˜W˜0 ]h, ,[

∂E( ˜˜W)

∂ ˜W˜n ]h)

T,

in order to solve Eq (10), assume k iterations to have been done and get the kth iteration point ˜˜Wk

REMARK 1 Since the Eq (12) are quadratic functions, supposing 0 ≤

oij = xij for i = 1, , m, j = 0, , n, we rewrite these as follows:

[E( ˜˜W)]l

h

Pm i=1

( P n j=0 [o ijW˜ L

j ] l

h −[Y˜ L

i ] l

h ) 2

P n j=0 [o ijW˜ U

j ] l

h −[Y˜ U

i ] l

h ) 2

2

= 12([ ˜˜WL]l

h)TQ[ ˜˜WL]l

h+ ([ ˜˜BL]l

h)T[ ˜˜WL]l

h+ [ ˜˜CL]l

h+

1

2([ ˜˜WU]l

h)TQ[ ˜˜WU]l

h+ ([ ˜˜BU]l

h)T[ ˜˜WU]l

h+ [ ˜˜CU]l

h, where

Q=









i=1oi2 Pm

i=1oin

Pm

i=1oi1 Pm

i=1o2i1 Pm

i=1oi1oi2 Pm

i=1oi1oin

Pm

i=1oin Pmi=1oinoi1 Pm

i=1oinoi2 Pm

i=1o2in









[ ˜˜B]h= ([˜˜b0]h,[˜˜b1]h, ,[˜˜bn]h)T, [ ˜˜C]h= 12Pm

i=1([ ˜˜Yi]h)2, with [˜˜bj]h= −Pm

i=1oij[ ˜˜Yi]h We have [∇E( ˜˜W)]lh= Q[ ˜˜WL]lh+ [ ˜˜BL]lh+ Q[ ˜˜WU]lh+ [ ˜˜BU]lh, (14)

and we can obtain

[∇E( ˜˜W)]rh= Q[ ˜˜WL]rh+ [ ˜˜BL]rh+ Q[ ˜˜WU]rh+ [ ˜˜BU]rh, (15)

for 0 < h ≤ wL

˜

Yi Also, we have

[∇E( ˜˜W)]lh= Q[ ˜˜WU]lh+ [ ˜˜BU]lh, (16) [∇E( ˜˜W)]rh= Q[ ˜˜WU]rh+ [ ˜˜BU]rh, (17) for wL

˜

Yi ≤ h ≤ wU

˜

Yi Now we consider its explicit scheme Hence we have [11, ?, 19]

˜˜

Wk+1= ˜˜Wk+ ∆ ˜˜Wk,

where k indexes the number of adjustments and µ is a learning rate (a positive real number)

5 Comparison with other methods

This study would not be completed without comparing it with the recent pa-pers (Kao,2003,Mosleh,2010,2011,2012,Otadi,2014,Tanaka,1989,1982,Sakawa,1992) Some comparisons are as follows:

• Fuzzy linear regression was first introduced by Tanaka et al (1989)

The objective was to minimize the total spread of the fuzzy parameters

subject to the support of the estimated values that cover the support of

the observed values for a certain h-level Although this approach was

later improved by Tanaka et al (1989), their model is very sensitive

to outliers Moreover,it can produce infinite solutions and the spread

of the estimated values become wider as more data are included in

the model Then Mosleh and et al (2010) used fuzzy neural network

and compared the performance of these two methods in estimation

Example three in (2010) shown that the fuzzy neural network method

is better than of the previous studies

• Sakawa and Yano (1992) formulated a fuzzy linear regression model

with fuzzy output and fuzzy parameters as a mathematical

program-ming problem Recently Otadi in (2014) shown fuzzy neural network

is better than of the mathematical programming problem