Multivariate approximation by translates of tensor product kernel on Smolyak grids

The purpose of this paper is to improve and extend the ideas in the recent paper 14 on sparse approximation by translates of the multivariate Korobov function. The motivation for the results given in 14, and those presented here come from Machine Learning, since certain cases of our results here relate to approximation of a function by sections of a reproducing kernel corresponding to specific Hilbert space of functions. This relationship to ML is described in the paper 14 and is not reviewed in detail here. We shall begin our discussion here by establishing notation used throughout the paper. In this regard, we merely follow closely the presentation in 14. The ddimensional torus denoted by T d is the cross product of d copies of the interval 0, 2π with the identification of the end points. When d = 1, we merely denote the dtorus by T. Functions on T d are identified with functions on R d which are 2π periodic in each variable. We shall denote by Lp(T d ), 1 ≤ p < ∞, the space of integrable functions on T d equipped with the norm

Multivariate approximation by translates of tensor product kernel on

Smolyak grids

Dinh D˜unga∗, Charles A Micchellib, Vu Nhat Huyc

a Vietnam National University, Information Technology Institute

144 Xuan Thuy, Hanoi, Vietnam

bDepartment of Mathematics and Statistics, SUNY Albany

Albany, 12222, USA

c College of Science, Vietnam National University

334 Nguyen Trai, Thanh Xuan, Ha Noi

August 9, 2014 Version 0.9

Abstract Keywords: Korobov space; Translates of the Korobov function; Reproducing kernel Hilbert space; Smolyak grids.

Mathematics Subject Classifications: (2010) 41A46; 41A63; 42A99.

The purpose of this paper is to improve and extend the ideas in the recent paper [14] on sparseapproximation by translates of the multivariate Korobov function The motivation for the resultsgiven in [14], and those presented here come from Machine Learning, since certain cases of our resultshere relate to approximation of a function by sections of a reproducing kernel corresponding to specificHilbert space of functions This relationship to ML is described in the paper [14] and is not reviewed

in detail here

We shall begin our discussion here by establishing notation used throughout the paper In thisregard, we merely follow closely the presentation in [14] The d-dimensional torus denoted by Tdis thecross product of d copies of the interval [0, 2π] with the identification of the end points When d = 1,

we merely denote the d-torus by T Functions on Td are identified with functions on Rdwhich are 2πperiodic in each variable We shall denote by Lp(Td), 1 ≤ p < ∞, the space of integrable functions

on Tdequipped with the norm

We will consider only real valued functions on Td However, all the results in this paper are true forthe complex setting Also, we will use the Fourier series of a real valued function in complex form.Here, we use the notation Nm for the set {1, 2, , m} and later for r, s ∈ Z we will use Zr,s

for the set {r, r + 1, , s} For vectors x := (xl : l ∈ Nd) and y := (yl : l ∈ Nd) in Td we use(x, y) := P

l∈N dxlyl for the inner product of x with y Also, for notational convenience we allow

N0 and Z0 to stand for the empty set Given any integrable function f on Td and any lattice vector

j = (jl: l ∈ Nd) ∈ Zd, we let ˆf (j) denote the j-th Fourier coefficient of f defined by the equation

where the sequence ( ˆf (j) : j ∈ Zd) forms a tempered sequence [Z,.]

Let λ := (λj : j ∈ Z) be a bounded sequence with nonzero components With the univariate λ

we associate a multivariate tensor product sequence λ := (λj : j ∈ Zd) defined on a lattice vectors

j := (jl: l ∈ Nd) whose component are given by

The case p = 2 is particularly interesting as it has an interpretation in ML which is described indetail in the paper [14] As in that paper we are concerned with the following concept Let W ⊂ Lp(Td)

be a prescribed subset of Lp(Td) and ψ ∈ Lp(Td) be a given function on Td We are interested in theapproximation in Lp(Td)-norm of all functions f ∈ W by arbitrary linear combinations of n translates

of the function ψ, that is, by the functions in the set {ψ(· − yl) : yl ∈ Td, l ∈ Nn} and measure theerror in terms of the quantity

Mn(Uλ,2(Td))2 := inf{Mn(Uλ,2(Td), ψ)2: ψ ∈ L2(Td)}

which gives information about the best choice of ψ

This paper is organized in the following manner

In this section, we introduce a method of approximation induced by translates of the function defined

in equation (1.2) in the univariate case We do this in some greater generality than described earlier

To the end, we start with the functions ϕλ, ϕβ given in equation (1.2) and we consider a even, increasing function h : R → [0, 1] defined on [0, ∞) such that

non-h(t) =

(

1, if t ∈ [−12,12]

0, if t 6∈ (−1, 1) (2.4)Corresponding to this function we introduce a trigonometric polynomial Hm ∈ Tm define at x ∈ T as

k∈Z|θk|p < ∞

o, Im,j = {k ∈ Z : (2m + 1)j − m ≤ k ≤(2m + 1)j + m},

For a function f ∈ Φλ,p(T) represented as f = ϕλ∗ g, g ∈ Lp(T), we define the operator

where δm := 2π/(2m + 1) and Vm(g) := Hm∗ g Our goal is to obtain an estimate for the error ofapproximating a function f ∈ Φλ,p(T) by Qm(f ) the linear combinations of n translates of the function

k∈Z

Γ2m,k},

where γk = αk0βk−1 and Γm,j = max{|γk| : k ∈ Im,j} for j ∈ Z, j 6= 0, Γm,0 = max{|γk| : k ∈(m/2, m)} Then there exists a positive constant c such that for all m ∈ N and f ∈ Φλ,2(T) we havethat

kf − Qm(f )k2 ≤ cεmkf kΦλ,2(T) (2.7)and

By the triangle inequality we have

kQm(f ) − f k2 ≤ kAmk2+ kBmk2 (2.13)Parseval’s identity gives

This together with (2.13) and (2.14) proves the theorem

Corollary 2.2 Let h(x) = 1 if x ∈ [−1, 1], h(x) = 0 if x 6∈ [−1, 1] and

εm = max{ sup

|k|>m

|λ−1k |,

sX

k∈Z\0

Γ2m,k},

where Γm,j = max{|γk| : k ∈ Im,j} for j ∈ Z Then for all m ∈ N and f ∈ Φλ,2(T) we have that

kf − Qm(f )k2 ≤ cεmkf kΦλ,2(T) (2.15)and

kQm(f )k2 ≤ ckf kΦλ,2(T).Proof From the definition of function h and the proof in above theorem we have

Corollary 2.3 Let λk = βk = λ−k = β−k for all k ∈ Z and h(x) = 1 if x ∈ [−1, 1], h(x) = 0 if

x 6∈ [−1, 1]; the sequence {|λk|}k∈N is non decreasing Then there exists a positive constant c such thatfor all m ∈ N and f ∈ Φλ,2(T) we have that

εm = max{|λ−1m |,

sX

k∈N

λ−2mk},

Proof From the hypothesis we have γk = λk and sup|k|>m|λ−1k | ≤ |λ−1

m | and Γm,k ≤ |λmk| for all

k ∈ N From this and corollary 2.2, we complete the proof

From the above corollary we have the following result

Corollary 2.4 Let λk = βk = λ−k = β−k for all k ∈ Z and h(x) = 1 if x ∈ [−1, 1], h(x) = 0 if

x 6∈ [−1, 1]; the sequence {|λk |

k r }k∈N is non decreasing for some r > 12 Then there exists a positiveconstant c such that for all m ∈ N and f ∈ Φλ,2(T) we have that

kf − Qm(f )k2 ≤ c|λ−1m |kf kΦλ,2(T)and

kQm(f )k2 ≤ ckf kΦ

λ,2 (T).Proof We see from the hypothesis that

|λmk|(mk)r ≥ |λm|

k∈N

λ−2mk≤ |λ−1m |

sX

k∈N

k−2r

Note that, since r > 12 we have P

k∈Nk−2r < ∞ and then by applying above corollary we completethe proof

Corollary 2.5 Let βk = β−k = λ2k = λ2−k for all k ∈ Z and h(x) = 1 if x ∈ [−1, 1], h(x) = 0 if

x 6∈ [−1, 1]; the sequence {|λk|}k∈N is non decreasing Then there exists a positive constant c such thatfor all m ∈ N and f ∈ Φλ,2(T) we have that

kf − Qm(f )k2 ≤ cεmkf kΦ

λ,2 (T)

and

kQm(f )k2 ≤ ckf kΦλ,2(T).where

εm = max{|λ−1m |,

sX

k∈N

λ−2mk},

Proof We see that |γk| = |λ−2k λk0| ≤ |λ−1k | and then it follows from the sequence {|λk|}k∈N is nondecreasing that Γm,k ≤ |λ−1mk| for all k ∈ N From this and corollary 2.2, we complete the proof.Corollary 2.6 Let h(x) = 1 if x ∈ [−1, 1], h(x) = 0 if x 6∈ [−1, 1], βk = λ2k = β−k = λ2−k and thesequence {|λk |

k r }k∈N is non decreasing for some r > 12 Then there exists a positive constant c such thatfor all m ∈ N and f ∈ Φλ,2(T) we have that

kf − Qm(f )k2 ≤ c|λ−1m |kf kΦλ,2(T) (2.16)and

kQm(f )k2 ≤ ckf kΦ

Proof We see from the hypothesis that

|λmk|(mk)r ≥ |λm|

k∈N

λ−2mk≤ |λ−1m|

sX

k∈N

k−2r

Note that, since r > 12 we have P

k∈Nk−2r < ∞ and then by applying above corollary we completethe proof

Corollary 2.7 Let h(x) = 1 if x ∈ [−1, 1], h(x) = 0 if x 6∈ [−1, 1], λk = βk = |k|1/2ln |k| for all

k ∈ Z, k 6= 0 Then there exists a positive constant c such that for all m ∈ N and f ∈ Φλ,2(T) we havethat

Theorem 2.8 Let λk = λ−k for all k ∈ Z and the sequence {|λk |

k r }k∈N is non decreasing for some

2.2 Results for L2(Td) spaces

Definition 2.9 For k ∈ Rd we define

|k|p =

((Pd j=1|kj|p)1/p if 1 ≤ p < ∞

Qm(f ) := 1

(2m + 1)d

X

l∈Z d 2m+1

j∈Z d

Γ2 m,j}

where γk= βk−1αk0, k0 ∈ [−m, m]d,kj−k

0 j

2m+1 ∈ Z for all j = 1, 2, , d, and

kQm(f )k2 ≤ ckf kΦ

λ,2 (T d ) (2.22)Proof We define the kernel Pm(x, t) for x, t ∈ Td as

Pm(x, t) := 1

(2m + 1)d

X

l∈Z d 2m+1

By the triangle inequality we have

kQm(f ) − f k2 ≤ kAmk2+ kBmk2 (2.27)Parseval’s identity gives

Definition 2.11 The sequence {θk}k∈Zd will be called a non decreasing-type sequence if θk≥ cθl forall k, l ∈ Zd satisfies |kj| ≥ |lj|, j = 1, 2, , d.

From the above corollary we have the following result

Corollary 2.12 Let h(x) = 1 if x ∈ [−1, 1], h(x) = 0 if x 6∈ [−1, 1]; the sequence { |βk |

|k| r |λk|}k∈Zd is anon decreasing-type sequence for some r > d2 Then there exists a positive constant c such that for all

m ∈ N and f ∈ Φλ,2(Td) we have that

Note that for 2r > d thenP∞

j=1jd−1j−2r is convergent The proof is complete

Corollary 2.13 Let h(x) = 1 if x ∈ [−1, 1], h(x) = 0 if x 6∈ [−1, 1], βk = λ2k for all k ∈ Zd; thesequence {|λk |

|k| r}k∈Zd is non decreasing-type for some r > d2 Then there exists a positive constant csuch that for all m ∈ N and f ∈ Φλ,2(Td) we have that

Corollary 2.14 Let h(x) = 1 if x ∈ [−1, 1], h(x) = 0 if x 6∈ [−1, 1], βk = αk for all k ∈ Zd; thesequence {|λk |

|k| r}k∈Zd is non decreasing-type for some r > d2 Then there exists a positive constant csuch that for all m ∈ N and f ∈ Φλ,2(Td) we have that

|γk+(2m+1)j| = |λ−1k+(2m+1)j| ≤ c1 sup

k∈Z d \[−m/2,m/2] d

|λ−1k ||j|rwhere |j| = |j1| + + |jd| Hence

Note that for 2r > d thenP∞

j=1jd−1j−2r is convergent The proof is complete

2.3 Results for Lp(T) spaces

For this purpose, we define, for m ∈ N, the quantity

Now, we are ready to state the the following result

Theorem 2.15 If 1 < p < ∞ then there exists a positive constant c such that for all f ∈ Φλ,p(T) and

m ∈ N, we have that

kf − Qm(f )kp ≤ cεmkf kΦ

Before we give the proof of the above theorem, we recall, for 1 < p < ∞, that there exists a positiveconstant c such that for all f ∈ Lp(T) there holds the inequality

From this definition , we easily obtain the following lemma

Lemma 2.16 If 1 < p < ∞ then there exists a positive constant c such that for all f ∈ Lp(T) and

r, s ∈ Z we have

kGr,s(f )kp ≤ ckf kp (2.33)Proof The proof of the result is strong forward when s = r + 2m for some nonnegative integer m wehave that

Gr,s(x) = ei(r+m)x(Smg1)(x)where g1 is defined at x ∈ T as

g1(x) = ei(r+m)xg(x)

From this formula and inequality (2.31) we obtain inequality (2.33)

When r = r + 2m + 1 we have that

Gr,s(x) = ei(r+m+1)x(Sm+1g1)(x) − ei(r+2m+2)x(S0g2)(x)when now g1, g2 are defined at x ∈ T as g1(x) = ei(r+m+1)xg(x) and g2(x) = ei(r+2m+2)xg(x)

Now, we ready to present the proof of Theorem 2.15

Proof (Proof of Theorem 2.15) We define the kernel Pm(x, t) for x, t ∈ T as

We shall express the right hand side of equation (2.41) in an alternate form by using summation

by part For this purpose, we introduce the modified difference operator defined on vectors as

Λγk=

(

γk− γk+1, if j(2m + 1) − m ≤ k < j(2m + 1) + m

γk, if k = j(2m + 1) + m (2.41)With this notation in hand and the fact that, for k ∈ Im,j we have k0 = k − j(2m + 1), we concludethat

From this inequality and the definition of εm given in equation (2.29) we conclude that

kA+mkp≤ c1εmkgkp (2.43)

A bound on kA−mkp follows by a similar argument and yields the inequality

kA−mkp≤ c1εmkgkp (2.44)There still remains the task of bounding the second sum in equation (2.38) As before, we split itinto two parts

kBm−kp ≤ c1εmkgkp (2.48)

So far, we have proved inequality (2.30) The proof is complete

Now, we are ready to state the the following result

Corollary 2.17 Let 1 < p < ∞, h(x) = 1 if x ∈ [−1, 1], h(x) = 0 if x 6∈ [−1, 1] and {λk}∞k=0 is thedecreasing sequence and λk = λ−k = βk = β−k for all k ∈ Z Then there exists a positive constant csuch that for all f ∈ Φλ,p(T) and m ∈ N, we have that

kf − Qm(f )kp ≤ cεmkf kΦλ,p(T) (2.49)and

kQm(f )kp ≤ ckf kΦλ,p(T) (2.50)where εm=P

k∈Nλ−1k[m/2].Note that for

λj =

(

|j|r if j 6= 0

1 if j = 0then Φλ,p become Korobov function and we have the following estimate which have known in [14]

kf − Qm(f )kp ≤ cm−rkf kΦ

λ,p (T)

Theorem 2.18 Let 1 < p < ∞, h(x) = 1 if x ∈ [−1, 1], h(x) = 0 if x 6∈ [−1, 1] Then there exists apositive constant c such that for all f ∈ Φλ,p(T) and m ∈ N, we have that

From this theorem we have the following corollary

Corollary 2.19 Let 1 < p < ∞, λk = βk = e−s|k| for all k ∈ Z where s > 0 and h(x) = 1

if x ∈ [−1, 1], h(x) = 0 if x 6∈ [−1, 1] Then there exists a positive constant c such that for all

f ∈ Φλ,p(T) and m ∈ N, we have that

kf − Qm(f )kp ≤ ce−smkf kΦλ,p(T) (2.53)and

kQm(f )kp ≤ ckf kΦλ,p(T) (2.54)Proof From the hypothesis we have

Definition 2.20 Let β > 0 A function b : R → R will be called a mask of type β if b is an even, 2times continuously differentiable such that for t > 0, b(t) = (1 + |t|)−βFb(log |t|) for some Fb : R → Rsuch that |Fb(k)(t)| ≤ c(b) for all t > 1, k = 0, 1 A sequence {bk}k∈Z will be called a sequence mask oftype β.

Definition 2.21 We put λk:= λ−1k and βk:= βk−1

Theorem 2.22 Let 1 < p < ∞ and the sequence {λk}k∈Z = {β}k∈Z be a sequence mask of type r > 1and h(x) = 1 if x ∈ [−1, 1], h(x) = 0 if x 6∈ [−1, 1] Then there exists a positive constant c such thatfor all f ∈ Φλ,p(T) and m ∈ N,

kf − Qm(f )kp ≤ cm−r kf kΦλ,p(T).and

kQm(f )kp ≤ ckf kΦ

λ,p (T).Proof According the hypothesis we have

kf − Qm(f )kp ≤ cεm kf kΦλ,p(T)where

X

|k|>m/2

|∆λ−1k | ≤ 2c(λ)r + 1

rX

k∈N k1r From (2.55) and (2.56) we complete the proof

Definition 2.23 A function b : R → R will be called a exponent - type if b is 2 times continuouslydifferentiable and there exists a positive constant s such that b(t) = e−s|t|Fb(|t|) for some decreasingfunction Fb : [0, +∞) → [0, +∞) The sequence {bk}k∈Z be called a sequence exponent - type

Theorem 2.24 Let 1 < p < ∞ and the sequence {λ}k∈Z be a sequence mash of type r > 1, thesequence {βk}k∈Z be a exponent - type Then there exists a positive constant c such that for all

f ∈ Φλ,p(T) and m ∈ N,

kf − Qm(f )kp ≤ cm−rkf kΦλ,p(T) (2.57)and

kQm(f )kp ≤ ckf kΦ

Proof We have known in Theorem 2.22 that

where the univariate operator Qmj is applied to the univariate function f by considering f as a function

of variable xj with the other variables held fixed, Zd+ := {k ∈ Zd : kj ≥ 0, j ∈ Nd} and kj denotesthe jth coordinate of k

Set Zd

−1 := {k ∈ Zd : kj ≥ −1, j ∈ Nd} For k ∈ Z−1, we define the univariate operator Tk in

Φλ,p(Td) by

Tk:= I − Q2k, k ≥ 0, T−1 := I,where I is the identity operator If k ∈ Zd−1, we define the mixed operator Tk in Φλ,p(Td) in themanner of the definition of (3.1) as

kTk(f )kp ≤ C2−r|k|kf kΦ

λ,p (T d )

with some constant C independent of f and k

Proof We prove the lemma by induction on d For d = 1 it follows from Theorems ?? and ?? Assumethe lemma is true for d − 1 Set x0:= {xj : j ∈ N [d − 1]} and x = (x0, xd) for x ∈ Rd We temporarilydenote by kf kp,x 0 and kf kKr

p (T d−1 ),x 0 or kf kp,x d and kf kKr

p (T),x d the norms applied to the function f

by considering f as a function of variable x0 or xdwith the other variable held fixed, respectively For

k = (k0, kd) ∈ Zd−1, we get by Theorem 2.22 and Corollary 2.12 and the induction assumption

For k ∈ Zd+, we write k → ∞ if kj → ∞ for each j ∈ Nd

Theorem 3.2 Let 1 < p < ∞, p 6= 2, r > 1 or p = 2, r > 1/2 and the function λ be a mask of type

r Then every f ∈ Φλ,p(Td) can be represented as the series

f = X

k∈Z d +

converging in Lp-norm, and we have for k ∈ Zd+,

kqk(f )kp ≤ C2−r|k|kf kΦ

with some constant C independent of f and k

Proof Let f ∈ Φλ,p(Td) In a way similar to the proof of Lemma 3.1, we can show that