3.3 Topological Variations in Terms of Kernel Memory
3.3.4 Representation of the Kernel Unit Activated
In the previous examples of the MIMO systems as shown in Figs. 3.5–3.8, some of the kernel units have (mono-/bi-)directional connections in between. Here, we consider the kernel unit that can be activated when a specific directional flow occurs between a pair of kernel units, by exploiting both the notation of the template matrix as given in (3.24) and modified output in (3.30) (the fundamental principle of which is motivated by the idea in Kinoshita (1996)).
3.3 Topological Variations in Terms of Kernel Memory 53
KB KA (A B)
KB KA
(A B) KAB
(B A)
KB KA
(A B) KB (A B) KA
xA(n) xB(n) xA(n) xB(n)
xA(n) xB(n)
xB(n)
xA(n)
KBA
KAB
Fig. 3.11.Illustration of both the mono- (on the left hand side) and bi-directional connections (on the right hand side) between a pair of kernel unitsKAandKB (cf.
the representation in Kinoshita (1996) on page 97); in the lower part of the figure, two additional kernel unitsKABandKBAare introduced to represent the respective directional flows (i.e. the kernel units thatdetectthe transfer of the activation from one kernel unit to the other):KA→KB andKB→KA
Fig. 3.11 depicts both the mono- (on the left hand side) and bi-directional connections (on the right hand side) between a pair of kernel units KA and KB (cf. the representation in Kinoshita (1996) on page 97).
In the lower part of the figure, two additional kernel unitsKABandKBA
are introduced to represent the respective directional flows (i.e. the kernel units that detect the transfer of the activation from one kernel unit to the other):KA→KB andKB →KA.
Now, let us firstly consider the case where the template matrix of both the kernel unitsKAB andKBAis composed by the series of activations from the two kernel unitsKA andKB, i.e.:
TAB/BA=
tA(1)tA(2). . . tA(p) tB(1)tB(2). . . tB(p)
(3.32)
54 3 The Kernel Memory Concept
whereprepresents the number of the activation status from timenton−p+1 to be stored in the template matrix and the element ti(j) (i: A or B, j = 1,2, . . . , p) can be represented using the modified output given in (3.30) as13
ti(j) =Ki(xi, n−j+ 1), (3.33) or, alternatively, the indicator function
ti(j) = 1 ; ifKi(xi, n−j+ 1)≥θK
0 ; otherwise (3.34)
(which can also represent a collection of the spike trains from two neurons.) Second, let us consider the situation where the activation regularisation factor of one kernel unitKA, say,κA satisfies the relation:
κA< κB (3.35)
so that, at timen, the kernelKB is not activated, whereas the activation of KA is still maintained. Namely, the following relations can be drawn in such a situation:
KA(xA(n−pd+ 1)) , KB(xB(n−pd+ 1))≥θK
KA(xA(n))≥θK
KB(xB(n))< θK (3.36)
wherepdis a positive value. (Nevertheless, due to the relation (3.35) above, it is considered that the decay in the activation of both the kernel unitsKAand KB starts to occur at timen, given the input data.) Figure 3.12 illustrates an example of the regularisation factor setting of the two kernel unitsKA and KB as in the above and the time-wise decaying curves. (In the figure, it is assumed thatpd= 4 andθK = 0.7.)
Then, ifpd < p, and, using the representation of the indicator function given by (3.34), for instance, the matrix
TAB=
0 1 1 1 1 0 0 0 1 1 1 1
(3.37) can represent the template matrix for the kernel unitKAB (i.e. in this case, p = 6 andpd = 4) and hence the directional flow of KA → KB, since the matrix representation describes the followingasynchronousactivation pattern betweenKA andKB:
1) At time n−5, neitherKA norKB is activated;
2) At time n−4, the kernel unitKAis activated (but not KB);
13Here, for convenience, a unique time indexnis considered for all the kernels in Fig. 3.11, without loss of generality.
3.3 Topological Variations in Terms of Kernel Memory 55
0 2 4 6 8 10
0 0.2 0.4 0.6 0.8 1
n K A (n)
0 2 4 6 8 10
0 0.2 0.4 0.6 0.8 1
n KB (n)
θK θK
Fig. 3.12.Illustration of the decaying curves exp(−κi×n) (i: A or B) for modelling the time-wise decaying activation of the kernel unitsKAandKB;κA= 0.03,κB= 0.2,pd= 4, andθK = 0.7
3) At time n−3, the kernel unitKB is then activated;
4) The activation of both the kernel unitsKA andKB lasts till the time n−1;
5) Eventually, due to the presence of the decaying factor κB, the kernel unitKB is not activated at timen.
In contrast to (3.37), the matrix (with inverting the two row vectors in (3.37))
TBA=
0 0 1 1 1 1 0 1 1 1 1 0
(3.38) represents the directional flow ofKB →KAand thus the template matrix of KBA.
Therefore, provided a Gaussian response function (with appropriately given the radius, as defined in (3.8)) is selected for either the kernel unit KAB or KBA, if the kernel unit receives a series of the lasting activations from KA and KB as the inputs (i.e. represented in spiky trains), and the activation patterns are close to those stored as in (3.37) or (3.38), the kernel units can represent the respective directional flows.
A Learning Strategy to Obtain the Template Matrix for Temporal Representation
When the asynchronous activation betweenKAandKB occurs and provided thatp= 3 (i.e. for the kernel unitKAB/KBA), one of the following patterns
56 3 The Kernel Memory Concept
can be obtained using the indicator function representation of the spike trains by (3.34):
KA(xA(n)):ã ã ã0 1 0 0 0 0 0ã ã ã KB(xB(n)):ã ã ã0 0 0 0 0 1 0ã ã ã
In the above, it is not sufficient to represent the asynchronous activation pattern byKAB (orKBA).
It is then considered that there are two alternative ways to adjust the template matrix for the kernel unit KAB (or KBA) that can represent the asynchronous activation pattern between the kernel unitsKA andKB:
1. Adjust the size of the template matrixTAB(i.e. varying the factor p; in this case, assuming thatκi =κinit (∀i)) ;
2. Update the activation regularisation factors for both the kernel units KAandKB
For the former, if we increase the number of columns of the template ma- trixp, until the activation fromKA and KB appears in both the rows (i.e.
p= 5):
KA(xA(n)):ã ã ã0 1 0 0 0 0 0ã ã ã KB(xB(n)):ã ã ã0 0 0 0 0 1 0ã ã ã
the asynchronous activation pattern can be represented by the template ma- trix, i.e.
TAB=
1 0 0 0 0 0 0 0 0 1
(3.39)
An Alternative Learning Scheme – Updating the Activation Regularisation Factors
Alternatively, the asynchronous activation pattern betweenKA andKB can be represented by updating the activation regularisation factors for both the kernel unitKA andKB, without varyingp: provided that the regularisation factor for all the kernel units are initially set asκi =κinit (where κinit is a certain positive constant), we update the activation regularisation factors for both the kernel unitKA andKB, i.e.κA andκB. Then, we may resort to the following updating rule: