The most important realization is that in our set up the tuning property remains valid in the presence of unmodeled dynamics for either the classical or the filtered excitation adaptive algorithm. This is a consequence of the fact that we start from the premise that our initial controller is stabilizing. It is in sharp contrast with the classical position in adaptive control, where arbitrarily small perturbations may lead to instability, destroying the tuning property. Moreover, if we restrict ourselves to the filtered excitation algorithm, we will deduce that in spite of mod- eling errors, we do identify the best possible model in our model class to represent S. This property does not hold for the classic adaptive algorithm due to the highly nonlinear interaction between identification and control. As a consequence the filtered excitation algorithm has a much larger robustness margin for inadequate modeling of the S parameter than the classic algorithm. The main result is there- fore restricted to the filtered excitation algorithm.
Tuning Property
Although we argue that the tuning property is close to necessary for an adaptive algorithm, it is clear that in the nonideal case this property plays a less important role. This is because the conditionzˆf,k−zf,k ≡0 can hardly be expected to be satisfied along the solutions of the adaptive system. Following the same arguments as in the ideal case we find nevertheless that the tuning property also holds in the nonideal case. Indeed, the stability of the closed-loop adaptive system hinges on
the stability of the system (neglecting the driving signalswk):
v1,k+1
vk+1 zf,k+1
zk+1 vˆk+1 ˆ zf,k+1
=
A1 0 0 B12 0 0
0 As 0 Bs2 0 0
−BfC1 −Bf4∗Cs Af 0 0 0
0 0 BqCf Aq+3 0 0
0 0 0 Bs2 As 0
0 0 0 0 −Bf4Cs Af
v1,k
vk zf,k
zk vˆk ˆ zf,k
+
B1
Bs 0 0 Bs 0
dk. (5.1)
Again usingzˆf,k ≡zf,k, it follows that the stability of the loop depends only on the stability of the matrix
A1 0 0
0 As 0
−BfC1 −Bf4∗Cs Af
and the design matrix
Aq+3 0 BqCf
Bs2 As 0
0 −Bq4Cs Af
.
The former is stable by assumption, the latter by construction. This establishes the tuning property.
Expression (5.1), however makes it very clear that due to the presence ofv1,k, and certainly in the case of a sufficiently rich dk, we can not expect to have zf,k≡ ˆ
zf,k. In the absence of any probing signal dk and with no external disturbances wk≡0, it is possible to have zf,k ≡ ˆzf,k.
Identification Behavior
Let us now focus on what model will be identified in closed loop via the filtered excitation adaptive algorithm. Again, we rely on slow adaptation and the stabil- ity Hypothesis 4.3. Whereas in the ideal case, it is clear that Hypothesis 4.3 is fulfilled under very reasonable assumptions, this is no longer guaranteed in the nonideal model case. We postpone a discussion of this crucial hypothesis until we have a clearer picture of what the possible stationary points for the identification algorithm are.
Clearly, as before we have (see (4.11)) ˆ
zf,k =X
i j
γi j,k4i j,k+X
i j
λi j,k4i j,k+O(à).
However
zf,k =X
i j
γi j,k4∗i j+λi j,k4i j∗
+ν1,k+ν2,k,
where
ν1,k+1= Afν1,k−BfC1v1,k, v1,k+1= A1v1,k+B1dk, ν2,k+1= Afν2,k−BfC1v2,k, v2,k+1= A1v2,k+B12kzk. The adaptive update equation can thus be written as:
4i j,k+1
=4i j,k−àγi j0,k X
t,`
γt`,k+λt`,k 4t`,k−4∗t`
+ ν1,k+ν2,k
+O(à2).
(5.2) In the above expressionsλt`,kandν2,kare functions of4k, but as observed before, for fixed4we clearly have
lim
N→∞
1 N
N
X
k=1
γi j,kν20,k(4)=0,
lim
N→∞
1 N
N
X
k=1
γi j,kλ0t`,k(4)=0,
because bothν2andλt`are filtered versions of the signal z which does not contain the spectrum of d, as this is eliminated by the filter(Af,Bf,Cf). Computing the averaged equation for (5.2), we have thus:
vec 4akv+1
=vec 4akv
k−à0vec 4akv−4∗ +àM, where M consists of the elements
Nlim→∞
1 N
N
X
k=1
γi j0,kν1,k,
in appropriate order. If follows that the averaged equation, under persistency of excitation such that0=00>0, has a unique equilibrium:
vec 4a∞v
=vec 4∗
+0−1M.
Reinterpreting the above averages in the frequency domain, we see that the iden- tification process is equivalent to finding the best4parameter in an`2approxi- mation sense, that is
4a∞v=arg mink(S(z)−4B(z))dk2,
where
S(z)=C1(z I−A1)−1B1+4∗B(z) .
This is clearly the best we can achieve in the present setting, but unfortunately it is not directly helpful in a control setting. We discuss this in the next subsection.
Let us now summarize our main result thus far:
Theorem 5.1. Consider the adaptive system (2.7) together with (3.1). Let As- sumption 3.1 and Hypothesis 4.3 hold. Assume that the external probing signal d is sufficiently exciting in thatkS(z)−4B(z)dk2has a unique minimizer. More- over the filter(Af,Bf,Cf)nulls the signal d. Assume that the external signalw is stationary and has a spectrum which does not overlap with that of d in that:
klim→∞
1 N
N
X
k=1
wkdk=0.
Then for all initial conditions satisfying Hypothesis 4.3 in a compact domain there exists aà∗ > 0 such that for allà ∈ (0, à∗)the adaptively controlled loop is stable, in that all signals remain bounded. Moreover,
1.
4k−4akv
=δ(à),
2. lim sup
k→∞
4k−4a∞v
=δ(à), for some order functionδ(à)such that limà→0δ(à)=0.
The main difficulty is of course Hypothesis 4.3 which we discuss now.
Identification for Control
As indicated earlier, the asymptotic performance of the adaptive algorithm is gov- erned by:
4∞=arg mink(S(z)−4B(z))dk2. The corresponding closed-loop stability depends on
(2∞, 3∞)=F(4∞) , where by constructionFensures that the matrix
As 0 Bs2∞
−Bf4∞Cs Af 0 0 BqCf Aq+3∞
(5.3)
is a stable matrix. The closed-loop stability of the system is however determined by the stability properties of the matrix
A1 0 0 B12∞
0 As 0 Bs2∞
−BfC1 −Bf4∗Cs Af 0
0 0 BqCf Aq+3∞
. (5.4)
Let us interpret these in terms of transfer functions. Introduce S∞(z)=4∞B(z),
S(z)=C1(z I−A1)−1B1+4B(z), Q(z)=2∞ z I−Aq−3∞−1
BqCf z I−Af−1
Bf, 1(z)=S(z)−S∞(z).
Thus (5.3) being a stable matrix states that(S∞(z),Q(z))is a stable loop, and (5.4) stable expresses that(S(z),Q(z))is a stabilizing pair. A sufficient condition for the latter is that:
(I −Q(z)S∞(z))−11(z)
∞<1. (5.5)
Via the adaptive algorithm we have ensured that
k1(z)dk22<1, (5.6)
which does go a long way in establishing (5.5) but is not quite enough. It is in- deed possible that the minimization of (5.6) does not yield (5.5), and may even lead to instability in the closed-loop adaptive system. This indicates that the adap- tive algorithm leads to unacceptable behavior. A finite-time averaging result (see Appendix C), allows us to conclude that the adaptive algorithm will indeed try to identify S∞(z). This leads to a temporarily unstable closed loop, characterized by exploding signals. At this point averaging would no longer be valid, Hypothe- sis 4.3 being violated. But it does indicate that large signals in the loop are to be expected. Invariably the performance of such a controlled system is bad, even if the adaptive loop may recover from this explosive situation. Understanding what type of behavior ensues from this is nontrivial. For a discussion of the difficul- ties one may encounter we refer to Mareels and Polderman (1996, Chapter 9).
Suffice it to say that chaotic dynamics and instability phenomena belong to the possibilities.
In order to have that the minimization of (5.6) leads to (5.5) being satisfied, we should have either
1. a sufficiently general model to ensure that C1will be small, or
2. ensure that outside the frequency spectrum of d the controlled loop (S∞(z),Q(z))has small gain.
The link between identification and control is obvious in the equations (5.5) and (5.6) and has been the focus of much research. See, for example Partanen (1995), Lee (1994), Gevers (1993).
Main Points of Section
In the nonideal case, the model class is insufficient to describe the mismatch be- tween the plant and the nominal plant; the filtered excitation adaptive algorithm attempts to identify the best possible model in an`2sense. Unfortunately, this may not be enough to guarantee stability let alone performance. Indeed despite the fact that the initial model and controller is stable it may be that the best possible model in the model class leads to instability. The interdependency of identification and control is clearly identified. The key design variables are the choice of model class, the probing signal and the control objective, as exhibited in equations (5.6) and (5.5).
Example. First we demonstrate the idea behind the filtered excitation adaptive al- gorithm, without using the(Q,S)framework. The example will deviate slightly from the theory developed in this chapter in order to illustrate the flexibility of- fered by the averaging analysis. The example is an abstraction of a problem en- countered in the control of the profile of rolled steel products.
Consider the plant represented in Figure 5.1. The input u and output e are mea- surable. The control objective is to regulate e to zero as fast as possible. The signalwis an unknown constant. The plant output y is not measurable. The gain g∈(0,g¯), is unknown butg is a known constant.¯
The proposed solution, in the light of the filtered excitation adaptive algorithm, is to use a probing signal dk = (−1)kd, where d is a small constant, leading to an acceptable error in the regulation objective. The controlled plant becomes as presented in Figure 5.2.
Whengˆ =g, then we have dead beat response and e is regulated in three time steps. More precisely, with q−1the unit delay operator
q3e=(q−1)(q−12)(q+13)w+g(q−12)(q+13)(−1)kd.
The probing signal leads thus to a steady state error of|gd/2|in magnitude. Of course, due to the integral in the plant, there is no steady state error for any con- stantw.
gz1 e
w
u SUM
PL MI
y
FIGURE 5.1. Plant
e
u y
1kd
g
z 1 z 1
z 12 1 56z 13 g
z 23
FIGURE 5.2. Controlled loop
It is easily verified that the above system is stable for all g/gˆ ∈(0,2). It is thus advantageous to over estimate g. The filtered excitation adaptive algorithm can be implemented as follows:
ˆ
gk+1= ˆgk−à(−1)k
ek+gˆkd 2 (−1)k
; gˆ0= ¯g. The complete control loop is illustrated in Figure 5.3.
Indeed because of the filter we expect the steady state behavior of ek due to the probing signal to be−(gd/2)(−1)k, our estimate for this is−(gˆkd/2)(−1)k, which leads to the above update law. Now provided g/gˆk ∈ (0,2)and for suf- ficiently smallà, we can look at the averaged update equation to see how the adaptive system is going to respond. According to our development this leads to
ˆ
gak+v1= ˆgkav−à
−gd
2 +gˆkavd 2
, gˆa0v= ˆg0= ¯g, or
ˆ
gak+v1= ˆgkav−àd
2 gˆakv−g.
e
u y
Adaptation
g
k 1kd
1kd
ef g
z 1 z 1
z 12
56z 13 z 23
FIGURE 5.3. Adaptive control loop
0 50 100 150 200 250 300 350 400 Time index
1.6
1.2
0.8 1.4
1 g
FIGURE 5.4. Response ofgˆ
0 5 10 15 20 25 30 35 40
Time index 10
0
Control error
5
5 10
( 103)
FIGURE 5.5. Response of e
Hence, as expectedgˆakvconverges monotonically to g whenever 0 < àd < 2.
Now from Theorem 4.4 we conclude that for all g/gˆ0∈(0,2)
gˆk− ˆgakv
=O(à) for all k.
This leads to asymptotically near optimal performance for all g∈(0,g¯), actually for all g∈(0,2g¯−ε), whereεis any small constant 1ε > à >0.
A response of the algorithm is illustrated in Figures 5.4 and 5.5. Figure 5.4 illustrates the response of theg variable, while Figure 5.5 displays the responseˆ of the regulated variable. For the simulation we chooseàd =0.2, and all other initial conditions set to 1. Notice that the averaging approximation predicts the closed-loop behavior extremely well.
As can be seen from this example, stability of the plant to be controlled is not essential for the filtered excitation algorithm to be applied. The stability of the closed loop is, of course, important.
Simulation Results
In this subsection, we present simulations for the case where an explicit adaptive LQG algorithm is used to design an adaptive Q filter, denoted Qk. The actual plant G(S), and the nominal controller K used are designed based on the nominal
plant G of the example in Section 5.2. The following LQ index penalizing r and s is used in the design of the adaptive LQG augmentation Qk.
JL Q = lim
k→∞
k
X
i=1
(ri2+si2). (5.7)
Table 5.1 shows a performance index comparison for the following various cases. The first is where the actual plant G(S)is controlled by the LQG controller K for the nominal plant G with no adaptive augmentation. The second case is when the LQG controller for G is augmented with an indirect LQG adaptive-Q algorithm. A third order model S is assumed in this case, and the estimate of S
‘converges’ to
S= 0.381z−1−0.092 5z−2−0.358 8z−3
1−0.423 9z−1−0.439 2z−2−0.018 1z−3. (5.8) A marked improvement over that of the nonadaptive case is recorded. Note that the average is taken after the identification algorithm ‘converges’. The third case is for the actual plant, G(S), controlled by the corresponding LQG controller, designed based on knowledge of G(S)rather than on that of a nominal model G.
Clearly, the performance of the adaptive scheme (Case 2) is drastically better than for the nonadaptive case, and approaches that of the optimal scheme (Case 3), confirming the performance enhancement ability of the technique.
Case 1
k
k
X
i=1
(yi2+0.005u2i) 1. Actual plant G(S) with LQG
controller for nominal plant G. 0.423 0 2. Actual plant G(S)with nominal
LQG controller and adaptive Qk. 0.186 0 3. Actual plant G(S) with op-
timal LQG controller for G(S). 0.175 6 TABLE 5.1. Comparison of performance
In a second simulation run, we work with a plant G which is not stabilized by the nominal controller K . Again S is identified on line using a third order model and there is employed an adaptive LQG algorithm, as in the run above, to design a Qkto augment K . Figure 5.6 shows the plant output and input. In this instance, the adaptive augmentation, Qktogether with K stabilizes the plant. The results show that the technique not only enhances performance but also can achieve robustness enhancement.
0
0 50 100 150 200 250 300 350 400 450 500
Time Samples y
0
0 50 100 150 200 250 300 350 400 450 500
Time Samples u
3 000 3 000
2 000 2 000
1 000 1 000
1 000 1 000
2 000 2 000
3 000 3 000
FIGURE 5.6. Plant output y and plant input u
Main Points of Section
The first example serves to illustrate the powerful nature of the averaging tech- niques. Clearly the analysis can be used for design purposes. The filtered exci- tation algorithm provides a suitable way of injecting an external signal such as to identify the plant characteristics in a particular frequency range, without com- promising the control performance too much. The trade off between desired con- trol objective and the identification requirements can easily be analyzed in the frequency domain. The second example clearly demonstrates the strength of the (Q,S)framework.