Unmodeled Dynamics Situation
In this section, in studying the unmodeled dynamics situation, we proceed along the same lines used for the analysis of the adaptive algorithm under ideal model- ing, see Section 6.4. In particular we assume that the external signalswkare both stationary and sufficiently rich.
Unfortunately, due to the presence of the unmodeled dynamics we can no longer conclude that J(2) is quadratic in 2. Indeed, we have for2constant, using Parseval’s Theorem:
J(2)= lim
N→∞
1 N
N
X
k=1
e0kRek
= 1 2π
Z 2π 0
W∗(eiθ)T2∗(eiθ)RT2(eiθ)W(eiθ)dθ.
(7.1)
Here the transfer function from the disturbancewk =w1,k w2,k
to ek =yk uk
, de- noted T2, has a state space realization as indicated in (5.5) and W(z)is the z- transform corresponding to the external signalw. Obviously T2 is not affine in 2, hence J(2)is not quadratic in2.
Under these circumstances, we can establish the following result:
Theorem 7.1. Consider the adaptive system described by (3.1), (3.2) and (5.5).
Letθ ≤ 2S. Assume that the conditions (2.5), (2.6) and (5.6) are met. Let the external signalwsatisfy the Assumptions 4.1 and 4.2. Then there exists aà∗ >
0 such that for allà ∈ (0, à∗)and k20k < θ and all x˜0, x0, z0 and v0, the system state is bounded in thatk2kk < 2Sand Lemma 5.2 holds. Consider the difference equation
2ai jv,k+1=2ai jv,k−à ∂J(2)
∂2i j
2=2av
k
+àbi j(2akv), (7.2) with= kCSkand
bi j(2)= lim
N→∞
1 N
N
X
k=1
(gi j,k(2)−γi j,k(2))0Rek(2), (7.3)
where gi j,k(2)−γi j,k(2)is described in (6.3) and ek(2)=
"
yk(2) uk(2)
#
(7.4) follows from (5.5) with2k = 2. Provided (7.2) has locally stable equilibria 2∗∈ B(0, 2S), then2k converges for almost all initial conditionsk20k< θto aà(1−γ )/2neighborhood of such an equilibrium.
Proof. Follows along the lines of Theorem 4.3.
Equation (7.2) is crucial in understanding the limiting behavior of2k. If no lo- cally stable equilibria exists inside the ball B(0, 2S), a variety of dynamical be- haviors may result, all characterized by bad performance. The offending term in (7.2) is the bias term bi j(2). Providedbi j(2)is small in some sense, good per- formance will be achieved asymptotically.
As explained, the bias bi j(2)will be minimal when the disturbance signal has little energy in the frequency band of the unmodeled dynamics. One conclusion is that the adaptive algorithm provides good performance enhancement under the reasonable condition that the model is a good approximation in the frequency band of the disturbances.
Notice that these highly desirable properties can be directly attributed to the exploitation of the Q-parameterization of the stabilizing controllers. Standard im- plementations of direct adaptive control algorithms are not necessarily robust with respect to unmodeled dynamics! (See for example Rohrs, Valavani, Athans and Stein (1985) and Anderson et al. (1986).)
The above result indicates that the performance of the adaptive algorithm may be considerably weaker than the performance obtained under ideal modeling, embodied in Theorem 4.3. Alternatively, Theorem 7.1 explores the robustness margins of the basic adaptive algorithm. Indeed, in the presence of model mis- match, the algorithm fails gracefully. Small model-plant mismatch (smallε) im- plies small deviations from optimality. Notice that the result in Theorem 7.1 re- covers Theorem 4.3, by settingε=0! However, for significant model-plant mis- match we may expect significantly different behavior. The departure from the de- sired optimal performance is governed by the bi j(2)term in (7.3). It is instructive to consider how this bias term comes about. A frequency domain interpretation is most appropriate:
bi j(2)= 1 2πε
Z 2π 0
W∗(eiθ)Tz∗(g−γ )
i j(eiθ)Tw∗z(eiθ)RTwe(eiθ)W(eiθ)dθ, where Tz(g−γ )i j has a state space realization as given in (6.3) and Twz,Twehave state space realizations given in (5.5), and are respectively the transfer functions from r to(g−γ )i j,wto z andwto e.
In Wang (1991) a more complete analysis of how the adaptive algorithm fails under model plant mismatch conditions is presented. It is clear however that under severe model mismatch the direct adaptive Q mechanism is bound to fail. Under such conditions reidentification of a model has to be incorporated in the adaptive algorithm. This is the subject of the next chapter.
Example. Having presented the general theory for direct adaptive-Q control, we develop now a simple special case were the external signal,w1 = 0, w2k = cosω1k and the adaptive-Q filter contains a single free parameter. Whereas the development of the general theory by necessity proceeded in the state space do- main, due to the slow adaptation the main insight could be obtained via transfer
J
y u
r
Adjustment law s
GS Hz cos 1k
FIGURE 7.1. Example
functions and frequency domain calculations. For the example we proceed by working with transfer functions.
Consider the control loop as in Figure 7.1. All signals u,y,r and s are scalar valued. Let the control performance variable be simply e=y. We are thus inter- ested in minimizing the rms value of y:
J(θ)= lim
N→∞
1 N
N
X
k=1
yk2(θ).
The plant is given by
G(z)= N(z)+S(z)V(z) M(z)+S(z)U(z). The controller is given by
K(z)=U(z)+θH(z)M(z) V(z)+θH(z)N(z). with M(z)V(z)−N(z)U(z)=1; N,S,V,M,U,H ∈ R H∞.
For constantθwe have
yk(θ)= −(M(z)+S(z)U(z))(V(z)+θH(z)N(z))
1−S(z)θH(z) cosω1k, uk(θ)= −M(z)+S(z)U(z))(U(z)+θH(z)M(z))
1−S(z)θH(z) cosω1k, γk(θ)=H(z)N(z)(M(z)yk(θ)−N(z)uk(θ)).
(7.5)
The update algorithm forθ(with exponential forgettingλ∈ (0,1)) is thus given by (compare with (3.5))
θk+1=(1−àλ)θk−àγkyk. (7.6) Following the result of Theorem 7.1 the asymptotic behavior of (7.6) is governed by the equation
θka+v1=(1−àλ)θkav−àg(θkav), (7.7) where g(θ)is given by
g(θ)= lim
N→∞
1 N
N
X
k=1
γk(θ)yk(θ).
When S(z)=0, that is the ideal model case, g(θ)can be evaluated using (7.5) as gi(θ)=
M(ejω1)
2
Re V(e−jω1)H(ejω1)N(ejω1)+θ
H(ejω1)N(ejω1)
2 , (7.8) where the index i reflects the situation that S(z)=0.
In general we obtain the rather more messy expression:
g(θ)=
M(ejω1)+S(ejω1)U(ejω1) 1−S(ejω1)H(ejω1)θ
(7.9)
ã
Ren
H(e−jω1)N(e−jω1)V(ejω1)o +θ
H(e−jω1)N(e−jω1)
2 . Let us discuss the various scenarios described in, respectively, Theorems 4.3, 4.5 and 7.1 using the above expressions (7.8) or (7.9).
1. Ideal case:λ=0 (Theorem 4.3), expression (7.8).
There is a unique, locally stable equilibrium; g(θ∗)=0, or θ∗= −Re
H(e−jω1)N(e−jω1)V(ejω1) H(e−jω1)N(e−jω1)
2 . (7.10)
This equilibrium achieves the best possible control. The adaptation approx- imates this performance. Indeed, it is easily verified that
J(θ∗)≤ J(θ) for allθ.
In this case the performance criterionkyk2= J(θ)is given by kyk2rms=
M(ejω1)
2
V(ejω1)+θHθ(ejω1)N(ejω1)
2.
In general, we can not expect to zero the output, unless the actual signal happened to be a constantω1 =0, in which case we indeed obtainθ∗ =
−V(1)/(H(1)N(1))and J(θ∗)=0.
2. Ideal case:λ6=0 (Theorem 4.5).
Again there is a unique, locally stable equilibrium, now given by θλ∗= −Re{H(e−jω1)H(ejω1)N(ejω1)}
M(ejω1)
2
λ+
M(ejω1)
2
Hθ(ejω1)N(ejω1)
2 . (7.11)
In this case, the performance ofθλ∗is no longer the best possible, but clearly for smallλ, we have that
θλ∗−θ∗
=O(λ), see (7.10) and (7.11).
3. Plant-model mismatch:λ=0 (Theorem 7.1).
Remarkably, there is again a locally stable equilibriumθ∗, but also there may exist an equilibrium at∞as g(θ)→ 0 forθ → ±∞. Clearly in the presence of unmodeled dynamics, the adaptive algorithm loses the property of global stability.
Despite the fact thatθ∗is always a locally stable equilibrium of (2.2), it may not be an attractive solution for the adaptive system. Indeed Theorem 7.1 requires that the closed-loop system(S(z),Hθ(z)θ)be stable, a A property that may be violated forθ =θ∗if S(z)is not small. Theorem 7.1 requires at least that
S(ejω1)H(ejω1)θ∗
< 1. If this is not the case, the adaptive system will undergo a phenomenon known as bursting. See Anderson et al.
(1986) or Mareels and Polderman (1996) for a more in-depth discussion of this phenomenon. Let it suffice here to state that wheneverθ∗, the equilib- rium of (2.2), is such that(S(z),H(z)θ∗)is unstable, the adaptive system performance will be undesirable.
The performance ofθ∗is also not optimal with respect to our control crite- rion. When S(z)6=0, the criterion becomes:
kyk2rms=
V(ejω1)+θH(ejω1)N(ejω1)
2
M(ejω1)+S(ejω1)U(ejω1)
2
1−S(ejω1)H(ejω1)θ
2 .
It can be verified that the performance at the equilibriumθ∗will be better than the performance of the initial controllerθ =0 if and only if
Ren
N(e−jω1)V(ejω1)H(ejω1)o Ren
S(ejω1)H(ejω1)o
≥ −1 2
Ren
N(e−jω1)V(ejω1)H(ejω1)o2
ã 1
V(ejω1)
+
S(ejω1) N(ejω1)
2! . This condition is always satisfied for sufficiently small
S(ejω1)
. The above expression for this example is the precise interpretation of our earlier ob- servation that the direct adaptive Q filter can achieve good performance provided S, the plant-model mismatch, is small in the passband of the con- troller and provided the external signals are inside the passband of the nom- inal control loop.
4. Plant-model mismatch:λ6=0.
Due to the presence ofλas well as model-plant mismatch, we now end up with the possibility of either 3 or 1 equilibria. Indeed the equilibria are the solutions ofλθ+g(θ)= 0 which leads to a third order polynomial inθ. For small values ofλ > 0, there is a locally stable equilibriumθλ∗ close toθ∗. Global stability is of course lost, and the same stability difficulties encountered in the previous subsection persist here.