Let us consider the adaptive algorithm consisting of (2.7) and (3.1) or (3.2) under the condition that C1 =0. To fix the ideas we focus on the pole placement prob- lem, but it will transpire that the analysis and main results apply to any reasonable design rule. First we study the possible steady state behavior of the closed-loop system under the condition that the external disturbances are zero(wk =0)and that the probing dksignal is stationary and sufficiently rich. The steady state anal- ysis indicates that the algorithm may be successful in an adaptive context. Next we consider an averaging analysis, first without signal perturbation, next with sig- nal perturbation. The results provide a fairly complete picture of the cases where the algorithm can be used with success and provide pointers to how it may fail.
This topic is taken up in the next section, where we study the influence of plants not belonging to the assumed model class.
Steady State Analysis
Let us consider the situationw1,k ≡0, w2,k ≡0 andzˆf,k−zf,k ≡0. The latter condition implies for either algorithm (3.1) or (3.2) that4k ≡4and(2k, 3k)= F(4) = (2, 3). There is no adaptation. Obviously, we want the closed-loop system to behave well under this condition for as there is no adaptation, there is no way the control algorithm can improve the performance. It can be argued that this property is almost necessary for any adaptive algorithm. In the literature it is often referred to as tunability or the tuning property, see Mareels and Polderman (1996). We show that either algorithm, the classic adaptive algorithm (3.2) as well as the filtered excitation (3.1), possesses the tuning property.
Indeed, givenzˆf,k −zf,k ≡ 0, we describe the adaptive closed-loop system via the following time-invariant system, making use of the fact thatw1,k≡0 and w2,k ≡0.
vk+1 zf,k+1
zk+1 vˆk+1 ˆ zf,k+1
=
As 0 Bs2 0 0
−Bf4∗Cs Af 0 0 0
0 BqCf Aq+3 0 0
0 0 Bs2 As 0
0 0 0 −Bf4Cs Af
vk zf,k
zk vˆk ˆ zf,k
+
Bs
0 0 Bs 0
dk. (4.1)
Because of the stability of A+H C we have thatx˜k converges to zero exponen- tially, and therefore, this state is omitted from (4.1). Moreover, xkis also omitted, as its stability is determined by the stability of the system (4.1). If the above sys- tem is stable, so is the complete system. We now exploit the factzˆf,k ≡ zf,k to rewrite (4.1) as:
vk+1 zf,k+1
zk+1 vˆk+1 ˆ zf,k+1
=
As 0 Bs2 0 0
−Bf4∗Cs Af 0 0 0
0 0 Aq+3 0 BqCf
0 0 Bs2 As 0
0 0 0 −Bf4Cs Af
vk zf,k
zk vˆk ˆ zf,k
+
Bs
0 0 Bs 0
dk. (4.2)
Observe now that the block diagonal matrix
"
As 0
−Bf4∗Cs Af
#
(4.3) is stable by construction. Moreover, the matrix
Aq+3 0 BqCf
Bs2 As 0
0 −Bf4Cs Af
(4.4)
is stable by virtue of the design with(2, 3)=F(4). It follows thus from equa- tion (4.2) that zk,vˆk,zˆf,k,vk and zf,k are all bounded. More importantly, from the observable signals point of view, it appears that the control objective has been achieved for the closed-loop system (2.7) with (3.1) or (3.2). Indeed, the closed- loop stability and performance hinges on the eigenvalues of the matrices:
• As, the unmodeled dynamics, which are outside the control bandwidth
• Af, the filter, free for the designer to choose, as long as it is stable
• A+H C, the observer eigenvalues for the nominal control design
• A+B F , the controlled nominal plant, and
• (4.4), the controlled model for the plant-model mismatch system, which are the poles of the closed loop(Q,S).
The above observation is independent of the nature of dk, and would even be true for dk ≡0. However, if dk is sufficiently rich in that it satisfies a condition like Assumption 6.4.2, then we have the additional result that the only steady state parameters are the true system parameters, that is,4 = 4∗and(2, 3) = F(4∗)=(2∗, 3∗). Actually, we desire that the spectrum of dkcontains at least as many distinct frequency lines as there are parameters in4to identify. This follows from the following construction. Introducev˜k =vk− ˜vk andz˜f,k =zf,k− ˆzf,k. Then from (4.1) we have:
v˜k+1=Asv˜k
˜
zf,k+1=Afz˜f,k−Bt(4∗−4)Csvˆk−Bf4∗v˜k. Also,
Csvˆk =
0 Cs 0
z I−
Aq+3 0 BqCf
Bs2 As 0
0 −Bq4Cs Af
−1
0 Bs 0
dk
from which it follows thatz˜f,k≡0 can only occur when4∗=4. We summarize our observations as follows:
Theorem 4.1 (Tuning Property). Consider the adaptive systems described by either (2.7) with the algorithm (3.1) or adaptive algorithm (3.2). Let the external disturbances be zero(wk ≡0). When the tuning errorzˆf,k −zf,k is identically zero, the algorithm’s stationary points(4k≡4, (2k, 3k)≡F(4))are such that the closed-loop system is stable and the desired control objective is realized.
Theorem 4.2 (Tuning property with excitation). Consider the adaptive sys- tems described by either (2.7) with algorithm (3.1) or (3.2). Let the external dis- turbances be zero(wk ≡0). Let the probing signal be sufficiently rich in that the spectrum of dk contains as many distinct frequency lines as there are elements in 4. The algorithm’s stationary point is unique.4k ≡4∗and the desired control objective is realized.
Remark. The difference between Theorem 4.1 and Theorem 4.2 is significant.
Polderman (1989) shows that only in the case of pole placement one can conclu- sively infer from Theorem 4.1 that the control achieved in the adaptive algorithm
equals the control one would have implemented if the system were completely known. In the case of LQ control, the achieved LQ performance is only optimal for the model, that is optimal for4, not for the plant4∗. Due to lack of excita- tion we are unable to observe this in the adaptively controlled loop. However, in the presence of excitation, due to the correct identification of4∗asymptotically optimal performance is obtained, for any control design. This goes a long way in convincing us why excitation, via the probing signal dk, is indeed important in an adaptive context. It is one of the main motivations for preferring the filtered exci- tation algorithm above the classical algorithm. Further motivation will emerge in the subsequent analysis.
Transient Analysis: Ideal Case
Exploiting standard results in adaptive control one can show that the classical algorithm (Algorithm 2), under the assumptions
• the plant belongs to the model class(C1=0)
• the external disturbances are zero(wk ≡0)
• along the solutions of the adaptive algorithmFis well defined
indeed realizes the desired control objective in the limit. Moreover, if the prob- ing signal is sufficiently rich, the actual plant will be correctly identified. The interested reader is referred to Mareels and Polderman (1996), Chapter 4, for a complete proof.
From a control performance perspective, which is the topic of this book of course, this result is however not very informative. Indeed the classical adaptive control results do not make any statements about important questions such as:
• How long do the transients take?
• How large is a bounded signal?
Indeed, a moment of reflection indicates that a general result can not make any statements about problems of the above nature. A result valid for (almost) all pos- sible initial conditions must allow for completely destabilizing controllers. For such cases it is not possible to limit either the size of the signals encountered in a transient nor the time it takes to reach the asymptotic performance. In general this situation is aggravated by imposing the condition of slow adaptation. How- ever, in the present situation, we can avoid the above disappointments, because we start from the premise that the nominal controller, however unsatisfactory its performance, is capable of stabilizing the actual plant. We proceed therefore with Algorithm 1 (Filtered excitation), exploiting explicitly the fact that our initial in- formation suffices to stabilize the system. By injecting a sufficiently rich prob- ing signal, which is conveniently filtered, we show that the adaptation improves (slowly) our information about the actual plant to be controlled, hence improving
our ability to control it. We regard this control strategy as one where we exploit the robustness margin of a robust stabilizing controller to such an extent that we learn the plant to be controlled in such a way as to improve the control perfor- mance. The existence of a robustness margin is crucial throughout the adaptation process. The more robust the controller, it turns out, the easier the adaptation pro- cess becomes. The algorithm is clearly achieving a successful symbiosis of robust control and adaptive control. Averaging techniques are exploited to establish the above results.
Let us be explicit about our stabilization premise:
Hypothesis 4.3. Along the sequence of estimates4k, k =0,1, . . ., the design ruleFis such that(2k, 3k) =F(4k)is a stabilizing controller for the actual plant to be controlled.
In the ideal scenario, the validity of this hypothesis is based on the following observations.
Introducev˜k = vk − ˆvk,z˜f,k = zf,k − ˆzf,k and4˜k = 4k −4∗. Along the solutions of the adaptive algorithms we have then, up to terms inwk:
v˜k+1
˜ zf,k+1
zk+1 vˆk+1 ˆ zf,k+1
=
As 0 0 0 0
−Bf4∗ Af 0 Bf4˜kCs 0 0 BqCf Aq+3k 0 BqCf
0 0 Bs2k As 0
0 0 0 −Bf4kCs Af
v˜k
˜zf,k zk vˆk ˆ zf,k
+
0 0 0 Bs 0
dk. (4.5)
By construction we have that the matrices
"
As 0
−Bf4∗ As
#
(4.6) and
Aq+3k 0 BqCf
Bs2k As 0
0 −Bf4kCs As
(4.7)
are stable (in that the eigenvalues for each instant k are less than 1 in modulus).
Hence, provided4k is slowly time varying in thatk4k+1−4kk is sufficiently small and4˜k is sufficiently small the overall system will be stable. As we will show
4˜k
is monotonically nonincreasing, andk4k+1−4kkis governed byà. Hence, assuming
4˜k
is sufficiently small, and à is sufficiently small, we have that Hypothesis 4.3 is satisfied along the solutions of the adaptive system.
In order to see that 4˜k
is decreasing, we proceed as follows. Introduce, λi j,k+1=Afλi j,k−BfEi jCshk, (4.8)
hk+1=Ashk+Bs2kzk. (4.9)
We have then, comparing (2.7) with (4.9) under the condition that C1=0, that zf,k =X
i j
γi j,k4∗i j+X
i j
λi j,k4∗i j, (4.10) and also
zˆf,k =X
i j
γi j,k4i j,k+X
i j
λi j,k4i j,k+O(à). (4.11) Here O(à) stands for a term fk that can be over bounded as |fk| ≤ K1àfor some K1>0 (see also Appendix C). Moreover, because the filter(Af,Bf,Cf) is designed such that it eliminates the spectrum of dk, and because we assume that the driving signalswk are orthogonal to dk, it follows that, for constant2:
lim
N→∞
1 N
N+m
X
k=m+1
γi j,kλi j,k−m =0 for all m=0,1,2, . . . and all i,j. The above expression embodies the most important design difficulty, on the one side dk must be sufficiently rich to lead to the identification of4∗, but we also need to filter it out of the Q loop via(Af,Bf,Cf), complicating the controller design.
For the adaptive update equation, see (3.1), we find thus after substituting (4.10) and (4.11):
4i j,k+1=4i j,k−àγi j,k
X
`,t
γ`t,k+λ`t,k 4∗`t−4`t,k
+O(à2)
for all i,j;k. (4.12) This equation (4.12) is in the standard form to apply averaging theory, see Ap- pendix C. Using the results from Appendix C we find for the averaged equation
vec 4akv+1
=vec 4akv
−à0vec 4akv−4∗
(4.13) where the matrix0contains as elements
Nlim→∞
1 N
N
X
k=1
γi j0,kγ`t,k
in appropriate order. For sufficiently rich dk, the matrix0 is positive definite, 0= 00 >0. It follows that4akvconverges exponentially to4∗, for sufficiently smallàand for sufficiently rich dk. Theorem C.4.2 informs us that,
à→lim0
4k−4akv =0.
Because dk has a finite spectrum, we are able to obtain the stronger result
4k−4akv
=O(à), for all k.
Hence for sufficiently smallà, optimal control is achieved up to small errors of the order of the adaptation step size, without losing the stability property of the initial stabilizing controller. We summarize:
Theorem 4.4 (Ideal case, filtered excitation). Consider the adaptive system (2.7) with (3.1) under the condition that the plant belongs to the model class C1 = 0. Let Assumption 3.1 be satisfied. Let d have a finite spectrum, but suf- ficiently rich as to enforce unique identifiability of4∗(d’s spectrum contains as many distinct frequency lines as there are elements in4∗). Let(Af,Bf,Cf)be chosen as to null the spectrum of d (2.5). Letwk be uncorrelated with dk. Then there exists aà∗ >0, such that for allà∈(0, à∗)and for all initial conditions such that40leads to a stable closed loop, we have that:
1. The adaptive system state is bounded, more precisely there exists constants C1>C2, C3>0 and 0< λ <1 such that
kXkk ≤C1λkkX0k +C2kdk +C3kwk, Xk=
v0k,vˆk0,zˆ0f,k,z0k,xˆ0k,xk00 , and
4k−4∗ ≤
40−4∗
+O(à).
2. Exponentially fast optimal control performance is achieved, in that there exists positive constants C4,C5,C6 >0 independent ofà, with 0 < 1− à∗C4<1, such that:
4k−4∗
≤C6(1−C4à)k
40−4∗ +C5à.
Remark.
The above result is applicable to Problems 1 and 2 of Section 7.2. We stress that it is important thatF(4)is Lipschitz continuous in4, otherwise the averaging result is not valid. (See Theorem C.2.2). This Lipschitz continuity may be guaranteed via Assumption 3.1. Indeed, under Assumption 3.1,Fdefined via either pole- placement or LQ control can be constructed to be Lipschitz continuous along the estimates generated by the adaptive algorithm.
More importantly, the same result would also apply to the whole class of adaptive algorithms(2k, 3k) = Fk(4k), such that Hypothesis 4.3 is satisfied. For pole placement and LQ control, this has been verified, but a range of alternatives is conceivable. Most noteworthy is the earlier suggestion to haveF(4k) = 0 for all k = 0,1, . . . ,k0, that is, wait to update the controller until4k0 is such that controller design via LQ or pole placement does not destroy the robustness mar- gin of the initial controller K . Also,Fk could reflect our desire to improve the closed-loop control performance as time progresses by, for example, increasing the bandwidth of the controlled loop.
Finally, we point out howFin the case of LQ control may be constructed. We consider the LQ index (see Problem 2 of Section 7.2):
J(S)= lim
N→∞
1 N
N
X
k=1
sk0Ssk+rk0rk
; S=S0>0 (4.14)
To constructFlet us assume that the matrix pair (2.9) is stabilizable and the matrix pair (2.10) is detectable. Denote
A= As 0
−Bf4Cs Af
!
, (4.15)
B= Bs 0
!
, (4.16)
C=
0 BqCf
. (4.17)
Under the stated conditions, (A,B) stabilizable and (A,C) detectable, we can solve the following Riccati equations for unique positive definiteRandP:
R=ARA0+I− ARC0
CRC0+I−1
CRA0 (4.18)
and
P=A0PA+S− A0PB
B0PB+S−1
B0PA. (4.19) Here I andSare the weighting matrices from the LQ index (4.14).
GivenRandPwe construct H= ARC0
CRC0+R−1
(4.20) and
K= B0PB+S−1
B0PA
, (4.21)
which have, respectively, the property thatA−HCandA−BKare stable matri- ces. The controller that solves the optimization of the index J of (4.14) can then be implemented in the standard way (see Chapter 2), with observer gainHand feedback gainK. In particular, we have
3= 0 0
−Bf4Cs 0
!
−HC (4.22)
and
2= −K. (4.23)
BecauseAis an affine function of4it follows thatR,Pand hence alsoH,K, and 3,2depend on4. Moreover it can be demonstrated that on any open subset of detectable matrix pairs(C,A(4))and on any open subset of stabilizable matrix pairs(A(4) ,B)this dependency on4is analytic. (See Mareels and Polderman (1996) for a proof of the analyticity property.)
Main Points of Section
The behavior of the adaptive closed-loop system is studied in the situation that the parameterized class of models contains the particular plant, the so-called ideal situation. First attention is paid to the possible no-adaptation behaviors. The in- direct adaptive algorithms introduced involving either filtered excitation or using classical adaptation, both enjoy the tuning property. The tuning property indicates that under conditions that the identification error is zero, the adaptively controlled closed loop appears to be controlled as if the true system is known. This estab- lishes that the steady state behavior of the controlled loop is as good as we can hope for. Next we consider the transient behavior. We establish that the filtered excitation adaptive algorithm in conjunction with a sufficiently rich probing sig- nal is capable of identifying the plant and hence achieves near optimal control performance. The result holds under the condition of sufficiently slow adapta- tion. The key idea in the filtered excitation algorithm is that as far as the probing signal is concerned the loop is essentially open. This achieves unbiased identifi- cation and allows one to exploit the robustness margin of the stabilizing controller without compromising the performance. Robust control and adaptation are truly complementary.