In many applications, we have an optimal control problem: ˙q = f(q, u), min T
0 f0(q, u)dt with state constraints of the form c(q) ≤ 0. This is the case for the shuttle re-entry problem, where we have two active constraints, on the thermal flux and the normal acceleration. Our aim is to present tools which can be used to analyze such problems. A first method is to apply nec- essary conditions from maximum principle. They are not straightforward and we shall restrict our presentation to the conditions obtained by Weierstrass for Riemannian problems in the plane with obstacles. A second method is the one used to analyze the re-entry problem which is the following. First we analyze the small time optimal control for the system without taking into account the state constraints using the maximum principle and second-order conditions.
Secondly we construct a model to take into account the constraints.
Weierstrass Necessary Optimality Conditions
We consider the problem of minimizing the length of a curveq(.) of the form
T
0 L(q,q)dt,˙ avoiding a domainD of the plane, with smooth boundary.
In order to analyze the problem we must:
1) Determine if a boundary arc is optimal.
2) Determine what the conditions for an optimal arc are when entering, departing or reflecting on the boundary.
To get necessary optimality conditions we make variations of a reference trajectory and we use a standard formula to estimate the length variation.
The variations introduced by Weierstrass are the following (see Fig. 3.1).
To determine if a reference boundary arc is optimal he introduced at a point P of the interior of the previous arc a vector u normal to boundary.
He compares the length of the reference boundary arc to trajectories of the domain with same extremities and passing throughu.We get:
Proposition 3.22.A necessary optimality condition for a boundary arc is 1
r ≥ 1
˜
r at each point P of the boundary where 1
˜
r is the curvature of the boundary arc and 1
r the curvature of the extremal curve tangent toP to the boundary.
Fig. 3.1.Weierstrass variations.
Next we present a necessary condition satisfied by an optimal arc connect- ing to or departing from the boundary. They correspond to variations of the point entering the boundary and departing.
Proposition 3.23.A necessary optimality condition for connecting or de- parting the boundary is to be tangent to the boundary.
Finally, when reflecting on the boundary on a point making variations of the point, we obtain the following condition.
Proposition 3.24.Assume that the metric is given by L = x˙2+ ˙y2 with q= (x, y). Then when reflecting the boundary the optimal straight lines must have equal angles with the tangent to the boundary.
Small Time Minimal Syntheses for Planar Affine Systems with State Constraints
Generalities. We consider a system of the form: ˙q=X+uY,|u| ≤1, q= (x, y)∈IR2 with state constraints. We denote byω=pdqthe clock form defined on the set whereX, Y are independent by:ω(X) = 1 and ω(Y) = 0.
The singular trajectories are located on the setS: det(Y,[Y, X]) = 0 and the singular controlus is solution of:
p,[[Y, X], X] +us p,[[Y, X], Y] = 0,
the two form dω being 0 on S. A boundary arc γb is defined by c(γb) = 0 and differentiating we get: ˙c(γb) = LXc+ubLYc = 0. If LYc = 0 along the boundary arc, it is called of order one and the boundary control is given by ub=−LXc
LYc and it has to be admissible|ub| ≤1.
We take q0 ∈ {c = 0}, identified to 0. The problem is to determine the local optimality status of a boundary arct→γb(t) corresponding to a control ub and to describe the time minimal synthesis near q0. The first step is to construct a normal form, assuming the constraint of order one.
Lemma 3.9.Assume:
1) X(q0), Y(q0)linearly independent
2) The constraint is of order1, i.e.LYc(q0) = 0.
Then changing if necessaryuintoưu,there exists a local diffeomorphism preservingq0 identified to 0such that the constrained system is:
˙
x= 1 +ya(q) , y˙ =b(q) +u , y≤0.
Proof .Using a local change of coordinates preserving 0,we can identifyY to
∂
∂y and the boundary arc toγb=t→(t,0).The admissible space isy≤0 or y≥0.Changing if necessary uintoưuit can be identified toy≤0.
The generic case. In this case, we make additional assumptions:
1. Y(q0),[X, Y](q0) are linearly independent.
2. The boundary arc is admissible and not saturating at 0.
Under these assumptions we construct a local model, which corresponds to a nilpotent approximation to evaluate the small time policy. Indeed we have a(0) = 0, |b(0)|<1 and we can takea=a(0), b=b(0),the model being:
˙
x= 1 +ya,
˙
y=b+u, y≤0, the clock form is:ω= dx
1 +ay anddω= a
(1 +ay)2 dx∧dy.
Next we compute the local syntheses. First consider the unconstrained case. For small time, each point can be reached by an arc γ+γ− or γ−γ+. If a >0, dω >0 and each time minimal policy is of the formγ+γ−, γ−γ+being time maximal, and the opposite ifa <0,this can be obtained by considering the clock form or direct computations, because observe that ifa >0,ify >0 we increase the speed along thex−axisand decrease ify <0.
For the constrained case, the same reasoning shows that the boundary arc is optimal if and only ifa >0, more precisely:
Lemma 3.10.Under our assumptions, we have:
1. For the unconstrained problem, ifa >0an arcγ+γ− is time minimal and an arc γ−γ+ is time maximal and conversely ifa <0.
2. For the constrained problem, a boundary arc is optimal if and only ifa >0 and in this case, each optimal trajectory is of the formγ+γbγ−. Ifa <0each optimal arc is of the form γ−γ+.
Small Time Minimal Syntheses for Systems in Dimension 3 with State Constraints
Preliminaries. We consider a system of the form: ˙q = X +uY, |u| ≤ 1, c(q)≤0 with q= (x, y, z)∈IR3. A boundary arcγb is defined by c(γb) = 0
and differentiating we get:
˙
c(γb) = (LXc+LYc)(γb) = 0.
The arc is of order one if LYc = 0 and in this case the generic small time optimal synthesis is as in the planar case. Hence in the sequel we shall restrict our analysis to the case whereLYc= 0 holds identically, that isY is tangent to each hypersurfacec=constant. It is also the situation encountered in the re-entry problem.
In this case, along a boundary arc we have LXc = 0 and differentiating we get:
L2Xc+ubLYLXc= 0.
IfLYLXc= 0 along the boundary, the constraint is called of order 2 and a boundary control is given by:
ub=− L2Xc LYLXc.
In order to describe the small time optimal synthesis, we proceed as in the planar case, considering first the unconstrained case.
Assumption 12 Let q0 ∈IR3 and we assume that X, Y,[X, Y] are linearly independent at q0.
In this case we have a standard result.
Lemma 3.11.IfX, Y,[X, Y] form a frame atq0,the small time accessibility set: A+(q0) = t small q(t, q0, u) is homeomorphic to a closed convex cone, where the boundary is formed by the surfaces S1, S2,whereS1 is the union of trajectories of the form γ−γ+ andS2 is the union of trajectories of the form γ+γ−. Each point in the interior of the cone is reached by a unique trajectory with two switchings of the formγ−γ+γ− and a unique trajectory of the form γ+γ−γ+.
To construct optimal trajectories we must analyze the boundary of the small time accessibility set for the time extended system. We proceed as fol- lows. Differentiating twice the switching functionΦ= p(t), Y(q(t)) we get:
Φ(t) =˙ p(t),[Y, X](q(t)),
Φ(t) =¨ p(t),[[Y, X], X+uY](q(t)) .
If p(t),[[Y, X], Y](q(t)) is not vanishing we can solve ¨Φ(t) = 0 to compute the singular control:
us=− p,[[Y, X], X](q) p,[[Y, X], Y](q) .
If Y and [X, Y] are independent, p can be eliminated by homogeneity and uscomputed as a feedback control. IntroducingD= det(Y,[Y, X],[[Y, X], Y]) and D = det(Y,[Y, X],[[Y, X], X]), we get D (q) +usD(q) = 0. Hence in dimension 3 through each generic point there is a singular direction. Moreover, as in the planar case, the Legendre-Clebsh condition allows to distinguish between slow and fast directions in the non exceptional case whereX, Y,[X, Y] are not collinear. We have two cases, see [7].
• Case 1 : If X and [[X, Y], Y] are on the opposite side with respect to the plane generated byY,[X, Y], then the singular arc is locally time optimal ifu(t)∈IR.
• Case 2 : On the opposite, ifX and [[X, Y], Y] are in the same side, the singular arc is locally time maximal.
In the two cases, the constraint|us| ≤1 is not taken into account and the singular control can be strictly admissible if|us|<1,saturating if|us|= 1 at q0,or non admissible if|us|>1.We have 3 generic cases. AssumeX, Y,[X, Y] not collinear and letporiented with the convention of the maximum principle:
p(t), X+uY ≥0.Lettbe a switching time of a bang-bang extremal:Φ(t) = p(t), Y(q(t)) = 0. It is called of order one if ˙Φ(t) = p(t),[Y, X](q(t)) = 0 and of order two if ˙Φ(t) = 0 but ¨Φ(t) = p(t),[[Y, X], X+uY](q(t)) = 0 for u=±1.The classification of extremals near a point of order two is similar to the planar case. We have three cases:
• parabolic case: ¨Φ± have the same sign.
• hyperbolic case: ¨Φ+>0 and ¨Φ− <0.
• elliptic case: ¨Φ+<0 and ¨Φ−>0.
In both hyperbolic and parabolic cases, the local time optimal synthesis are obtained by using only the first-order conditions from the minimum principle and hence from extremality, together with Legendre-Clebsch condition in the hyperbolic case. More precisely we have:
Lemma 3.12.In the hyperbolic or parabolic case, each extremal policy is lo- cally time optimal. In the hyperbolic case each optimal policy is of the form γ±γsγ±. In the parabolic case, each optimal policy is bang-bang with at most two switchings.
The set B(q0, T) describing the time minimal policy at fixed time T is homeomorphic to a closed disk, whose boundary is formed by extremities of arcs γ−γ+ and γ+γ− with lengthT and the stratification in the hyperbolic case is represented on Fig. 3.2.
In the elliptic case, the situation is more complicated because there exists a cut-locus. The analysis is related to the following crucial result based on the concept of conjugate points defined in Section 3.5.7, adapted to the bang-bang case.
γ+γ−
γ−γ+
γ− γ+
γsγ−
γ−γsγ−
γ−γs
γs γ+γs
γ+γsγ+
γ−γsγ+ γsγ+
(a) hyperbolic γ+γsγ−
γ− γ+
γ+γ−
γ−γ+
γ+γ−γ+ γ−γ+γ−
L(q0)
(b) elliptic
Fig. 3.2. Time optimal synthesis.
Lemma 3.13.Consider a system q ˙ = X + uY , |u| ≤ 1 , q(t) ∈ IR3. Let q0 be a point such that each of the two triplets Y,[X, Y], [X −Y,[X, Y]] and Y,[X, Y], [X + Y,[X, Y]] are linearly independent at q0. Then near q0 each bang-bang locally time optimal trajectory has at most two switchings.
The local time optimal policy at fixed time is represented on the previous figures. There exists a cut locusL(q0) whose extremities are conjugate points on the boundary of the reachable set.
We shall now analyze the constrained case. If the order of the constraint is one, the situation is similar to the planar case analyzed in Section 3.5.8. Hence we shall assume that the constraint is of order 2. We restrict our analysis to the parabolic case, which corresponds to the space shuttle problem .
Geometric normal form in the constrained parabolic case and local synthesis.
For the unconstrained problem the situation is clear in the parabolic case.
Indeed X, Y, [X, Y] form a frame near q0 and writing:
[[Y, X], X ±Y ] = aX+ bY + c[X, Y],
the synthesis depends only upon the sign of a at q0. The small time reachable set is bounded by the surfaces formed by arcs γ−γ+ and γ+γ−. Each interior point can be reached by an arcγ−γ+γ−and an arc γ+γ−γ+. Ifa < 0 the time minimal policy is γ−γ+γ− and the time maximal policy is γ+γ−γ+ and the opposite if a > 0. To construct the optimal synthesis one can use a nilpotent model where all Lie brackets of length greater than 4 are 0. In particular the existence of singular direction is irrelevant in the analysis and a model where [Y,[X, Y]] is zero can be taken. This situation is called the geometric model. A similar model is constructed next taking into account the constraints, which are assumed of order 2. Moreover we shall first suppose that is Y.Xc= 0 along γb and the boundary control is admissible and not saturating. We have the following:
Lemma 3.14. Under our assumptions, a local geometric model in the parabolic case is:
˙
x=a1x+a3z
˙
y = 1 +b1x+b3z
˙
z= (c+u) +c1x+c2y+c3z, |u| ≤1
witha3>0,where the constraint isx≤0and the boundary arc is identified to γb :t →(0, t,0).Moreover we have [Y, X] =−a3 ∂
∂x −b3 ∂
∂y, [[Y, X], Y] = 0, [[Y, X], X] = (a1a3+a3c3) ∂
∂x+ (a3b1+b3c3)∂
∂y + (a3c1+b3c2+c23)∂
∂z,and [[Y, X], X] =aX mod{Y,[X, Y]}, with a=a3b1−a1b3 = 0. If the boundary arc is admissible and not saturating we have|c|<1. Moreover a3= [X, Y]c.
Proof .We give the details of the normalizations.
Normalization 1. Since Y(0) = 0, we identify locally Y to ∂
∂z. The local diffeomorphismsϕ= (ϕ1, ϕ2, ϕ3) preserving 0 andY satisfy: ∂ϕ1
∂z =∂ϕ2
∂z = 0 and ∂ϕ3
∂z = 1. Since the constraint is of order 2, LYc = 0 near 0 and Y is tangent to all surfacesc=α, αsmall enough, hence ∂c
∂z = 0.
Normalization2. Sincecis not depending onz,using a local diffeomorphism preserving 0 andY = ∂
∂z, we can identify the constraint toc=x.Then the system can be written: ˙x= X1(q), y˙ = X2(q), z˙ = X3(q) +u,and x≤ 0.
The secondary constraint is ˙x= 0,and by assumption a boundary arcγb is contained inx= ˙x= 0 and passing through 0.In the parabolic case the affine approximation is sufficient for our analysis and the geometric model is:
˙
x=a1x+a2y+a3z
˙
y=b0+b1x+b2y+b3z
˙
z=c0+c1x+c2y+c3z+u,
whereγb is approximated by the straight line:x= 0, a2y+a3z= 0.
Normalization3. Finally we normalize the boundary as follows. In the plane x= 0, making a transformation of the form:z =αy+z,we can normalize the boundary arc to x = z = 0. Using a diffeomorphism y = ϕ(y), the boundary arc can be parameterized as γb : t → (0, t,0). The normal form follows, changing if necessaryutoưu,and hence permuting the arcsγ+ and γ−.
Theorem 3.11.Consider the time minimization problem for the system:q˙= X(q)+uY(q), q(t)∈IR3,|u| ≤1with the constraintc(q)≤0.Letq0∈ {c= 0}
and assume the following:
1. Atq0, X, Y and[X, Y] form a frame and[[Y, X], X±Y](q0) = aX(q0) +bY(q0) +c[X, Y](q0),with a <0.
2. The constraint is of order 2 and both assumptions LY LXc = 0 along γb and the boundary control is admissible and not saturating, are satisfied at q0.
Then the boundary arc through q0 is small time optimal if and only if the arc γ−is contained in the non admissible domain c ≥0. In this case the local time minimal synthesis with a boundary arc is of the form γ−γ+Tγbγ+Tγ−, where γ+T are arcs tangent to the boundary arc.
Proof . The proof is straightforward and can be done using a simple reason- ing visualized on the normal form. In this case q0 = 0, the boundary arc is identified to t →(0, t,0) and due to a3 > 0, arcs tangent to γb corresponding to u = ±1, are contained in c ≤0 ifu = −1 and inc ≥0 ifu = +1. Let B be a point of the boundary arcγb, for small enoughB = (0, y0, 0). If u = ±1, we have the following approximations for arcs initiating from B :
x(t) = a3(c0 + c2y0 + u)t2
2 +o(t2) z(t) = (c0+c2y0+u)t+o(t).
The projections in the plane (x, y) of the arcsγ−γ+γ−andγ+γ−γ+joining 0 to Bare loops denoted ˜γ−γ˜+˜γ−and ˜γ+γ˜−γ˜+. The loops ˜γ−γ˜+˜γ−(resp.˜γ+γ˜−γ˜+) are contained inx≤0 (resp. x≥0). We can now achieve the proof.
Taking the original system, if the arc γ−γ+γ− joining 0 to B which is the optimal policy for the unconstrained problem is contained inc≤0,it is time minimal for the constrained case and the boundary arc is not optimal. In the opposite, we can join 0 toB by an arcγ+γ−γ+ in c≤0,but this arc is time maximal. Hence clearly the boundary arcγb is optimal.
Lie Bracket Properties of the Longitudinal Motion and Constraints Consider the system ˙q=X+uY, q= (x, y, z)∈IR3,where
X =ψ vsinγ ∂
∂r −(gsinγ−kρv2)∂
∂v + cosγ −g v +v
r
∂
∂γ , Y =ψk ρv ∂
∂γ, ψ= 1 ϕ,
ϕ= thermal flux, which describes longitudinal motion when the rotation of the Earth is neglected (Ω= 0) and g = g0
r2 is assumed to be constant. The following results, coming from computations, are crucial:
Lemma 3.15.In the flight domain wherecosγ= 0, we have:
1. X, Y,[X, Y] are linearly independent.
2. [[Y, X], Y]∈Span{Y,[Y, X]}.
3. [[Y, X], X](q) =a(q)X(q) +b(q)Y(q) +c(q)[X, Y](q)with a <0.
Lemma 3.16.AssumingCD andCL are constant, the constraints are of or- der2.
Application of classification to the space shuttle. The constraints are of order 2. In the part of the flight domain where the boundary arc is admissible and not saturating, the arc γ− is violating the constraint along the boundary.
Hence we proved:
Corollary 3.2.Assume Ω = 0 and consider the longitudinal motion in the re-entry problem. Then in the flight domain, a boundary arc is locally optimal and the small time optimal synthesis with fixed boundary conditions on(r, v, γ) is of the form γ−γ+Tγbγ+Tγ−.