# ECE 515, Homework 5

$$\def\Res{\mathtt{Res}}\def\Real{\mathbb{R}}$$

1. Consider optimal control problem for $$x\in\Real, u\in\Real$$:
$\dot{x}=u,\quad J_0^T=\int_0^T\frac{u^2}{2}dt +\cos(x(T))\to\min.$

• [20]
Find maximal $$T$$ for which the cost function (referred to as $$S$$ in the course notes) at time $$0$$,
$V(x, 0)=inf_{\{u(t)\}_{0\leq t\leq T}, x(0)=x}\ J_0^T$
is smooth as a function of the initial point $$x$$ (i.e. has continuous derivative).

Solution. As we discussed, the cost function for the problem above is given by
$V(x,0)=\min_X C(x,X,T), \mathrm{where} C(x,X,T)=\frac{(x-X)^2}{2T}+\cos(X)$
(here $$X=x(T)$$ is the potential position of the end of the trajectory, and $$(x-X)^2/2T$$ is the minimal cost of reaching it from $$x$$). The cost function loses smoothness when $$C$$ acquires non-unique minimum for some $$x$$. This can happen only if $$C$$ becomes nonconvex (in $$X$$, i.e. when the second derivative
$\frac{d^2C}{dX^2}=\frac1T-\cos(X)$
vanishes somewhere. This happens at $$T=1$$.

Graphs of $$X^2/2T+\cos(X)$$ for $$T=1.4, 1, .6$$.

• [15]
Describe $$V(0,0)$$ as a function of $$T$$.

Solution. We are looking at
$V(0,0)=\min_X \frac{(X)^2}{2T}+\cos(X)$
For $$T\leq 1$$ this is a convex function (of $$X$$), having vanishing derivative at $$0$$, and the corresponding minimum attained at $$0$$ is $$1$$.
For $$T\gt 1$$, there are two competing (global) minima, at point $$X(T)$$ solving
$X(T)/T=\sin(X(T)), 0\lt X(T)\lt \pi$
(there is no closed form expression for it), and the resulting cost, as a function of $$T$$, is shown below. Note that it converges to $$-1$$ as $$T\to\infty$$.

Cost at the origin, as a function of the time horizon.

2. Consider LQR problem with
$\dot{x}=Ax+bu, J=\int_0^T |c x|^2+r|u|^2 dt +m|x(T)|^2,$
where
$A=\left( \begin{array}{cc}-2&0\\0&1\end{array} \right), b=\left(\begin{array}{c}0\\1\end{array}\right),c=(1\quad 1).$

1. [15] Solve corresponding Riccati differential equation for $$T=1, r=1, m=0$$;
2. [20]
Same for $$T=1, r=1, m=-6$$. Is the solution regular (bounded, smooth) for all $$t$$?
3. [30]
Find smallest $$m$$ for which the RDE with $$P(T)=mE$$ – with $$E$$ the identity matrix, – is regular for any $$T$$.

Solution:
One constructs, in the usual fashion, the Hamiltonian matrix
$H=\left( \begin{array}{cccc} -2&0&0&0\\0&1&0&-1\\-1&-1& 2& 0\\-1& -1& 0& -1 \end{array} \right)$

Using computer algebra, one can find $$\exp(-T H)$$, and apply it to the $$2n\times n$$ matrix $$E;mE$$, corresponding to the terminal value of $$X,Y$$. Then one can find $$P=YX^{-1}$$ – the solution of RDE at $$t=0$$. The result is somewhat long:

$$P$$ matrix at $$t=0$$. For general $$t$$, replace $$T$$ by $$T-t$$.

To check, whether \P\) is bounded, it is enough to check that $$\det X$$ does not vanish.

Again, using computer algebra we get
$\det X(t)=e^{(2 – \sqrt{2}) T} (2 + \sqrt{2} – \sqrt{2} m + e^{2 \sqrt{2} T} (2 – \sqrt{2} + \sqrt{2} m))/4.$

We see that for $$m=0$$, the determinant is positive for all $$T$$, while for $$m=-6$$ it is vanishing at
$T-t=\frac{1}{2\sqrt{2}}\log{\frac{2+7\sqrt{2}}{7\sqrt{2}-2}}\approx .14485…$

It is immediate that $$\det X$$ vanishes for some $$T-t>0$$ iff
$m=-\frac{\sqrt{2}+1-e^{2 \sqrt{2}(T-t)}(\sqrt{2}-1)}{e^{2 \sqrt{2}(T-t)}-1}.$
As $$e^{2 \sqrt{2}(T-t)}$$ varies from $$1$$ to $$\infty$$, the values of $$m$$ that could lead to vanishing of $$\det X$$ lie in the interval $$(-\infty,-\sqrt{2}+1)$$. Thus the smallest $$m$$ ensuring regularity of $$P$$ is $$-\sqrt{2}+1$$.

### 15 Responses to ECE 515, Homework 5

1. Nicholas Alan Thompson December 5, 2018 at 8:43 am #

In the Problem 2 cost function, should |c*x| be |cx| so the dimensions match?

• yuliy December 5, 2018 at 1:03 pm #

It should, and it is, – belatedly, – now. Thanks!

2. Austin Hamlett December 7, 2018 at 10:26 am #

How would we solve |x(T)|^2 if x is a 2×1 column vector?

• Nicholas Alan Thompson December 7, 2018 at 10:59 am #

Think of |x(T)| as the norm, rather than the absolute value.

3. Anthony S Podkowa December 8, 2018 at 2:43 pm #

How are you defining “regular” in this context? I didn’t see anything in Basar’s notes, but it’s possible I missed something.

• yuliy December 8, 2018 at 3:28 pm #

Not exploding…

4. Anthony S Podkowa December 8, 2018 at 2:54 pm #

Can you be more explicit about the arguments of V in problem 1? The notes use a number of different conventions, with variations on x, u, and t.

• yuliy December 8, 2018 at 3:28 pm #

Updated.

• Heng-Sheng Chang December 9, 2018 at 11:17 am #

I thought $$V(x,0)$$ is minimizing $$J_0^T$$ with respect to $$u(t)$$ and $$x(T)$$, and therefore, $$V(x,0)$$ will be a function of $$x(0)$$. But the definition of $$V(x,0)$$ stated in problem 1.a is minimizing $J_0^T$ with respect to $$u(t)$$ and $$x(0)$$, then $$V(x,0)$$ will not be a function of $$x(0)$$. Not sure at which part i am getting it wrong.

• Heng-Sheng Chang December 9, 2018 at 11:22 am #

I thought $$V(x,0)$$ is minimizing $$J_0^T$$ with respect to $$u(t)$$ and $$x(T)$$, and therefore, $$V(x,0)$$ will be a function of $$x(0)$$. But the definition of $$V(x,0)$$ stated in problem 1.a is minimizing $$J_0^T$$ with respect to $$u(t)$$ and $$x(0)$$, then $$V(x,0)$$ will not be a function of $$x(0)$$. Not sure at which part I am getting it wrong.

5. Tixian Wang December 8, 2018 at 8:43 pm #

Isn’t the problem 2 too complicated to do hand calculation? It’s frustrating…… Using matlab doesn’t work either because the result is to complicated in variables. Is there a simpler way to deal with p2?

6. yuliy December 8, 2018 at 9:47 pm #

The result is moderately complicated, but does involve normal form of a 4×4 matrix, so that I expect some computer assistance.

• Tixian Wang December 8, 2018 at 10:19 pm #

Thanks

7. Tixian Wang December 8, 2018 at 10:16 pm #

For Problem 1 (b), is there an analytical solution to V(0,0) or do we just need to plot it?

• yuliy December 9, 2018 at 12:23 pm #

No close form solution is known; just describe the minimizers and sketch the value of the function.