ECE 490, week of Jan. 14

\(\def\Real{\mathbb{R}}\)

Course outline is here.

Taylor formula

Spaces of differentiable functions on an interval \(I=[a,b]\subset\Real\):
\[
C(I,\Real), C^1(I,\Real),\ldots, C^n(I,\Real),\ldots
\]

For a function in \(C^1(I,\Real)\),
\[
f(y)=f(x)+\int_x^yf'(s)ds.
\]
Iterating (for functions in \(C^n(I,\Real)\), we obtain
\[
f(y)=f(x)+f'(x)(y-x)+\frac{f”(x)}{2!}(y-x)^2+\ldots+\frac{f^{(n-1)}(x)}{(n-1)!}(y-x)^{n-1}+
\int_{x<s_1<s_2<\ldots<s_n<y}f^{(n)}(s_1)\frac{(y-s_1)^{n-1}}{(n-1)!}ds.
\]

Implications:

  • Mean value theorem
  • If \(f”\geq 0\) on \(I\) and \(f'(x)=0\), then \(x\) is a global minimum of \(f\).

Several variables

Domain: \(U\subset\Real^n\), an open set.

Reminder: open, closed, bounded, compact sets.

Exercise: is the set \(\{|x|\leq 1, |y| \lt 1\}\) open? closed? bounded?

Partial and directional derivatives; \(\nabla f\).
Remark: the definition of the directional derivative depends on the other coordinates held constant.

Gateaux and Frechet derivatives
Gateaux derivative is in essence directional one. Frechet derivative asks for the function locally to be close, up to higher order terms;
\[
f(x+v)=f(x)+(l,v)+o(|v|).
\]

Reminder: \(o,O,\sim\) notation.

Theorem: Frechet differentiability implies Gateaux differentibility.

Not vice versa:

Example: \[f(x,y)=\frac{x^3y}{x^4+y^2}\]

Multivariate mean value:
If \(f\) is Gateaux differentiable,
\[
f(x+v)=f(x)+(f'(z),v)
\]
for some \(z\in(x,x+v)\)…

Gap between Gateaux and Frechet differentiability:

Theorem: if Gateaux differentiable function has continuous gradient, then it is Frechet differentiable.

Vector-Valued Functions

Differentibility for
\[
F:\Real^n\to\Real^m
\]
carries over from scalar (\(m=1\)) case: component-wise. Derivative now is a linear function \(\Real^n\to\Real^m\); given (if we fix coordinates) as matrix.

Jacobian \(DF(x)\), an \(m\times n\) matrix.

Chain rule for (Frechet) differentiable functions:
If
\[
H=G\circ F (F:\Real^n\to\Real^m; G:\Real^m\to\Real^l; H:\Real^n\to\Real^l)
\]
then
\[
DH=DG\circ DF.
\]

Remark: Gateaux differentiability is not enough.

Mean value theorem also fails (example?). But one has inequalities:
\[
\parallel F(x+v)-F(x)- DF(x) v\parallel\leq L \parallel v\parallel
\]
where \(L\) is a bound on the operator norm od \(DF(z)-DF(x)\) over the segment \((x,x+v)\).

Higher derivatives.

Iterating, one obtains higher derivatives of multivariate functions. They are essentially symmetric polylinear forms.

They are needed to formulate a multidimensional Taylor formula.

Taylor approximation: polynomial approximating the function to higher order,

\[
f(x+v)=f(x)+\sum_{|k|\leq K} \frac{1}{k_1!\cdots k_n!}\prod \frac{\partial^k f}{\partial^{k_1}x_1\cdots\partial^{k_n}x_n} v_1^{k_1}\cdots v_n^{k_n}+o(|v|^K).
\]

Examples:

  • Find coefficient \(a_{2,0,1}\) of \(\sin(2x_1-3x_2+x_3)\)
  • Find gradient of \(\log\det(X)\), where \(X\) is an invertible \(n\times n\) matrix
  • Find Jacobian of \(X^{-1}\).

Exercises:

  • Find derivative of
    \[
    t\mapsto \exp(tA)B\exp(tC)
    \]
    at \(t=0\) (here \(A,B,C\) are \(n\times n\) matrices).
  • Find Jacobian of
    \[
    \left(\begin{array}{c}x\\y\end{array}\right)\mapsto\left(\begin{array}{c}x^3-3xy^2\\3x^2y-y^3\end{array}\right)
    \]
No comments yet.

Leave a Reply