Talk5 Optimization

10 mars 2019 1 / 30

Plan
1the Fixed point algorithms
2proximal operator
3primal-dual problem
4Alternating Direction Method of Multipliers
10 mars 2019 2 / 30

the Fixed point algorithms
Recall :
(Fixed point) : Let f : X →R a function continues on X. We say that αis
a fixed point of f if f(α) = α.
(Fermat’s rule) : Let f a proper function and x∈X. We have the
equivalence : x∈argminf ⇔0∈∂f(x).In other words
argminf =zer∂f=x∈X/0∈∂f(x)
10 mars 2019 3 / 30

the Fixed point algorithms
Objective :
We want to calculate numerically a minimizer of a convex function
f:X→]∞,+∞]. According to Fermat’s rule you have to find a point x
such as 0 ∈∂f(x). The case where f derivable, this is equivalent to finding
a zero of the gradient 0 =∇f(x). In this case a popular algorithm is the
gradient algorithm, consist tô generate a suite (xk:k=0,1, ...)defined
by :
xk+1=xk−γ∇f(xk).
We can note T(x) = x−γ∇f(x)and the algorithm becomes in form
xk+1=T(xk)such as any fixed point of T is a minimizer of f. There are
many optimization algorithms written in this form where T is an
application chosen so that his fixed points are solutions to the problem. It
is therefore necessary to highlight conditions on T which guarantee the
convergence of the algorithm to a fixed point.
10 mars 2019 4 / 30

α-averaged applications
Notation
Let L>0 and R,T:X→Xtwo applications. The image of x by R is
noted R(x) or Rx. The TR compound will also be noted more compactly
TR. We note the identity (I(x) = x)by I.
Definition 1 :
Application R is said L-Lipschitz if ∀(x,y)∈X2:||Rx−Ry|| ≤ L||x−y||.
If L < 1, we say that R is a contraction and if L=1, R so-called
non-expansive.
10 mars 2019 5 / 30