We show that operator splitting techniques for solving optimization problems, such as Forward Backward Splitting (FBS) and Douglas Rachford Splitting (DRS), can be interpreted as scaled gradient methods applied to the unconstrained minimization of continuously differentiable functions. Inspired by the connection between the proximal minimization algorithm and the Moreau envelope, we call these functions Forward-Backward and Douglas-Rachford envelope. The new interpretation paves the way of devising new algorithms for composite and separable nonsmooth optimization problems, by simply applying any well known method for unconstrained smooth optimization. We present two specific applications of the proposed theory: First, a Forward-Backward Newton method with asymptotic quadratic convergence rate. Second, we derive complexity estimates for DRS and an accelerated version of DRS.

• (i) 2-step Q-linearly to a solution when one exists or
• (ii) 2-step Q-linearly to points on the linear subspace closest to the convex set defined by bounds when the problem is infeasible.

Fast Nonlinear Model Predictive Control (NMPC) and Moving Horizon Estimation (MHE) solvers are developed for mechatronic and aerospace applications. Here we present an automatic C-code generation strategy for real-time optimal control. The ACADO Code Generation tool is realized within the ACADO Toolkit software package for dynamic optimization (www.acadotoolkit.org). This tool exports highly customized solvers for NMPC and MHE which allow for advanced control strategies, including nonlinear measurement functions as well as the formulation of general nonlinear constraints along the process trajectory. Moreover, the tool allows the export of a customized integrator as a standalone tool. The use of code generated integrators with tailored sensitivity propagation schemes results in an efficient treatment of models ranging from explicit systems of Ordinary Differential Equations (ODE) to stiff and/or implicit systems of Differential Algebraic Equations (DAE).

This talk will give an overview of features and methods which are implemented in the code generation tool. This will include algorithms for efficient structure exploitation of optimal control problems, as well as the structure of an ODE/DAE. Furthermore, a recently developed exact Hessian method based on second order algorithmic differentiation (AD) techniques and preconvexification is going to be discussed.

Throughout the talk, some synthetic benchmark results will be presented as well as experimental results coming from real-world applications.

We describe a multiplicative update algorithm originally developed for solving large-scale non-negatively constrained convex quadratic programs. The iteration update is obtained by splitting the cost function into two quadratic forms, and achieves linear convergence to a fixed point, which is the optimum, without projection. The algorithm allows for rapid verification of correctness, fine parallelization, and asynchronous updates. A deeper analysis reveals that the method is an adaptive-scaling gradient algorithm, and hence it works well on mediocrely conditioned problems, while the slow-downs that occur when the trajectory gets near the boundaries can be alleviated by acceleration techniques.
General convex constrained quadratic programs can be solved by this method through their dual formulation, for which convergence is still guaranteed.
At today, the algorithm has been been used in real industrial applications in fields such as computer vision, radiation therapy, laser processing machines, and air conditioning. Ongoing research problems are the tight bounding of the number of operations for weakly convex problems, a deeper analysis of the theory of cost function splitting, and the customization to specific processor architectures.

Given an orthonormal basis representation, we consider the optimal (in the l_2 error sense) compression scheme of maintaining high energy coefficients. We study the optimization problems related to obtaining the tightest lower/upper bound on the distance between any two compressed datapoints. In particular, we consider the problem where a distinct set of coefficients is maintained for each data sequence, along with the energy of the compression error. We develop a fast algorithm to obtain an exact solution to the problem. The suggested solution provides the tightest provable estimation of the Euclidean distance or the correlation, and executes at least two order of magnitudes faster than a generic convex solver.

Most data mining operations include an integral search component at their core. The performance of similarity search or classification based on Nearest Neighbors is largely dependent on the underlying compression and distance estimation techniques. In the face of ever increasing data repositories and given that most mining operations are distance-based, it is vital to perform accurate distance estimation directly on the compressed data. Naturally, the quality or tightness of the estimated distances on the compressed objects directly affects the search performance. Our experimental results indicate that using the optimal bounds leads to a significant improvement in the pruning power of search compared to previous state-of-the-art, in many cases eliminating more than 80% of the candidate search sequences.