Ipopt interior point optimizer
[x_sol, f_sol, extra] = ipopt(problem) [x_sol, f_sol, extra] = ipopt(x0, f, df, g, dg, sparse_dg, dh, sparse_dh, var_lin_type, constr_lin_type, constr_rhs, constr_lhs, x_lower, x_upper, int_cb, params)
A structure with fields having the same name and meaning as the formal input arguments described below. When using this one argument syntax, parameters are given as fields of the problem structure.
real vector, the initial guess for x.
a function, a list or a string, the objective function.
the Scilab prototype can take one of the following forms
f_value = f(x,x_new) [f_value, df_value] = f(x,x_new) [f_value, df_value, h_value] = f(x,x_new)
x_new
is false if any evaluation method was previously
called with the same values in x
, true
otherwise.
The second form is expected when df is an empty matrix. It has to return the value of the gradient of the objective function when two output arguments are present.
The third form is expected when df and dh are empty matrices, no non-linear constraints are specified and exact hessian is expected (params.hessian_approximation == "exact"")). It has to return the value of the Hessian of the objective function when three output arguments are present.
the C prototype of the objective function is:
int f_C(double * x, double * f, int n_size_x, double x_new);
x_new
is false if any evaluation method was previously
called with the same values in x
, true
otherwise.
x
is a vector of size n_size_x
. It containts the
evaluation point.
f
is a scalar. It will containt the
value of the objective function.
This function returns 1 is nothing wrong happens. It returns 0 when something during the evaluation went wrong. By doing so, the optimization will be interrupted.
a function, a list or a string, the gradient of the objective function. It can be an empty matrix if f is a Scilab function returning the the objective function and its gradient (see above).
the Scilab prototype is:
df_value = df(x,x_new)
x_new
false if any evaluation method was previously
called with the same values in x
, true
otherwise.
the C prototype is:
int df_C(double * x, double * f, int n_size_x, double x_new);
x_new
is false if any evaluation method was previously
called with the same values in x
, true
otherwise.
x
is a vector of size n_size_x
. It containts the
evaluation point.
f
is a vector of size n_size_x
. It will containt the
value of the gradient of the objective function.
This function returns 1 is nothing wrong happens. It returns 0 when something during the evaluation went wrong. By doing so, the optimization will be interrupted.
a function, a list or a string, the value of the constraints.
the Scilab prototype can take one of the following two forms
g_value = g(x,x_new) [g_value, dg_value] = g(x,x_new)
x_new
is false if any evaluation method was previously
called with the same values in x
, true
otherwise.
The second form is expected when dg is an empty matrix. It has to return the values of the non empty elements of the Jacobian when two output arguments are present.
the C prototype is:
int g_C(double * x, int n_size_x, double * g, int n_size_g, double x_new);
x_new
false if any evaluation method was previously
called with the same values in x
, true
otherwise.
x
is a vector of size n_size_x
. It containts the
evaluation point.
g
is a vector of size n_size_g
. It will containt the
value of the constraints function.
This function returns 1 is nothing wrong happens. It returns 0 when something during the evaluation went wrong. By doing so, the optimization will be interrupted.
a function, a list or a string, the derivative/Jacobian of the constraints. It can be an empty matrix if g is a Scilab function returning the constraints and the constraints Jacobian (see above) or if jacobian_approximation field of params is set to "finite-difference-values".
the Scilab prototype is:
dg_value = dg(x,x_new)
The parameters of the prototype are:
The output dg_value
contains the non-zero values of the constraints Jacobian with following convention:
If sparse_dg is empty then dg_value
must contain the full Jacobian.
If sparse_dg is a full matrix then dg_value
must return the vector of non-zeros terms of the Jacobian.
If sparse_dg is a sparse matrix then dg_value
must return the Jacobian as a sparse matrix with the same zero terms.
x_new
false if any evaluation method was previously
called with the same values in x, true otherwise.
the C prototype is:
int dg_C(double * x, int n_size_x, double new_x, int n_size_g, int nnz_jac, double * values);
x_new
is false if any evaluation method was previously
called with the same values in x, true otherwise.
x
is a vector of size n_size_x
. It containts the
evaluation point.
values
is a vector of size nnz_jac
. It will containt the
values of the non zero elements of the Jacobian.
This function returns 1 is nothing wrong happens. It returns 0 when something during the evaluation went wrong. By doing so, the optimization will be interrupted.
A full or sparse matrix.
If sparse_dg is a full matrix with 2 columns it gives the sparsity pattern of the constraints Jacobian. Each row
has the form row,col
indicating that the corresponding term of the Jacobian is non-zero.
If sparse_dg is a sparse matrix it defines the sparsity pattern of the constraints Jacobian, i.e. all zero terms of sparse_dg are supposed to vanish in the Jacobian.
Note: the sparsity pattern is fixed at initialization hence supposed to be the same from one iteration to another.
If sparse_dg is empty the constraints Jacobian is supposed to be full.
a function, a list or a string, the Hessian of the Lagrangian. If dh is the empty matrix, then the Hessian is approximated as a full matrix with a limited-memmory BFGS formula.
the Scilab prototype is:
dh_value = dh(x,x_new,weight,lambda,lambda_new)
The parameters of the prototype are:
The output dh_value
contains the non-zero values of the Hessian with the following convention:
If sparse_dh is empty then dh_value
must contain the full Hessian as a square matrix.
If sparse_dh is a full matrix then dh_value
must return the vector of non-zeros terms of the Hessian.
If sparse_dh is a sparse matrix then dh_value
must return the Hessian as a sparse matrix with the same zero terms.
x_new
false if any evaluation method was previously
called with the same values in x, true otherwise.
weight
factor in front of the objective
term in the Hessian, .
lambda
the values for the constraint
multipliers, , at which the Hessian is
to be evaluated.
lambda_new
false if any evaluation method was previously
called with the same values in lambda, true
otherwise.
the C prototype is:
int dh_C(double * x, int n_size_x, double x_new, double obj_factor, int n_size_g, double * lambda, double lambda_new, int nnz_hess, double * values);
The parameters of the prototype are:
x
is a vector of size n_size_x
. It containts the
evaluation point.
values
is a vector of size nnz_hess
. It will contain the
values of the non-zero elements of the lower triangle of the Hessian.
lambda
the values for the constraint
multipliers (a vector of size n_size_g
,
, at which the Hessian is
to be evaluated.
obj_factor
is the weight coefficient in front of the objective
term in the Hessian, .
x_new
false if any evaluation method was previously
called with the same values in x
, true otherwise.
lambda_new
false if any evaluation method was previously
called with the same values in lambda
, true
otherwise.
hessian_approximation
field of params is
forced to "limited-memory".A full or sparse matrix.
If sparse_dh is a full matrix with 2 columns it gives the sparsity pattern of the Hessian. Each row
has the form row,col
indicating that the corresponding term of the Hessian is non-zero.
If sparse_dh is a sparse matrix it defines the sparsity pattern of the Hessian, i.e. all zero terms of sparse_dh are supposed to vanish in the Hessian.
Note: the sparsity pattern is fixed at initialization hence supposed to be the same from one iteration to another.
Since the Hessian is symmetric, you can eventually specify only terms of the lower triangular part.
If sparse_dh is empty the Hessian is supposed to be full.
a vector which indicates if a variable is linear or not
a vector which indicates if a constraints is linear or not
the right hand side vector of boundary of the constraints
the left hand side vector of boundary of the constraints
the lower boundary of the variables
the upper boundary of the variables
A function which is called at each iterations. This function allows (among other tasks, like plotting) the user to stop ipopt when he wants. The prototype of the intermediate callback function is:
continue_ipopt = intermediate_callback(params)
If continue_ipopt is equal to 1, then ipopt continues. Otherwise, if continue_ipopt is equal to 0, ipopt stops.
params is a structure which has the following fields:
x
: the current value of the primal variable.
lambda
: the current value of the Lagrange multiplier of the general constraints.
zL
: the current value of the Lagrange multiplier of the lower bound constraints.
zU
: the current value of the Lagrange multiplier of the upper bound constraints.
algorithm_mode
: 0 -
RegularMode or 1 - RestorationPhaseMode. The above 4 fields (primal and dual variables) are not available in restoration mode.
iter
: the current iteration
count. This includes regular iterations and iterations
while in restoration phase. If the algorithm is in the
restoration phase, the letter r' will be appended to the
iteration number.
obj_value
: the unscaled
objective value at the current point. During the
restoration phase, this value remains the unscaled
objective value for the original problem.
prev_obj_value
: the unscaled
objective value at the previous point.
inf_pr
: the scaled primal
infeasibility at the current point. During the restoration
phase, this value is the primal infeasibility of the
original problem at the current point.
inf_du
: the scaled dual
infeasibility at the current point. During the restoration
phase, this is the value of the dual infeasibility for the
restoration phase problem.
mu
: the value of the barrier
parameter mu.
d_norm
: the infinity norm
(max) of the primal step (for the original
variables x
and the internal slack
variables s
). During the restoration
phase, this value includes the values of additional
variables, p
and n
regularization_size
: the value
of the regularization term for the Hessian of the
Lagrangian in the augmented system.
alpha_du
: the stepsize for the
dual variables.
alpha_pr
: the stepsize for the
primal variables.
ls_trials
: the number of
backtracking line search steps.
curr_nlp_error
: the total optimality error for the original NLP at the current iterate, using scaling factors based on multipliers.
a structure which handles parameters allowing a fine tuning of ipopt (see Ipopt parameters).
the solution found by ipopt
the value of the objective function corresponding to the solution found by ipopt
a structure which handles the state of ipopt at the end of the optimization
Parameters of the extra structure
extra.status
:
0 - Optimal Solution Found.
This message indicates that IPOPT found a (locally) optimal point within the desired tolerances.
1 - Maximum Number of Iterations Exceeded.
This indicates that IPOPT has exceeded the maximum number of iterations as specified by the option 'max_iter'.
2 - Maximum CPU time exceeded.
This indicates that IPOPT has exceeded the maximum CPU time s as specified by the option 'max_cpu_time'.
3 - Search Direction is becoming Too Small.
This indicates that IPOPT is calculating very small step sizes and making very little progress. This could happen if the problem has been solved to the best numerical accuracy possible given the current scaling.
4 - Solved To Acceptable Level.
This indicates that the algorithm did not converge to the 'desired' tolerances, but that it was able to obtain a point satisfying the 'acceptable' tolerance level as specified by acceptable-* options. This may happen if the desired tolerances are too small for the current problem.
5 - Converged to a point of local
infeasibility. Problem may be infeasible.
The restoration phase converged to a point that is a minimizer for the
constraint violation (in the -norm), but is not
feasible for the original problem. This indicates that the
problem may be infeasible (or at least that the algorithm
is stuck at a locally infeasible point). The returned
point (the minimizer of the constraint violation) might
help you to find which constraint is causing the
problem. If you believe that the NLP is feasible, it might
help to start the optimization from a different point.
6 - Stopping optimization at current point as requested by user.
This message is printed if the user call-back method intermediate_callback returned false.
7 - Feasible point for square problem found.
This message is printed if the problem is 'square' (i.e., it has as many equality constraints as free variables) and IPOPT found a feasible point.
8 - Iterates divering; problem might be unbounded.
This message is printed if the max-norm of the iterates becomes larger than the value of the option 'diverging_iterates_tol'. This can happen if the problem is unbounded below and the iterates are diverging.
9 - Restoration Failed.
This indicates that the restoration phase failed to find a feasible point that was acceptable to the filter line search for the original problem. This could happen if the problem is highly degenerate, does not satisfy the constraint qualification, or if your NLP code provides incorrect derivative information.
10 - Error in step computation.
This messages is printed if IPOPT is unable to compute a search direction, despite several attempts to modify the iteration matrix. Usually, the value of the regularization parameter then becomes too large. One situation where this can happen is when values in the Hessian are invalid (%nan or %inf). You can check whether this is true by using the 'check_derivatives_for_naninf' option.
11 - Invalid number in NLP function or derivative detected.
12 - Problem has too few degrees of freedom.
This indicates that your problem, as specified, has too few degrees of freedom. This can happen if you have too many equality constraints, or if you fix too many variables (IPOPT removes fixed variables).
13 - Invalid option
(details about the particular error will be output to the console)
This indicates that there was some problem specifying the options. See the specific message for details.
14 - Not enough memory.
An error occurred while trying to allocate memory. The problem may be too large for your current memory and swap configuration.
15 - Unknown SolverReturn value - Notify IPOPT Authors.
An unknown internal error has occurred. Please notify the authors of IPOPT via the mailing list.
extra.lambda
: the Lagrange
multipliers (the constraints multiplier)
extra.zL
: the Lagrange
multipliers for the lower bound constaint
extra.zU
: the Lagrange
multipliers for the upper bound constraint
extra.it_count
: iteration needed to solve
the problem
extra.cpu_time
: cpu time needed to solve
the problem
extra.fobj_eval
: number of objective
function evaluation
extra.fobj_grad_eval
: number of gradient
of objective function evaluation
extra.constr_eval
: number of constraint
function evaluation
extra.constr_jac_eval
: number of gradient
of constraint function evaluation
extra.hess_eval
: number of hessian
function evaluation
extra.dual_inf
: dual infeasibility of the
problem
extra.constr_viol
: constraint
violation
extra.complementarity
:
complementarity
extra.kkt_error
: kkt error
Optimize the following non linear optimization problem with equality and inequality constraints:
This interface uses the Ipopt (Interior Point Optimizer) method to perform the optimization of a non-linear optimization problem with constraints.
// Definition of the optimization problem // The objective function function y=f(x, x_new) y=4*x(1) - x(2)^2 - 12; end function y=df(x, x_new) y(1) = 4; y(2) = -2*x(2); end // The constraints function y=g(x, x_new) y(1) = - 10*x(1) + x(1)^2 - 10*x(2) + x(2)^2 + 34; y(2) = 20 - x(1)^2 - x(2)^2; end function y=dg(x, x_new) y(1) = -10 + 2*x(1) y(2) = -10 + 2*x(2); y(3) = -2*x(1); y(4) = -2*x(2); end // The sparsity structure of the constraints sparse_dg = [1 1; 1 2; 2 1; 2 2]; // The Hessian of the Lagrangian (lower triangle non-zero terms) function y=dh(x, x_new, obj_weight, lambda, lambda_new) y(1) = lambda(1)*2 - lambda(2)*2; y(2) = -obj_weight*2 + lambda(1)*2 - lambda(2)*2; end // The sparsity structure of the lower triangle of the Hessian of the Lagrangian sparse_dh = [1 1;2 2]; upper = [15;15]; lower = [-15;-15]; x0 = [-12;-12]; // Feasible starting point var_lin_type = [1 1]; // Non-Linear constr_lin_type = [1 1]; // Non-Linear constr_rhs = [0 0]; constr_lhs = [-10000 0]; //////////////////////////////////////////////////////////////////////// params = struct(); params.hessian_approximation = "exact"; [x_sol, f_sol, extra] = ipopt(x0, f, df, g, dg, sparse_dg, dh, sparse_dh, var_lin_type, constr_lin_type, ... constr_rhs, constr_lhs, lower, upper, [], params); disp(x_sol) disp(extra) | ![]() | ![]() |
function f=residual(x, _t, _y) z=x(1)*cos(x(2)*_t+x(3)).*exp(x(4)*_t.^2); r=z-_y; f=sum(r.*r); endfunction // a combined objective and gradient of objective function // with user parameters _t and _y function [f, df]=costf(x, new_x, _t, _y) f = residual(x); if argn(1) == 2 then n=length(x) df=zeros(n,1); xi=x; //gradient with complex step dx = 1e-100; for k=1:n xi(k)=complex(x(k),dx); df(k) = imag(residual(xi,_t,_y))/dx; xi(k)=x(k); end end endfunction // a combined constraints and constraints Jacobian function function [g, dg]=constr(x, new_x) g = norm(x)^2-6; if argn(1) == 2 then dg = 2*x; end endfunction // intermediate callback function with user parameters _t and _y function out=intcb(param, _t, _y) if isfield(param,"x") x = param.x; z=x(1)*cos(x(2)*_t+x(3)).*exp(x(4)*_t.^2); drawlater clf plot(_t,z,_t,_y,'o') gca().data_bounds(3:4)=[-1.5,1.5]; legend model data drawnow end out = %t; endfunction // generate the data to be fitted t=linspace(-10,10,100); y=cos(2*t+1).*exp(-0.1*t^2); y=y+rand(y,'normal')/10; // specify the optimization problem using a structure problem = struct(); problem.x0 = rand(4,1); problem.f = list(costf,t,y); problem.g = constr; problem.constr_rhs = 0; problem.constr_lhs = -%inf; problem.x_lower = [0 0 0 -%inf]; problem.x_upper = [%inf %inf %inf 0]; problem.int_cb = list(intcb,t,y); [x,f,extra] = ipopt(problem); disp(x,extra) | ![]() | ![]() |
function [f, g, H]=rosenfgh(x, xnew) f = 100*(x(2) - x(1)^2)^2 + (1-x(1))^2; if argn(1) > 1 // g required g = [-400*(x(2)-x(1)^2)*x(1)-2*(1-x(1)); 200*(x(2)-x(1)^2)]; if argn(1) > 2// H required H = [1200*x(1)^2-400*x(2)+2, -400*x(1); -400*x(1), 200]; end end end problem = struct(); problem.f = rosenfgh; problem.x0 = [0;0]; problem.hessian_approximation = "exact"; x=ipopt(problem); | ![]() | ![]() |
Wächter, A., Biegler, L. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106, 25–57 (2006). https://doi.org/10.1007/s10107-004-0559-y