Optimization.Marquardt Function

Minimizes the function of several variables by using the Marquardt optimization algorithm.

Pascal

function Marquardt(Fun: TRealFunction; GradHess: TGradHess; var Pars: Array of double; Const Consts: array of double; Const ObjConst: Array of TObject; out FMin: double; IHess: TMtx; out StopReason: TOptStopReason; const FloatPrecision: TMtxFloatPrecision; MaxIter: Integer = 500; Tol: double = 1.0E-8; GradTol: double = 1.0E-8; Lambda0: double = 1e-2; Verbose: TStrings = nil): Integer; overload;

File

Optimization

Parameters

Parameters	Description
Fun	Real function (must be of TRealFunction type) to be minimized.
GradHess	The gradient and Hessian procedure (must be of TGradHess type), used for calculating the gradient and Hessian matrix.
Pars	Stores the initial estimates for parameters (minimum estimate). After the call to routine returns adjusted calculated values (minimum position).
Consts	Additional Fun constant parameteres (can be/is usually nil).
ObjConst	Additional Fun constant parameteres (can be/is usually nil).
FMin	Returns function value at minimum.
IHess	Returns inverse Hessian matrix.
StopReason	Returns reason why minimum search stopped (see TOptStopReason).
FloatPrecision	Specifies the floating point precision to be used by the routine.
MaxIter	Maximum allowed numer of minimum search iterations.
Tol	Desired Pars - minimum position tolerance.
GradTol	Minimum allowed gradient C-Norm.
Lambda0	Initial lambda step, used in Marquardt algorithm.
Verbose	If assigned, stores Fun, evaluated at each iteration step. Optionally, you can also pass TOptControl object to the Verbose parameter. This allows the optimization procedure to be interrupted from another thread and optionally also allows logging and iteration count monitoring.

Returns

the number of iterations required to reach the solution(minimum) within given tolerance.

Example

Problem: Find the minimum of the "Banana" function by using the Marquardt method.

Solution:The Banana function is defined by the following equation:

Also, Marquardt method requires the gradient and Hessian matrix of the function. The gradient of the Banana function is:

and the Hessian matrix is :

Uses MtxVec, Math387, Optimization; function Banana(const Pars: TVec; const Consts: TVec; const PConsts: array of TObject): double; begin Banana := 100*Sqr(Pars[1]-Sqr(Pars[0]))+Sqr(1-Pars[0]); end; procedure GradHessBanana(Fun: TRealFunction; const Pars: TVec; const Consts: TVec; const PConsts: array of TObject; const Grad: TVec; const Hess: TMtx); begin Grad[0] := -400*(Pars[1] - Sqr(Pars[0]))*Pars[0] - 2*(1 - Pars[0]); Grad[1] := 200*(Pars[1] - Sqr(Pars[0])); Hess[0,0] := -400*Pars[1] + 1200*Sqr(Pars[0])+2; Hess[0,1] := -400*Pars[0]; Hess[1,0] := Hess.Values[0,1]; // symmetric ! Hess[1,1] := 200; end; procedure Example; var Iters : integer; Pars : Array [0..1] of double; StopReason : TOptStopReason; begin // initial estimates for x1 and x2 Pars[0] := 0; Pars[1] := 0; Iters := Marquardt(Banana,GradHessBanana,Pars,[],[],FMin,StopReason,mvDouble,IHess); //stop if Iters > 500 or Tolerance < 1e-8 // Returns Pars = [1,1] and FMin = 0, meaning x1=1, x2=1 and minimum value is 0 end;

#include "MtxExpr.hpp" #include "Math387.hpp" #include "Optimization.hpp" // Objective function double __fastcall Banana(TVec* const Parameters, TVec* const Constants, System::TObject* const * ObjConst, const int ObjConst_Size) { double* Pars = Parameters->PValues1D(0); return 100.0*IntPower(Pars[1]-IntPower(Pars[0],2),2)+IntPower(1.0-Pars[0],2); } // Analytical gradient and Hessian matrix of the objective function void __fastcall BananaGradHess(TRealFunction Fun, TVec* const Parameters, TVec* const Consts, System::TObject* const * ObjConst, const int ObjConst_Size, TVec* const Grad, TMtx* const Hess) { double* Pars = Parameters->PValues1D(0); Grad->Values[0] = -400*(Pars[1]-IntPower(Pars[0],2))*Pars[0] - 2*(1-Pars[0]); Grad->Values[1] = 200*(Pars[1]-IntPower(Pars[0],2)); Hess->Values1D[0] = -400*Pars[1]+1200*IntPower(Pars[0],2)+2; Hess->Values1D[1] = -400*Pars[0]; Hess->Values1D[2] = -400*Pars[0]; Hess->Values1D[3] = 200; } void __fastcall Example(); { double Pars[2]; double fmin; TOptStopReason StopReason; // initial estimates for x1 and x2 Pars[0] = 0; Pars[1] = 0; int iters = Marquardt(Banana,GradHessBanana,Pars,1,NULL,-1,NULL,-1,fmin, StopReason, mvDouble, true,false,1000,1.0e-8,1.0e-8,NULL); // stop if Iters >1000 or Tolerance < 1e-8 }

Optimization, Functions, Example, See Also

What do you think about this topic? Send feedback!