Optimization.Marquardt Method

Overload List

#	Signature	Description
1	Int32 Marquardt(TRealFunction Fun, TGradHess GradHess, ref Double[] Pars, Double[] Consts, Object[] ObjConst, ref Double FMin, TMtx IHess, ref TOptStopReason StopReason, TMtxFloatPrecision FloatPrecision, Int32 MaxIter, Double Tol, Double GradTol, Double Lambda0)	Minimizes the function of several variables by using the Marquardt optimization method with no log.
2	Int32 Marquardt(TRealFunction Fun, TGradHess GradHess, ref Double[] Pars, Double[] Consts, Object[] ObjConst, ref Double FMin, TMtx IHess, ref TOptStopReason StopReason, TMtxFloatPrecision FloatPrecision, Int32 MaxIter, Double Tol, Double GradTol, Double Lambda0, TStrings Verbose)	Minimizes the function of several variables by using the Marquardt optimization algorithm.
3	Int32 Marquardt(TRealFunction Fun, TGradHess GradHess, ref Double[] Pars, Double[] Consts, Object[] ObjConst, ref Double FMin, TMtxFloatPrecision FloatPrecision)	Minimizes the function of several variables by using the Marquardt optimization method with no log.
4	Int32 Marquardt(TRealFunction Fun, TGradHess GradHess, ref Double[] Pars, Double[] Consts, Object[] ObjConst, ref Double FMin, ref TOptStopReason StopReason, TMtxFloatPrecision FloatPrecision)	Minimizes the function of several variables by using the Marquardt optimization method with no log.

Overload 1: Int32 Marquardt(TRealFunction Fun, TGradHess GradHess, ref Double[] Pars, Double[] Consts, Object[] ObjConst, ref Double FMin, TMtx IHess, ref TOptStopReason StopReason, TMtxFloatPrecision FloatPrecision, Int32 MaxIter, Double Tol, Double GradTol, Double Lambda0)

Minimizes the function of several variables by using the Marquardt optimization method with no log.

#	Name	Type	Description
1	Fun	TRealFunction
2	GradHess	TGradHess
3	Pars	Double[] (ref)
4	Consts	Double[]
5	ObjConst	Object[]
6	FMin	Double (ref)	output
7	IHess	TMtx	source TMtx
8	StopReason	TOptStopReason (ref)
9	FloatPrecision	TMtxFloatPrecision
10	MaxIter	Int32
11	Tol	Double	scalar
12	GradTol	Double	scalar
13	Lambda0	Double	scalar

Returns: Int32

Overload 2: Int32 Marquardt(TRealFunction Fun, TGradHess GradHess, ref Double[] Pars, Double[] Consts, Object[] ObjConst, ref Double FMin, TMtx IHess, ref TOptStopReason StopReason, TMtxFloatPrecision FloatPrecision, Int32 MaxIter, Double Tol, Double GradTol, Double Lambda0, TStrings Verbose)

Minimizes the function of several variables by using the Marquardt optimization algorithm.

#	Name	Description
1	Fun	Real function (must be of Dew.Math.TRealFunction type) to be minimized.
2	GradHess	The gradient and Hessian procedure (must be of Dew.Math.TGradHess type), used for calculating the gradient and Hessian matrix.
3	Pars	Stores the initial estimates for parameters (minimum estimate). After the call to routine returns adjusted calculated values (minimum position).
4	Consts	Additional Fun constant parameteres (can be/is usually nil).
5	ObjConst	Additional Fun constant parameteres (can be/is usually nil).
6	FloatPrecision	Specifies the floating point precision to be used by the routine.
7	FMin	Returns function value at minimum.
8	IHess	Returns inverse Hessian matrix.
9	StopReason	Returns reason why minimum search stopped (see Dew.Math.TOptStopReason).
10	MaxIter	Maximum allowed numer of minimum search iterations.
11	Tol	Desired Pars - minimum position tolerance.
12	GradTol	Minimum allowed gradient C-Norm.
13	Lambda0	Initial lambda step, used in Marquardt algorithm.
14	Verbose	If assigned, stores Fun, evaluated at each iteration step. Optionally, you can also pass Dew.Math.TOptControl object to the Verbose parameter. This allows the optimization procedure to be interrupted from another thread and optionally also allows logging and iteration count monitoring.

Returns: Int32 - the number of iterations required to reach the solution(minimum) within given tolerance.

Remarks:

What it computes. Minimizes a smooth real function of several
variables with the Levenberg-Marquardt method, solving a damped Newton step (H+lambda I) delta=-g and adapting the damping lambda (started from Lambda0) between gradient-descent and Newton behaviour. It needs both the gradient and the Hessian via GradHess. The standard test objective is

f(x_0,x_1) = 100 (x_1 - x_0^2)^2 + (1 - x_0)^2

with global minimum f=0 at $(1,1)$ , gradient

(d f)/(d x_0) = -400 (x_1 - x_0^2) x_0 - 2 (1 - x_0), (d f)/(d x_1) = 200 (x_1 - x_0^2)

and Hessian

H(f) = [ 1200 x_0^2 - 400 x_1 + 2  -400 x_0 ]
       [ -400 x_0                  200      ]

Domain. Pars holds the n starting coordinates;
GradHess returns the gradient and (symmetric) Hessian; MaxIter > 0; Tol,GradTol ≥ 0; Lambda0 > 0.

Defined behaviour. Returns the iteration count; Pars
holds the minimizer, FMin the minimum and IHess the inverse Hessian. StopReason reports the cause - Dew.Math.TOptStopReason.OptResConverged/Dew.Math.TOptStopReason.OptResSmallGrad on success, Dew.Math.TOptStopReason.optNANValue if the objective returned NaN, or Dew.Math.TOptStopReason.OptResBigLambda/Dew.Math.TOptStopReason.OptResSmallLambda/ Dew.Math.TOptStopReason.OptResMaxIter otherwise.

Examples

// Objective function
    private double Banana(TVec x, TVec c, params object[] o)
    {
        return 100*Math387.IntPower(x[1] - Math387.IntPower(x[0],2),2) + Math387.IntPower(1-x[0],2);
    }

    // Analytical gradient and Hessian matrix of the objective function
    private void BananaGradHess(TRealFunction Fun, TVec Pars, TVec Consts, object[] obj, TVec Grad, TMtx Hess)
    {
        double[] Pars = Parameters.PValues1D(0);

        Grad[0] = -400.0*(Pars[1]-Math387.IntPower(Pars[0],2))*Pars[0] - 2*(1-Pars[0]);
        Grad[1] = 200.0*(Pars[1]-Math387.IntPower(Pars[0],2));
        Hess.Values1D[0] = -400.0*Pars[1]+1200*Math387.IntPower(Pars[0],2)+2;
        Hess.Values1D[1] = -400.0*Pars[0];
        Hess.Values1D[2] = -400.0*Pars[0];
        Hess.Values1D[3] = 200.0;
    }

    private void Example()
    {
        double[2] Pars;
        double fmin;
        Matrix iHess = new Matrix(0,0);
        TOptStopReason StopReason;
    // initial estimates for x1 and x2
    Pars[0] = 0;
    Pars[1] = 0;
    int iters = Optimization.Marquardt(Banana,BananaGradHess,Pars, null, null, out fmin, iHess, out StopReason,
        TMtxFloatPrecision.mvDouble, 1000, 1.0e-8, 1.0e-8, null);
    // stop if Iters >1000 or Tolerance < 1e-8
}

See Also: TGradHess, TRealFunction, MtxIntDiff.NumericGradHess

Overload 3: Int32 Marquardt(TRealFunction Fun, TGradHess GradHess, ref Double[] Pars, Double[] Consts, Object[] ObjConst, ref Double FMin, TMtxFloatPrecision FloatPrecision)

Minimizes the function of several variables by using the Marquardt optimization method with no log.

#	Name	Type	Description
1	Fun	TRealFunction
2	GradHess	TGradHess
3	Pars	Double[] (ref)
4	Consts	Double[]
5	ObjConst	Object[]
6	FMin	Double (ref)	output
7	FloatPrecision	TMtxFloatPrecision

Returns: Int32

Overload 4: Int32 Marquardt(TRealFunction Fun, TGradHess GradHess, ref Double[] Pars, Double[] Consts, Object[] ObjConst, ref Double FMin, ref TOptStopReason StopReason, TMtxFloatPrecision FloatPrecision)

Minimizes the function of several variables by using the Marquardt optimization method with no log.

#	Name	Type	Description
1	Fun	TRealFunction
2	GradHess	TGradHess
3	Pars	Double[] (ref)
4	Consts	Double[]
5	ObjConst	Object[]
6	FMin	Double (ref)	output
7	StopReason	TOptStopReason (ref)
8	FloatPrecision	TMtxFloatPrecision

Returns: Int32