m_n_kappa.fitting.LevenbergMarquardt#

class m_n_kappa.fitting.LevenbergMarquardt(f_i, x_0, maximum_iterations=100, tolerance=0.1, max_line_search_iterations=100, minimum_residuum_change=1e-09, initial_damping_factor=1.0, exceptions_to_catch=<class 'IndexError'>)#

Bases: GaussNewton

Levenberg-Marquardt algorithm

New in version 0.2.0.

Enhances the GaussNewton algorithm, especially in cases when the starting-conditions are poor.

Parameters:

  • f_i (list[Callable]) –

    Number of methods representing a system of equations. Each method must …

    • … accept a list[float] that is in size equal to x_0.

    • … be of the form f(x) = 0

  • x_0 (list[float]) – starting-values for the iteration

  • maximum_iterations (int) – maximum number of iterations (must be greater zero, Default: 20)

  • tolerance (float) – tolerance-limit, must be greater zero (Default: 1.0).

  • max_line_search_iterations (int) – maximum nuumber of line-search iterations (Default: 100)

  • minimum_residuum_change (float) – Iteration stops in case the residuum changes less than the given value (Default: 0.000000001)

  • exceptions_to_catch (tuple[Exception] | Exception) – Exceptions that may arise during a run of the problem and that must be caught (Default: IndexError)

  • initial_damping_factor (float) –

Notes

To enhance the GaussNewton algorithm the damping-factor \(\lambda \cdot diag(\mathbf{J}^T \mathbf{J})\) is added to \(\Delta \vec{x}_{n}\). The full problem to solve by the Levenberg-Marquardt algorithm is then given as follows.

\[ \begin{align}\begin{aligned}A \Delta \vec{x}_{n} & = b\\\left[\mathbf{J}^T \mathbf{J})^{-1} + \lambda^{2} \cdot diag(\mathbf{J}^T \mathbf{J}) \right] \Delta \vec{x}_{n} & = - \mathbf{J}^T f(x_{n})\end{aligned}\end{align} \]

where \(\lambda`\) is a factor increasing the dumping factor. \(diag(\mathbf{J}^T \mathbf{J})\) describes the main diagonal of \(\mathbf{J}^T \mathbf{J}\).

\(diag(\mathbf{J}^T \mathbf{J})\) allows that the damping-factor scales with the Jacobian Matrix \(\mathbf{J}\) and makes sure that proportions between damping-factor and rest of the problem are kept (see Fletcher (1971)).

Factor \(\lambda`\) is initially chosen by the user and adapted during the iterative process depending on the step-size of the line-search \(\alpha_\mathrm{n}`\) in the previous iteration.

  • \(\alpha_\mathrm{n} < 0.1`\): \(\lambda_{n+1} = 2.0 \cdot \lambda_{n}`\) if \(\lambda_{n} > 1.0`\) or \(\lambda_{n+1} = 2.0`\) if \(\lambda_{n} \leq 1.0`\)

  • \(\alpha_\mathrm{n} = 1.0`\): \(\lambda_{n+1} = 0.5 \cdot \lambda_{n}`\)

With increasing damping-factor the method moves from a Gauss-Newton algorithm to a method of gradient descent.

References

  • Marquardt, D. W. (1963). An algorithm for least-squares estimation of nonlinear parameters. Journal of the society for Industrial and Applied Mathematics, 11(2), 431-441.

  • Fletcher, R. (1971) A modified Marquardt subroutine for non-linear least squares, report, Theoretical Physics Division, Atomic Energy Research Establishement, Harwell, Berkshire

Methods

info()

print info regarding iterations

Attributes

damping_factor

Damping factor

f_i

Number of methods representing a system of equations

f_x

Most recent result of the iterations

iteration

current iteration

line_search_step_size

current step-size of the line-search

max_line_search_iterations

maximum number of iterations during the line-search

maximum_iterations

maximum number of iterations

minimum_residuum_change

boundary condition regarding the minimum residuum

not_successful_reason

gives a reason if the computation was not successful

successful

indicates the success of the computation

tolerance

tolerance-limit

x_0

starting-values for the iteration

x_i

computed variables at iteration-step \(i\)
info()#

print info regarding iterations

Return type:

str

property damping_factor: float#

Damping factor

property f_i: list[Callable]#

Number of methods representing a system of equations

property f_x: Vector#

Most recent result of the iterations

property iteration: int#

current iteration

property line_search_step_size: float#

current step-size of the line-search

property max_line_search_iterations: int#

maximum number of iterations during the line-search

property maximum_iterations: int#

maximum number of iterations

property minimum_residuum_change: float#

boundary condition regarding the minimum residuum

property not_successful_reason: None | m_n_kappa.general.NotSuccessfulReason#

gives a reason if the computation was not successful

Returns:

Reason why computation was not successful

Return type:

None | NotSuccessfulReason

property successful: bool#

indicates the success of the computation

property tolerance: float#

tolerance-limit

property x_0: list[float]#

starting-values for the iteration

property x_i: Vector#

computed variables at iteration-step \(i\)