Solvers

Basin ships a growing catalog of solvers, all driven by the same Executor loop. Pick one based on what your problem provides (a gradient? only costs? residuals?) and whether it is constrained.

Each solver name links to its API page on docs.rs; the trailing note cites the paper it implements (or the source it ports).

First-Order

• GradientDescent: steepest descent with a pluggable step rule (Constant, Backtracking, MoreThuente, or Wolfe line searches) and optional heavy-ball momentum (Polyak, 1964).
• Sgd: mini-batch stochastic gradient descent with a constant learning rate and optional Polyak (1964) heavy-ball momentum.
• ProjectedGradientDescent: gradient descent for box-constrained problems, projecting each step back onto the feasible box.
• BrentDerivative: Brent’s method using first derivatives (“dbrent”): 1-D minimization on a bracketed interval, the gradient-using sibling of Brent/GoldenSection (uses the sign of f' to bracket and secant extrapolation on f' to step; Brent, 1973, as transcribed in Numerical Recipes §10.3).

Derivative-Free

• NelderMead: the classic reflection, expansion, and contraction simplex; needs only a CostFunction. Implements Lagarias et al. (1998); NelderMead::adaptive() uses the dimension-aware coefficients of Gao & Han (2012), and the projected variant follows Luersen & Le Riche (2004).
• Mads: mesh adaptive direct search (the deterministic OrthoMADS instance): polls a positive spanning set of 2n directions on a shrinking mesh, generated from the Halton sequence and a scaled Householder reflection. Unlike Nelder-Mead it has a convergence guarantee on nonsmooth or non-continuous objectives, a box-constrained variant (Mads::bounded()) via the extreme barrier, and a nonlinearly-constrained variant (Mads::constrained()) handling c(x) ≤ 0 via the progressive barrier (which tolerates an infeasible start). Needs only a CostFunction (plus NonlinearInequalityConstraints for the constrained mode). Implements Audet & Dennis (2006), OrthoMADS (Abramson, Audet, Dennis & Le Digabel 2009), and the progressive barrier (Audet & Dennis 2009).
• Newuoa: Powell’s model-based trust-region method: maintains a quadratic surrogate interpolating the objective on 2n+1 points and updates it by the least-Frobenius-norm rule, so each iteration needs only one new value. Needs only a CostFunction. Implements Powell (2006), cross-validated against PRIMA v0.7.2.
• Bobyqa: the bound-constrained sibling of Newuoa: the same least-Frobenius-norm quadratic model, with a box-aware trust-region step (TRSBOX) and geometry step (ALTMOV), plus the RESCUE restoration procedure. Needs a CostFunction and BoxConstraints. Implements Powell (2009), cross-validated against PRIMA v0.7.2.
• Lincoa: the linearly-constrained sibling of Newuoa: the same least-Frobenius-norm quadratic model, with a projected truncated-CG trust-region step (TRSTEP) and active-set QR (GETACT) that keep every iterate feasible under A x ≤ b. Needs a CostFunction and LinearConstraints (the general linear form: box bounds, equalities, and inequalities, all folded into A x ≤ b). Implements Powell (2015), ported from PRIMA v0.7.2.
• Cobyla: the nonlinearly-constrained Powell solver, and the odd one out: instead of a quadratic model it builds linear models by interpolation at the n+1 vertices of a simplex and steers by an L-infinity exact-penalty merit function, so it is the only one handling general nonlinear inequality constraints c(x) ≤ 0. Needs a CostFunction and NonlinearInequalityConstraints. Implements Powell (1994), ported from PRIMA.
• SolisWets: adaptive random local search: a randomized hill-climber testing x + b + d and x − b − d with d ~ N(0, ρ²I), a success-direction bias b, and a step size ρ that expands after 5 successive successes and contracts after 3 failures. The cheapest local search in the library (O(n) memory and time, cost evaluations only) and the classic memetic LS operator. Needs only a CostFunction. Implements Solis & Wets (1981).
• Brent: robust 1-D minimization on a bracketed interval (Brent, 1973, as transcribed in Numerical Recipes §10.2); also used inside line searches.
• GoldenSection: golden-section search: 1-D minimization on a bracketed interval, the robust linearly-converging companion to Brent (Kiefer, 1953; Numerical Recipes §10.1).

Quasi-Newton

• Bfgs: dense quasi-Newton with a full approximate Hessian (Nocedal & Wright, 2006).
• Lbfgs: limited-memory BFGS for larger problems (two-loop recursion, Nocedal & Wright Alg. 7.4).
• Lbfgsb: L-BFGS with box bounds; a faithful port of the Nocedal–Zhu L-BFGS-B v3.0 Fortran source (Byrd, Lu & Nocedal, 1995; Zhu, Byrd, Lu & Nocedal, 1997, ACM TOMS Alg. 778).

Trust Region

For problems exposing a Hessian or, matrix-free, a HessianProduct (either analytic, or synthesized by FiniteDiff):

• TrustRegion: second-order trust-region Newton minimizer (Nocedal & Wright, 2006, Alg. 4.1). The subproblem strategy is pluggable: Steihaug (truncated CG, the default), Dogleg, or CauchyPoint. TrustRegion::matrix_free() drives the subproblem purely through Hessian-vector products (HessianProduct), never forming a matrix, so it scales to large problems (Steihaug and CauchyPoint only; Dogleg needs the matrix).

Nonlinear Least Squares

For problems expressed as residuals (Residual + Jacobian):

• GaussNewton: undamped normal-equations solver (Madsen, Nielsen & Tingleff, 2004, §3.1).
• LevenbergMarquardt: damped least squares; the workhorse for curve fitting. Marquardt (1963) with Nielsen’s (1999) smooth damping update and Moré (1978)/MINPACK column scaling (Madsen, Nielsen & Tingleff, 2004, §3.2).
• Trf: trust-region reflective: Levenberg–Marquardt with box bounds (Branch, Coleman & Li, 1999, affine scaling).

Global & Stochastic

• CmaEs/BoundedCmaEs: covariance-matrix adaptation evolution strategy (Hansen, 2016), unconstrained and box-bounded; the bounded variant uses Hansen’s adaptive BoundPenalty (the pycma default).
• De: differential evolution (DE/rand/1/bin) over a feasible box (Storn & Price, 1997).
• RandomSearch: elitist (1+λ) uniform sampling over a box.
• Ssga: steady-state real-coded genetic algorithm (replace-worst) with BLX-α crossover (Eshelman & Schaffer, 1993), negative assortative mating (Fernandes & Rosa, 2001), and BGA mutation (Mühlenbein & Schlierkamp-Voosen, 1993); the SSGA component of Molina et al. (2010), §4.4.
• BasinHopping: basin-hopping (Wales & Doye, 1997): a Metropolis Monte-Carlo walk over the Ẽ(x) = min{f(x)} transform, wrapping any local solver with a pluggable step taker and acceptance test; adaptive step size on by default.

Memetic (Composed)

These run a local solver inside a global one, an example of Basin’s solver composition primitives:

• CmaInject/BoundedCmaInject: CMA-ES with per-generation local polishing of the best individuals, re-injected via Hansen’s (2011) injection mechanism.
• DeInject: the DE-flavored sibling of CmaInject: differential evolution (Storn & Price, 1997) with per-generation top-k local refinement and Hansen-style (2011) injection.
• MaLsChCma: a memetic algorithm with persistent local-search chains and a CMA-ES inner (Molina et al., 2010).
• MaLsChSw: the high-dimensional sibling of MaLsChCma: the same local-search-chain framework with a Solis-Wets inner, whose O(n) chain snapshots keep chain-memetic search viable when the dimension grows (MA-SW-Chains: Molina, Lozano, and Herrera, 2010, winner of the CEC’2010 large-scale competition). Both are type aliases of the generic MaLsCh<V, LS>, which accepts any local-search operator implementing the ResumableInner (seed + snapshot + resume) trait.

Constrained (Composed)

These wrap any gradient inner solver in an outer loop that handles linear constraints:

• BarrierMethod: log-barrier interior-point continuation for linear inequality constraints (A x ≤ b); needs a strictly feasible start (Boyd & Vandenberghe, Convex Optimization, §11.3, Alg. 11.1).
• AugmentedLagrangianMethod: quadratic penalty plus multiplier updates for linear equality constraints (A x = b); tolerates an infeasible start (Nocedal & Wright, §17.3, Alg. 17.4, LANCELOT-style).

The pluggable line searches are documented alongside the first-order solvers: Backtracking (Armijo; Nocedal & Wright §3.1), Wolfe (strong Wolfe; Nocedal & Wright Alg. 3.5/3.6), and MoreThuente (Moré & Thuente, 1994; a port of MINPACK-2’s dcsrch).

Backends and Constraints

Two design rules shape which solver you can use where:

• Constraints are first-class and type-checked. Box bounds live on the problem (BoxConstraints). Handing a constrained problem to an unconstrained solver is a compile error, not a runtime surprise.
• Backends are tiered. First-order and derivative-free solvers stay generic over the parameter type (Vec<f64>, nalgebra, ndarray, faer). Linear-algebra heavy solvers (the quasi-Newton and least-squares families) require a backend that implements the richer math they need, so an unsupported parameter type fails to compile rather than at runtime.

The matrix below summarizes support: ✓ means it compiles and runs on that parameter type, ✗ means it is a compile-time error (the tiering rule above).

Brent, GoldenSection, and BrentDerivative minimize over a scalar f64 interval (1-D), so the vector-backend choice does not apply.

† DeInject and BasinHopping are themselves backend-generic, but their effective coverage is the intersection of the outer driver and the chosen inner solver. The shipped inners (NelderMead, LevenbergMarquardt, Lbfgsb) are all backend-generic, so a backend-specific inner of your own is the only thing that would narrow it.

§ MaLsChSw bounds only on the vector tier—no matrix type is involved at all, unlike MaLsChCma, whose inner CMA-ES needs each backend’s symmetric eigendecomposition.

‡ TrustRegion’s coverage depends on the subproblem strategy: Steihaug (the default) and CauchyPoint need only matrix-vector products, so they run on every backend; Dogleg additionally needs a Cholesky solve (LinearSolveSpd), now available on all four backends. In exact mode on ndarray the Hessian must be supplied analytically as an Array2<f64>; FiniteDiff cannot synthesize one there (no dense-matrix constructor). Matrix-free mode has no such caveat: no matrix type is bound at all, and FiniteDiff synthesizes the Hessian-vector product on every backend.

Each solver’s page on docs.rs carries a Backends note listing exactly which parameter types it supports.

See Getting Started for a worked example, or open the Visualizer to watch several of these solvers converge.