pub fn huber(delta: f64, r: f64) -> f64Expand description
Huber Loss function
$$ L_\delta(r) = \begin{cases} \infty & \delta < 0 \\ \frac{1}{2} r^2 & |r| \le \delta \\ \delta \left(|r| - \frac{1}{2} \delta \right) & \text{otherwise} \end{cases} $$
Translated from scipy.special.huber into pure Rust.
§Notes
huber is useful as a loss function in robust statistics or machine learning to reduce the
influence of outliers as compared to the common squared error loss, residuals with a magnitude
higher than $\delta$ are not squared 1.
Typically, $r$ represents residuals, the difference between a model prediction and data.
Then, for $|r| \leq \delta$, huber resembles the squared error and for $|r| > \delta$ the
absolute error. This way, the Huber loss often achieves a fast convergence in model fitting for
small residuals like the squared error loss function and still reduces the influence of
outliers ($|r| > \delta$) like the absolute error loss. As $\delta$ is the cutoff between
squared and absolute error regimes, it has to be tuned carefully for each problem. huber is
also convex, making it suitable for gradient based optimization.
§See also
Peter Huber, “Robust Estimation of a Location Parameter”. 1964. Annals of Statistics. 53 (1): 73 - 101 ↩