Skip to main content

huber

Function huber 

Source
pub fn huber(delta: f64, r: f64) -> f64
Expand description

Huber Loss function

$$ L_\delta(r) = \begin{cases} \infty & \delta < 0 \\ \frac{1}{2} r^2 & |r| \le \delta \\ \delta \left(|r| - \frac{1}{2} \delta \right) & \text{otherwise} \end{cases} $$

Translated from scipy.special.huber into pure Rust.

§Notes

huber is useful as a loss function in robust statistics or machine learning to reduce the influence of outliers as compared to the common squared error loss, residuals with a magnitude higher than $\delta$ are not squared 1.

Typically, $r$ represents residuals, the difference between a model prediction and data. Then, for $|r| \leq \delta$, huber resembles the squared error and for $|r| > \delta$ the absolute error. This way, the Huber loss often achieves a fast convergence in model fitting for small residuals like the squared error loss function and still reduces the influence of outliers ($|r| > \delta$) like the absolute error loss. As $\delta$ is the cutoff between squared and absolute error regimes, it has to be tuned carefully for each problem. huber is also convex, making it suitable for gradient based optimization.

§See also


  1. Peter Huber, “Robust Estimation of a Location Parameter”. 1964. Annals of Statistics. 53 (1): 73 - 101 ↩