pub fn kl_div(x: f64, y: f64) -> f64Expand description
Elementwise function for computing the Kullback-Leibler divergence
$$ \mathrm{kl\_div}(x_i,\, y_i) = \begin{cases} x_i \ln(x_i / y_i) - x_i + y_i & x_i > 0 \wedge y_i > 0 \\ y_i & x_i = 0 \wedge y_i \ge 0 \\ \infty & \text{otherwise} \end{cases} $$
Translated from scipy.special.kl_div into pure Rust.
§Notes
This function is non-negative and is jointly convex in $x$ and $y$.
The origin of this function is in convex programming 1. This is why the function contains
the extra $-x_i + y_i$ terms over what might be expected from the Kullback-Leibler divergence.
For a version of the function without the extra terms, see rel_entr.
§See also
S. Boyd and L. Vandenberghe. Convex optimization. Cambridge University Press, 2004. https://doi.org/10.1017/CBO9780511804441 ↩