Skip to main content

kl_div

Function kl_div 

Source
pub fn kl_div(x: f64, y: f64) -> f64
Expand description

Elementwise function for computing the Kullback-Leibler divergence

$$ \mathrm{kl\_div}(x_i,\, y_i) = \begin{cases} x_i \ln(x_i / y_i) - x_i + y_i & x_i > 0 \wedge y_i > 0 \\ y_i & x_i = 0 \wedge y_i \ge 0 \\ \infty & \text{otherwise} \end{cases} $$

Translated from scipy.special.kl_div into pure Rust.

§Notes

This function is non-negative and is jointly convex in $x$ and $y$.

The origin of this function is in convex programming 1. This is why the function contains the extra $-x_i + y_i$ terms over what might be expected from the Kullback-Leibler divergence. For a version of the function without the extra terms, see rel_entr.

§See also


  1. S. Boyd and L. Vandenberghe. Convex optimization. Cambridge University Press, 2004. https://doi.org/10.1017/CBO9780511804441 ↩