mars.tensor.special.
rel_entr
Elementwise function for computing relative entropy.
x (array_like) – Input arrays
y (array_like) – Input arrays
out (ndarray, optional) – Optional output array for the function results
Relative entropy of the inputs
scalar or ndarray
See also
entr, kl_div
entr
kl_div
Notes
This function is jointly convex in x and y.
The origin of this function is in convex programming; see 1. Given two discrete probability distributions \(p_1, \ldots, p_n\) and \(q_1, \ldots, q_n\), to get the relative entropy of statistics compute the sum
See 2 for details.
References
Grant, Boyd, and Ye, “CVX: Matlab Software for Disciplined Convex Programming”, http://cvxr.com/cvx/
Kullback-Leibler divergence, https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence