How to prove the following known (Pinsker's) inequality?
For two strictly positive sequences $(p_i)^n_{i=l}$ and $(q_i)^n_{i=l}$ with $\sum_{i=1}^np_i=\sum_{i=1}^nq_i=1$ one has $$\sum_{i=1}^np_i\log\frac{p_i}{q_i}\ge \frac{1}{2}\left(\sum_{i=1}^n|p_i-q_i|\right)^2.$$
$\endgroup$ 52 Answers
$\begingroup$See here. Pinsker's inequality
$\endgroup$ 3 $\begingroup$Take a look at Tsybakov 2009 Introduction to Nonparametric Estimation, p.88, also available in the net. Good and simple proof.
$\endgroup$
