Fonctionne avec Indico
v3.2.9
Specific relative entropy between two probability measures plays a fundamental role in many tasks in information theory , statistics and physics. In many practical cases the measures are not known and it is therefore natural to seek a method to estimate the relative entropy given realisations of the two underlying measures. Because the specific relative entropy differs from the specific cross entropy by the specific entropy which can be universally estimated following the Lempel--Ziv parsing algorithm, we will focus on estimation of the specific cross entropy . We propose a slight modification of a well-known cross entropy estimator called the Ziv-Merhav estimator and prove that it is strongly consistent for a large class of decoupled measures.