Nicolas Chopin


Noise contrastive estimation: asymptotics, comparison with MC-MLE

Statistics Seminar

23rd November 2018, 2:00 pm – 3:00 pm
Main Maths Building, SM2

A statistical model is said to be un-normalised when its likelihood
function involves an intractable normalising constant. Two popular
methods for parameter inference for these models are MC-MLE (Monte
Carlo maximum likelihood estimation), and NCE (noise contrastive
estimation); both methods rely on simulating artificial data-points to
approximate the normalising constant. While the asymptotics of MC-MLE
have been established under general hypotheses (Geyer, 1994), this is
not so for NCE. We establish consistency and asymptotic normality of
NCE estimators under mild assumptions. We compare NCE and MC-MLE under
several asymptotic regimes. In particular, we show that, when m goes
to infinity while n is fixed (m and n being respectively the number of
artificial data-points, and actual data-points), the two estimators
are asymptotically equivalent. Conversely, we prove that, when the
artificial data-points are IID, and when n goes to infinity while m/n
converges to a positive constant, the asymptotic variance of a NCE
estimator is always smaller than the asymptotic variance of the
corresponding MC-MLE estimator. We illustrate the variance reduction
brought by NCE through a numerical study. (based on joint work with Simon Barthelmé and Lionel Riou-Durand)

Organiser: Mathieu Gerber

Comments are closed.