### Noise contrastive estimation: asymptotics, comparison with MC-MLE

Statistics Seminar

23rd November 2018, 2:00 pm – 3:00 pm

Main Maths Building, SM2

A statistical model is said to be un-normalised when its likelihood

function involves an intractable normalising constant. Two popular

methods for parameter inference for these models are MC-MLE (Monte

Carlo maximum likelihood estimation), and NCE (noise contrastive

estimation); both methods rely on simulating artificial data-points to

approximate the normalising constant. While the asymptotics of MC-MLE

have been established under general hypotheses (Geyer, 1994), this is

not so for NCE. We establish consistency and asymptotic normality of

NCE estimators under mild assumptions. We compare NCE and MC-MLE under

several asymptotic regimes. In particular, we show that, when m goes

to infinity while n is fixed (m and n being respectively the number of

artificial data-points, and actual data-points), the two estimators

are asymptotically equivalent. Conversely, we prove that, when the

artificial data-points are IID, and when n goes to infinity while m/n

converges to a positive constant, the asymptotic variance of a NCE

estimator is always smaller than the asymptotic variance of the

corresponding MC-MLE estimator. We illustrate the variance reduction

brought by NCE through a numerical study. (based on joint work with Simon Barthelmé and Lionel Riou-Durand)

*Organiser*: Henry Reeve

## Comments are closed.