In models of high complexity, the computational burden involved in calculating the maximum likelihood estimator can be forbidding. Proper scoring rules (Brier 1950, Good 1952, Bregman 1967, de Finetti 1975) such as the logarithmic score, the Brier score, and others, induce natural unbiased estimating equations that generally lead to consistent estimation of unknown parameters. The logarithmic score corresponds to maximum likelihood estimation whereas a score function introduced by Hyvärinen (2005) leads to linear estimation equations for exponential families.

We shall briefly review the facts about proper scoring rules and their associated divergences, entropy measures, and estimating equations. We show how Hyvärinen’s score matching estimator (SME) leads to particularly simple estimating equations for Gaussian graphical models and discuss issues of existence of the SME.

The lecture is largely based on Forbes, P.G.M and Lauritzen, S. Linear Estimating Equations for Exponential Families with Application to Gaussian Linear Concentration Models. Linear Algebra and Its Applications (2015), pp. 261-283 DOI: 10.1016/j.laa.2014.08.015