Model Criticism in Latent Space

Abstract

It is commonly accepted that ‘all models are wrong but some are useful’. An aspect of statistical modelling is, therefore, to understand the limitations of the fitted model since this may help in extending the model to a more suitable one. This process is known as model criticism. Model criticism uses statistical tests to assess various aspects of the fitted model in order to identify its deficiencies. This is usually carried out, by Posterior Predictive Check, in the observation space by assessing if replicated data generated under the fitted model looks similar to the observed data. I will describe an alternative approach, referred to as the Aggregated Posterior Check, that pulls the data back into the space of latent variables, and carries out model criticism in the latent space. The principle of this approach is that if the model fits, then posterior inferences should match the prior assumptions. I will demonstrate the method with examples of model criticism in latent space applied to factor analysis, linear dynamical systems and Gaussian processes on three real world examples from image analysis, time series modelling, and time series extrapolation.