Laurence Aitchison

A unified statistical theory of semi-supervised learning, out-of-distribution detection and the cold posterior effect.

Benchmark image classification datasets such as CIFAR-10 and ImageNet are carefully curated to exclude ambiguous or difficult to classify images. We develop a generative model of dataset curation in which multiple annotators label every image, with the image being included in the dataset only if all the annotators agree. If any of the annotators disagree, the image is excluded. Remarkably, this simple generative model unifies three apparently unrelated areas of machine learning:
Semi-supervised learning, where we use unlabelled points to improve the performance of a classifier.
Out-of-distribution detection, where we explicitly detect test points that are far from the training data, as our predictions might be inaccurate in those regions.
The "cold posterior effect", where artificially reducing uncertainty in the Bayesian posterior over neural network weights gives better test performance.