Pier Luigi Dragotti

Model-Based Machine Learning for Inverse Problems in Imaging

Deep Neural Networks are currently able to achieve state-of-the-art performance in many imaging tasks. In this talk, we argue that in inverse imaging problems efficient deep neural networks with more predictable performances can only be achieved by combining model-based solvers with learned models. There is plenty of evidence for this and typical examples where this integration has had an impact include the plug-and-play framework and the network unfolding strategy.

In the first part of the talk, we propose to connect wavelet theory and in particular the lifting scheme to the design of invertible neural networks (INN). We focus on image denoising and present a training strategy and an architecture for the INNs that mimics the multi-resolution property of the wavelet transform as well as the ability of the transform to provide a sparse representation of images. We show that the proposed INN achieves state-of-the-art performance for image denoising and that generalizes better to unseen noise than existing methods.

In the second part of the talk, we develop the interplay between learning and computational imaging. We discuss the problem of monitoring the activity of neurons with two-photon microscopes and present a model-based neural network for the extraction of neural activity. The architecture of the network is model-based and is designed using the unfolding technique.

Finally, we focus on the heritage sector which is experiencing a digital revolution driven in part by the increasing use of non-invasive, non-destructive imaging techniques. These new imaging methods provide a way to capture information about an entire painting and can give us information about features at or below the surface of the painting. We focus on Macro X-Ray Fluorescence (XRF) scanning which is a technique for the mapping of chemical elements in paintings and introduce a method that can process XRF scanning data from paintings. The results presented show the ability of our method to detect and separate weak signals related to hidden chemical elements in the paintings. We analyse the results on Leonardo’s “The Virgin of the Rocks” and show that our algorithm is able to reveal, more clearly than ever before, the hidden drawings of a previous composition that Leonardo then abandoned.