On Synergy between deep learning and Information theory
Abstract: Information theory was invented by C. Shannon in 1948. While its original focus was mainly on characterizing the fundamental limits of signal processing and communication problems, gradually, its scope has moved beyond those areas. On the other hand, deep learning is a very active area of research that is about training deep neural networks for solving machine learning tasks. In this talk, I will start by describing main challenges in the areas of information theory and deep learning. Then, I will show how each area could enable the other one address some of its key challenges and move beyond its status quo. Specifically, in the first half of the talk, I will describe an information theoretic approach to deep learning. I will show how such as approach enables us to shed light on the role of the number of levels and layers on the approximation power of neural networks. In the second half of the talk, I will discuss using deep leaning tools to solve information theory problems. Specifically, I will describe how auto-encoders constructed based on trained neural networks can be used to solve inverse problems. The goal of this approach is to go beyond standard (relatively simple) source models currently used for solving such problems and hence improve their performance.
Bio: Shirin Jalali is a research scientist in the Mathematics and Algorithms Group at Nokia Bell Labs in Murray Hills, NJ. Prior to that she held positions as a research scholar at the department of Electrical Engineering at Princeton University and as a faculty fellow at NYU Tandon School of Engineering. She received her B.Sc. and M.Sc. degrees in Electrical Engineering from Sharif University of Technology. She obtained her M.Sc. in Statistics and Ph.D. in Electrical Engineering from Stanford University. Her research interests are in the areas of information theory, high-dimensional statistics and machine learning.