Memory, Modularity, and the Theory of Deep Learnability

A Google TechTalk, 2019/04/04, presented by Rina Panigrahy. ABSTRACT: Why does deep learning work well for some applications and not for others? Do we need major architectural changes in deep learning to solve complex problems like natural language understanding and logic? Does memory and modular organization play an important role, and if so, how do we store complex concepts in memory? We will try to get a conceptual understanding of these questions by studying learning problems arising from synthetic mathematical function classes such as the learnability of polynomials, shallow teacher networks, and possible cryptographic hardness of learning deeper teacher networks. Finally we will present nascent ideas about how we should model memory and evolve a modular view of deep learning for higher level cognitive functions. About the Speaker: Rina Panigrahy is a Research Scientist at Google specializing in applied and theoretical algorithms in areas such as deep learning, high dimensional search, hashing, sketching, streaming, prediction and graph analysis with engineering and research impact covering over 75 publications and 50 patents. His Masters thesis work at MIT was used in founding Akamai Technologies. He has held research and engineering positions at Microsoft(principal researcher) and Cisco Systems. He obtained his Ph.D. in Algorithms from Stanford, and did his undergrad from IIT Mumbai after securing the top rank at the IIT-JEE entrance examination all over India. He is a recipient of a Gold medal at the International Math Olympiad and a winner of several best paper awards.