[SAIF 2020] Day 1: Towards End-to-End Speech Recognition – Tara Sainath | Samsung

Posted by

End-to-end models allow us to represent the entire speech recognition pipeline (i.e., conventional acoustic, pronunciation and language models) by one neural network. This has huge implications for privacy, reliability and latency. In this talk, we will discuss the research developments around E2E models that have allowed it to reach the accuracy of conventional models, at a fraction of the model size and reduced latency. In addition, we will present further benefits of these models with respect to multi-lingual speech recognition.... Read More إقرأ المزيد | Share it now!

[SAIF 2020] Day 1: From Few-Shot Adaptation to Uncovering Symmetries – Chelsea Finn | Samsung

Posted by

Machine learning systems are often designed under the assumption that they will be deployed as a static model in a single static region of the world. However, the world is constantly changing, such that the future no longer looks exactly like the past, and even in relatively static settings, the system may be deployed in new, unseen parts of its world. While such continuous shifts in the data distribution can place major challenges on models acquired in machine learning, the model need not be static either: it can and should adapt. In this talk, I’ll discuss how we can allow deep networks to be robust to such distribution shift via adaptation. I will focus on meta-learning algorithms that enable this adaptation to be fast, first introducing the concept of meta-learning, then briefly overviewing several successful applications of meta-learning ranging from robotics to drug design, and finally discussing several recent works at the frontier of meta-learning research.... Read More إقرأ المزيد | Share it now!

[SAIF 2020] Day 1: Towards Discovering Casual Representations – Yoshua Bengio | Samsung

Posted by

Up to now deep learning has focused on learning representations which are useful in many applications but differ from the kind of high-level representations humans can communicate with natural language and capture semantic (verbalizable) variables and their causal dependencies. Capturing causal structure is important for many reasons: (a) it allows an agent to take appropriate decisions (interventions) by having a good causal model of its effects, (b) it can lead to robustness with respect to changes in distribution (a major current limitation of state-of-the-art machine learning), and (c) it makes it easier to understand natural language (which refers to such causal concepts, the semantic variables named with words) and thus interact more meaningfully with humans. Whereas causality research has focused on inference (like how strong is the causal effect of A on B?) and to a lesser extent on causal discovery (is A a direct cause of B?), an important open question to which deep learning researchers can contribute is that of discovering causal representations, i.e., transformations from low-level sensory data to high-level representations of causal variables, where the high-level variables are not always labeled by humans. This must necessarily be done at the same time as one learns the structure of the causal graph which links these variables since both are generally unknown. This talk will report on early efforts towards these objectives, as part of a larger research programme aimed at expanding deep learning from system 1 (unconscious) processing to system 2 (conscious-level) processing of semantic variables.... Read More إقرأ المزيد | Share it now!