Linguistic Structure from Deep Learning

Abstract

Modern deep neural networks achieve impressive performance in tasks demanding extensive linguistic skills, such as machine translation. This has revived interest in probing the extent to which these models are inducing genuine grammatical knowledge from the raw data they are exposed to, and on whether they can thus shed new light on long-standing issues about the fundamental priors necessary for language acquisition. In this article, we survey representative studies investigating the syntactic abilities of deep networks, and we discuss the broader implications that similar work has for theoretical linguistics.