RESEARCH

SPEECH & AUDIO

Fitting New Speakers Based on a Short Untranscribed Sample

July 11, 2018

Abstract

Learning-based Text To Speech systems have the potential to generalize from one speaker to the next and thus require a relatively short sample of any new voice. However, this promise is currently largely unrealized. We present a method that is designed to capture a new speaker from a short untranscribed audio sample. This is done by employing an additional network that given an audio sample, places the speaker in the embedding space. This network is trained as part of the speech synthesis system using various consistency losses. Our results demonstrate a greatly improved performance on both the dataset speakers, and, more importantly, when fitting new voices, even from very short samples.

Download the Paper

AUTHORS

Written by

Eliya Nachmani

Adam Polyak

Lior Wolf

Yaniv Taigman

Publisher

ICML

Research Topics

Speech & Audio

Related Publications

November 13, 2026

RESEARCH

NLP

Unbundling Multimodal Vision Encoding

Ellis Brown, Chris Hoang, Shang-Wen Li, Mike Lewis, Marjan Ghazvininejad, Luke Zettlemoyer, Lucas Beyer, Mengye Ren, Jakob Verbeek, Saining Xie, Hu Xu

November 13, 2026

November 04, 2026

SPEECH & AUDIO

GSLM2: Multi-Stream Speech Representation Learning and Generative Spoken Language Modeling

Yosuke Higuchi, Tom Fizycki, Mahi Luthra, Charles-Éric Saint-James, Azeez Syed, Angel Villar Corrales, Mido Assran, Nicolas Ballas, Jiayi Shen, Dongyan Lin, Phillip Rust, Surya Parimi, Vanessa Stark, Alexandre Allauzen, Balázs Kegl, Juan Pino, Emmanuel Dupoux

November 04, 2026

October 19, 2026

RESEARCH

NLP

Muse Glimmer 30B: An On-Device Agentic Model

Muse Glimmer team

October 19, 2026

October 14, 2026

RESEARCH

Human-Guided Agentic AI for End-to-End EHR Research: Sequential Antibody-Drug Conjugate Pathways in Breast Cancer

Arturo Loaiza-Bonilla

October 14, 2026

Help Us Pioneer The Future of AI

We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.