CONVERSATIONAL AI

NLP

Llama 2: Open Foundation and Fine-Tuned Chat Models

July 18, 2023

Abstract

In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases. Our models outperform open-source chat models on most benchmarks we tested, and based on our human evaluations for helpfulness and safety, may be a suitable substitute for closedsource models. We provide a detailed description of our approach to fine-tuning and safety improvements of Llama 2-Chat in order to enable the community to build on our work and contribute to the responsible development of LLMs.

Download the Paper

AUTHORS

Written by

Hugo Touvron

Louis Martin

Kevin Stone

Peter Albert

Amjad Almahairi

Yasmine Babaei

Nikolay Bashlykov

Soumya Batra

Prajjwal Bhargava

Shruti Bhosale

Dan Bikel

Lukas Blecher

Cristian Canton Ferrer

Moya Chen

Guillem Cucurull

David Esiobu

Jude Fernandes

Jeremy Fu

Wenyin Fu

Brian Fuller

Cynthia Gao

Vedanuj Goswami

Naman Goyal

Anthony Hartshorn

Saghar Hosseini

Rui Hou

Hakan Inan

Marcin Kardas

Viktor Kerkez

Madian Khabsa

Isabel Kloumann

Artem Korenev

Punit Singh Koura

Marie-Anne Lachaux

Thibaut Lavril

Jenya Lee

Diana Liskovich

Yinghai Lu

Yuning Mao

Xavier Martinet

Todor Mihaylov

Pushkar Mishra

Igor Molybog

Yixin Nie

Andrew Poulton

Jeremy Reizenstein

Rashi Rungta

Kalyan Saladi

Alan Schelten

Ruan Silva

Eric Michael Smith

Ranjan Subramanian

Xiaoqing Ellen Tan

Binh Tang

Ross Taylor

Adina Williams

Jian Xiang Kuan

Puxin Xu

Zheng Yan

Iliyan Zarov

Yuchen Zhang

Angela Fan

Melanie Kambadur

Sharan Narang

Aurelien Rodriguez

Robert Stojnic

Sergey Edunov

Thomas Scialom

Publisher

arxiv

Related Publications

August 24, 2023

NLP

CORE MACHINE LEARNING

Code Llama: Open Foundation Models for Code

Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Ellen Tan, Yossef (Yossi) Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Defossez, Jade Copet, Faisal Azhar, Hugo Touvron, Gabriel Synnaeve, Louis Martin, Nicolas Usunier, Thomas Scialom

August 24, 2023

August 22, 2023

SPEECH & AUDIO

NLP

SeamlessM4T—Massively Multilingual & Multimodal Machine Translation

Seamless Communication, Loic Barrault, Andy Chung, David Dale, Ning Dong (AI), Paul-Ambroise Duquenne, Hady Elsahar, Hongyu Gong, Kevin Heffernan, John Hoffman, Christopher Klaiber, Peng-Jen Chen, Daniel Licht, Jean Maillard, Alice Rakotoarison, Kaushik Ram Sadagopan, Guillaume Wenzek, Abinesh Ramakrishnan, Alexandre Mourachko, Amanda Kallet, Ann Lee, Anna Sun, Bapi Akula, Benjamin Peloquin, Bernie Huang, Bokai Yu, Brian Ellis, Can Balioglu, Carleigh Wood, Changhan Wang, Christophe Ropers, Cynthia Gao, Daniel Li (FAIR), Elahe Kalbassi, Ethan Ye, Gabriel Mejia Gonzalez, Hirofumi Inaguma, Holger Schwenk, Igor Tufanov, Ilia Kulikov, Janice Lam, Jeff Wang (PM - AI), Juan Pino, Justin Haaheim, Justine Kao, Prangthip Hasanti, Kevin Tran, Maha Elbayad, Marta R. Costa-jussa, Mohamed Ramadan, Naji El Hachem, Onur Çelebi, Paco Guzmán, Paden Tomasello, Pengwei Li, Pierre Andrews, Ruslan Mavlyutov, Russ Howes, Safiyyah Saleem, Skyler Wang, Somya Jain, Sravya Popuri, Tuan Tran, Vish Vogeti, Xutai Ma, Yilin Yang

August 22, 2023

August 21, 2023

NLP

SONAR: Sentence-Level Multimodal and Language-Agnostic Representations

Paul-Ambroise Duquenne, Holger Schwenk, Benoit Sagot

August 21, 2023

August 19, 2023

NLP

EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis

Tu Anh Nguyen, Wei-Ning Hsu, Antony D'Avirro, Bowen Shi, Itai Gat, Maryam Fazel-Zarandi, Tal Remez, Jade Copet, Gabriel Synnaeve, Michael Hassid, Felix Kreuk, Yossef Mordechay Adi, Emmanuel Dupoux

August 19, 2023

Help Us Pioneer The Future of AI

We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.