CONVERSATIONAL AI

NLP

Llama 2: Open Foundation and Fine-Tuned Chat Models

July 18, 2023

Abstract

In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases. Our models outperform open-source chat models on most benchmarks we tested, and based on our human evaluations for helpfulness and safety, may be a suitable substitute for closedsource models. We provide a detailed description of our approach to fine-tuning and safety improvements of Llama 2-Chat in order to enable the community to build on our work and contribute to the responsible development of LLMs.

Download the Paper

AUTHORS

Written by

Hugo Touvron

Louis Martin

Kevin Stone

Peter Albert

Amjad Almahairi

Yasmine Babaei

Nikolay Bashlykov

Soumya Batra

Praj Bhargava

Shruti Bhosale

Dan Bikel

Lukas Blecher

Cristian Canton Ferrer

Moya Chen

Guillem Cucurull

David Esiobu

Jude Fernandes

Jeremy Fu

Wenyin Fu

Brian Fuller

Cynthia Gao

Vedanuj Goswami

Naman Goyal

Anthony Hartshorn

Saghar Hosseini

Rui Hou

Hakan Inan

Marcin Kardas

Viktor Kerkez

Madian Khabsa

Isabel Kloumann

Artem Korenev

Punit Singh Koura

Marie-Anne Lachaux

Thibaut Lavril

Jenya Lee

Diana Liskovich

Yinghai Lu

Yuning Mao

Xavier Martinet

Todor Mihaylov

Pushkar Mishra

Igor Molybog

Yixin Nie

Andrew Poulton

Jeremy Reizenstein

Rashi Rungta

Kalyan Saladi

Alan Schelten

Ruan Silva

Eric Michael Smith

Ranjan Subramanian

Xiaoqing Ellen Tan

Binh Tang

Ross Taylor

Adina Williams

Andrew Kuan

Puxin Xu

Zheng Yan

Iliyan Zarov

Yuchen Zhang

Angela Fan

Melanie Kambadur

Sharan Narang

Aurelien Rodriguez

Robert Stojnic

Sergey Edunov

Thomas Scialom

Publisher

arxiv

Research Topics

Conversational AI

Natural Language Processing (NLP)

Related Publications

September 05, 2024

CONVERSATIONAL AI

NLP

Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model

Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Luke Zettlemoyer, Omer Levy, Xuezhe Ma

September 05, 2024

Read the Paper

August 20, 2024

CONVERSATIONAL AI

NLP

Lumos : Empowering Multimodal LLMs with Scene Text Recognition

Ashish Shenoy, Yichao Lu, Srihari Jayakumar, Debojeet Chatterjee, Mohsen Moslehpour, Pierce Chuang, Abhay Harpale, Vikas Bhardwaj, Di Xu (SWE), Shicong Zhao, Ankit Ramchandani, Luna Dong, Anuj Kumar