NLP

LLaMA: Open and Efficient Foundation Language Models

February 24, 2023

Abstract

We introduce LLaMA, a collection of founda- tion language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla- 70B and PaLM-540B. We release all our models to the research community.

Download the Paper

AUTHORS

Written by

Faisal Azhar

Hugo Touvron

Armand Joulin

Aurelien Rodriguez

Baptiste Rozière

Eric Hambro

Gautier Izacard

Guillaume Lample

Marie-Anne Lachaux

Naman Goyal

Thibaut Lavril

Timothee Lacroix

Xavier Martinet

Edouard Grave

Publisher

ArXiV

Research Topics

Natural Language Processing (NLP)

Related Publications

November 20, 2024

NLP

CORE MACHINE LEARNING

Llama Guard 3-1B-INT4: Compact and Efficient Safeguard for Human-AI Conversations

Igor Fedorov, Kate Plawiak, Lemeng Wu, Tarek Elgamal, Naveen Suda, Eric Smith, Hongyuan Zhan, Jianfeng Chi, Yuriy Hulovatyy, Kimish Patel, Zechun Liu, Yangyang Shi, Tijmen Blankevoort, Mahesh Pasupuleti, Bilge Soran, Zacharie Delpierre Coudert, Rachad Alao, Raghuraman Krishnamoorthi, Vikas Chandra