Scribd, Inc.'s Everand reading service is home to a global library of millions of ebooks, audiobooks, and more. For the newest version of Everand’s AI-powered content discovery assistant—Ask AI—Scribd, Inc. envisioned a discovery experience that combined Everand’s complex catalog with a nuanced understanding of each customer.

The team used three Llama models to create the new Ask AI: Llama 3.1 8B, 70B, and 405B. In addition to engineering an intuitive, knowledgeable AI agent, the team worked to optimize performance and manage costs for every workload.

“The open source nature of Llama allowed us to stay at the forefront of innovation, adapting the model to improve content recommendations quickly,” says Prabdheep Cheema, Senior Machine Learning Engineer at Scribd, Inc. “For companies like ours, open source models provide flexibility and enable rapid experimentation to meet our user needs.”

Before Scribd, Inc. integrated AI, finding a title or topic on Everand relied primarily on a keyword search. With over 195 million pieces of content across the Scribd, Inc. brands, the service’s 200 million unique monthly visitors could browse recommendations, but suggestions were based on pre-generated topics that customers couldn’t change. That made it difficult to find specific content and discover new titles of interest.

With the new Ask AI, Everand customers can explore an immense range of topics and ask obscure questions like, “How do ancient martial arts techniques feature in modern romance stories?"

“Creating a magical content experience was the most important factor,” says Steve Neola, Senior Director of Product, Generative AI, at Scribd, Inc. Llama stood out because of its superior ability to understand a person’s intent and deliver accurate results quickly.

The new Ask AI takes discovery beyond specific title searches. The retrained Llama 3.1 8B model at the heart of the service has a nuanced understanding of customer intent, and the Everand library that enables it to generate intuitive recommendations based on plot types, settings, genres, and other books a user likes.

To develop the new version, the team used Llama 3.1 405B to create synthetic data for a training dataset that simulated a wide range of consumer behavior.

Parameter-efficient fine-tuning (PEFT) with QLoRA/LoRA and supervised fine-tuning enabled Scribd, Inc. to create a highly accurate and tailored version of Llama 3.1 8B. Because Llama is open source, the team was able to push beyond closed model limitations and achieve deeper customization. The retrained model accurately detected customer intent—including an understanding of uncommon questions—routing customers to the best service for their requests. Fine-tuning Llama 3.1 8B helped the team deliver better results with minimal latency for real-time components of the Ask AI feature, while managing the model’s footprint and computing demands.

As more books are published and become a part of the Everand library, Llama 3.1 70B works in the background to generate metadata for each piece of content to improve discovery and accuracy. Llama’s flexible deployment options also made it easier for the team to integrate the model into its Ask AI assistant workflow without any major infrastructure changes. Scribd, Inc. used Amazon Web Services (AWS) and Databricks batch inference to analyze massive amounts of data and support solution development. The application delivers the model’s structured output in JSON format to improve metadata extraction and ensure high-quality, real-time responses.

Over time, Scribd, Inc. says Ask AI will mature into a next-level discovery agent with the power to enhance customer retention, improve loyalty, and create higher lifetime value for readers. Looking ahead, Scribd, Inc. plans to integrate Llama into more areas of its user experience and tap into Llama Guard 3 for additional content trust and moderation support.