CodeGPT is a popular coding assistant that is available as an extension to Visual Studio Code or integrated development environments (IDEs) from JetBrains. It integrates large language models like Llama to make developers and CTOs more productive in many ways—not just generating code but answering their questions about their codebase, helping them debug code, and onboarding new developers to existing projects.

Since its launch in March 2023, CodeGPT has amassed over 1.4 million downloads. It has users in more than 180 countries and has been adding hundreds of thousands more every month.

A few months after CodeGPT launched, Meta released Code Llama, an LLM based on Llama 2 and designed to generate code in response to text prompts. That got the attention of the CodeGPT team right away.

“We were impressed by Llama’s performance and flexibility,” says CodeGPT CTO & Co-Founder Daniel Avila. That led the team to experiment with chat-based and fill-in-the-middle models for interacting with code repositories. Their experiments were so successful that they decided to integrate Llama into the CodeGPT platform, using it to provide AI-powered assistance for its customers. The company has since built on that foundation, adding AI agents that are experts in APIs and frameworks and upgrading their LLM to Llama 3.2 (90B).

The impact of CodeGPT’s work with Llama is significant. Developers using CodeGPT see their productivity improve by at least 30% because it reduces the time they spend debugging, searching for solutions, and generating code. The company’s customers can also onboard new developers much faster—in days instead of months.

CodeGPT has significantly expanded its use of Llama since first starting with code suggestions and autocompletion. The platform today includes the ability to generate project folders and files autonomously. It also includes a codebase graph mechanism that lets Llama fully understand an entire repository’s structure. That enables developers to ask CodeGPT questions and, using Llama, to effectively “talk” with their repository. That makes it easier for anyone joining a project to understand what the code is about. It also simplifies debugging and looking up information for developers during code creation.

The implementation wasn’t without its challenges. The biggest was integrating Llama into complex workflows that required the LLM to understand large codebases. CodeGPT addressed this by creating the graph-based mechanism mentioned above, which enables Llama to understand the codebase more holistically. The team also optimized Llama to handle multi-step tasks, such as generating code and calling external tools through API calls and spent a lot of time fine-tuning the LLM for each use case.

Fine-tuning has been a matter of optimizing Llama models to handle specific programming tasks, such as code autocompletion, bug detection, and exploring a code repository. To do this, the team needed to train the models on a wide range of codebases, programming languages, and debugging scenarios. The team also incorporated external knowledge sources, such as technical documentation and conversations on popular coding forums.

“Llama has transformed the way developers interact with their codebases, making coding more intuitive and efficient,” says Avila “These models have enormous potential, not just in accelerating coding tasks, but in fundamentally reshaping software development workflows.”

Open source has been a crucial aspect of the CodeGPT development process, giving the CodeGPT team the benefit of being able to connect with a global developer community for their expertise and to solve problems—which enables faster iteration and more rapid development of new features into CodeGPT.

Additionally, says Avila, their customers appreciate the ability to use open source LLMs. “We have seen a huge demand for open source models from our users. Developers like to have an open source option for several reasons, including data privacy.”

For smaller companies like CodeGPT, open source models like Llama provide access to cutting-edge AI technologies, allowing them to innovate quickly without needing large-scale R&D budgets. An open source solution allows startups to build world-class projects.

As Avila points out, “CodeGPT is one of the top players in the AI for developers space, and Llama models played a big role in that.”

CodeGPT has big plans for its future. As its ecosystem of LLMs and developer tools evolves, the team plans to incorporate the latest Llama models into more advanced features, including real-time code collaboration and AI-powered refactoring tools. They’re also exploring ways to scale Llama across larger projects, further enhancing its repository comprehension and debugging capabilities.