How to Apply LLMs in Pharo?

LLMPharo

When I started my PhD in 2024, I came from a professional background where I was mostly applying LLMs inside software tools and building tools around them. At the beginning of my PhD, my supervisor told me that I could attend ESUG’24. There, I saw the Feenk presentation about AI. At the same time, I started looking at what was happening around LLMs in the Pharo ecosystem, and I noticed that there was not much work in this direction. Benoit was one of the few people I knew who was already experimenting with LLMs in Pharo. I became really interested in understanding how we could apply LLMs inside Pharo. That is why I launched Pharo-LLM. At the time of writing, the Pharo-LLM organization has 21 followers, and I can see more and more people becoming interested in the tools we have developed. But Pharo-LLM has also changed a lot since I started it.

What is Pharo-LLM?

At the beginning, my goal was quite simple: I wanted to have the tools necessary to integrate LLMs into Pharo. But I did not want Pharo-LLM to be just another wrapper around an external API. The idea gradually became into build a set of tools and libraries that allow us to create, run, and interact with LLMs directly inside Pharo. More importantly, I want these tools to feel native to Pharo. That means integrating LLMs with the things that make Pharo interesting: objects, inspectors, debuggers, and our development workflows.

Today, the Pharo-LLM ecosystem includes several projects:

The objective is not simply to connect Pharo to LLMs. The objective is to explore what an LLM ecosystem built specifically for Pharo could look like.

From using LLMs to controlling them

While developing Pharo-LLM, I started thinking about another problem. Today, when we use an LLM, we often depend on an external provider. We depend on their APIs, their prices, their rate limits, their infrastructure, and a network connection. When we are working with source code or private projects, we also have to think about privacy.

So my initial question, How can we apply LLMs in Pharo?, slowly became another question: How can we build a sovereign LLM ecosystem in Pharo?. For me, there are three important parts to this, i. Having the tools inside Pharo to create, run, and interact with LLMs, ii. Having the ability to train and fine-tune our own models, iii. Having the ability to deploy and maintain our own models. This is the direction I now want to explore with Pharo-LLM.

Training models for Pharo

One concrete example is Pharo-Copilot. Pharo-Copilot is a code completion engine that uses LLMs to predict the next Pharo code tokens. Instead of completing only one token or one word, the idea is to generate multi-token code completions. But if we want a model that is really useful for Pharo, we need it to understand Pharo. So we experimented with open-weight code LLMs. You can find the paper describing this work and our results here.

Running our own models

Training or fine-tuning a model is only one part of the problem. After that, we need to run it. Today, Pharo-LLM can interact with cloud APIs such as ChatGPT, Claude, and DeepSeek. We can also use local APIs provided by tools such as Ollama, LM Studio, or vLLM. But I wanted to explore what happens if we go one step further. What if we can run the model through infrastructure that belongs directly to the Pharo ecosystem? This is why I started Pharo-Infer. An inference engine loads a trained language model and prepares it for execution. It converts prompts into tokens, generates tokens as output, and manages things such as memory, context, sampling, and streaming during generation. With Pharo-Infer, the idea is to provide this inference layer as part of the Pharo ecosystem, using open-source components such as llama.cpp for low-level model execution.

Having our own models and inference infrastructure gives us several interesting properties. We are not constrained by artificial API limits, rate limits, or quotas imposed by somebody else’s service. Privacy is also improved because sensitive source code can stay on the developer’s machine or inside the organization’s infrastructure. It also gives us better reliability, since we do not have to depend on an Internet connection during development or during a demo. Finally, and perhaps most importantly, it gives us more control: we can adapt both the model and the infrastructure to specific domains and workflows and integrate them directly with Pharo.

This last point is particularly important to me. Pharo is an environment where we like the idea that everything is moldable. If our AI infrastructure lives completely outside Pharo, behind another application or service, then we lose part of that idea. Of course, having our own inference engine also comes at a cost. It means more maintenance work, performance optimization, hardware management, and model management. But in return, it gives us greater independence, better privacy, and the possibility of much deeper integration with Pharo.

What about teams?

Having our own models does not mean that every developer needs to run a model locally on their laptop. For a team, we can deploy a model once on a shared server and let the whole team use it. We can connect that model to the team’s repositories, documentation, and project knowledge using RAG. This allows us to keep the model up to date with the latest project changes without retraining it every time something changes. More importantly, the source code and sensitive data can remain inside the organization’s infrastructure. The team can then share the same LLM infrastructure without depending entirely on an external provider.

The Pharo-LLM vision

When I started Pharo-LLM, I mainly wanted to understand how we could apply LLMs inside Pharo. Today, I see the project differently. The goal is not to train a huge general-purpose model. The goal is not to replace every external LLM provider. And the goal is not simply to put a ChatGPT-like interface inside Pharo. The goal is to give the Pharo community choice and control. We should be able to use an external provider when it makes sense. We should be able to run a local model when privacy or independence matters. We should be able to fine-tune a model specifically for Pharo when we need specialization. And we should be able to deploy and maintain our own models when we want control over the complete infrastructure. There are still limitations. Training and fine-tuning require time, data preparation, and experimentation. Bigger models require more resources and are harder to maintain. Building this ecosystem also requires the community to share datasets, benchmarks, models, tools, and improvements. But our experiments already show something interesting:

A smaller model specialized for Pharo can perform much better than its general-purpose base version on Pharo-specific tasks.

This makes me think that there is a real opportunity here. For me

Pharo-LLM is not only a technical project. It is a strategic investment in independence.

Contributions are very welcome!