One of the practical limitations of a locally hosted LLM is that it has no access to current information — its knowledge is frozen at training time. This article extends the setup from my previous article by adding real-time web search to the Pi Coding Agent via the Model Context Protocol (MCP). The search backend is SearXNG, a privacy-respecting open-source metasearch engine that runs locally in Docker.
Note: This guide is not limited to local LLMs — it works equally well with hosted models from providers such as OpenAI, Anthropic, and others.
This article documents my experiments with the Pi Coding Agent and locally hosted Large Language Models (LLMs) served via llama.cpp on a MacBook. The following steps describe the setup process from start to finish.
Overview
The diagram below illustrates how the individual components fit together.
┌─────────────────────┐
│ Pi Coding Agent │
└──────────┬──────────┘
│ OpenAI-compatible API
▼
┌─────────────────────┐
│ llama.cpp │
│ (llama-server) │
└──────────┬──────────┘
│ loads & runs
▼
┌─────────────────────┐
│ LLM Model │
│ (GGUF on disk) │
└─────────────────────┘
The Pi Coding Agent serves as the command-line interface for the developer. It communicates with llama-server, which exposes an OpenAI-compatible REST API and handles all model lifecycle management. The LLM itself is stored on disk as a GGUF file and is loaded into RAM by llama-server on demand. The setup follows a bottom-up approach: we first download the model files, then configure and start llama-server, and finally install and configure the Pi Coding Agent on top.
I’ve been meaning to tackle this task for a few years now. I wanted to refresh my social media accounts because I hadn’t been active there for ages and felt a bit outdated. As I was going through those social media platforms, I also realized I hadn’t touched my blog in a decade. Instead of just deleting it, I decided to give it another shot and started fresh.
I won’t be posting as often as I’d like, but I’m giving it another shot to see if I can keep up a regular schedule. If it doesn’t work out, I’ll have to take down the blog pages.
