Skip to main content

​
Overview

OLLamaLLMService provides access to locally-run Ollama models through an OpenAI-compatible interface. It inherits from BaseOpenAILLMService and allows you to run various open-source models locally while maintaining compatibility with OpenAI’s API format for privacy and cost control.

Ollama LLM API Reference

Pipecat’s API methods for Ollama integration

Example Implementation

Browse examples using Ollama models

Ollama Documentation

Official Ollama documentation and model library

Download Ollama

Download and setup instructions for Ollama

​
Installation

To use Ollama services, you need to install both Ollama and the Pipecat dependency:
  1. Install Ollama on your system from ollama.com/download
  2. Install Pipecat dependency:
pip install "pipecat-ai[ollama]"
  1. Pull a model (first time only):
ollama pull llama2

​
Prerequisites

​
Ollama Local Setup

Before using Ollama LLM services, you need:
  1. Ollama Installation: Download and install Ollama from ollama.com
  2. Model Selection: Pull your desired models (llama2, mistral, codellama, etc.)
  3. Local Service: Ensure Ollama service is running (default port 11434)
  4. Hardware: Sufficient RAM and storage for your chosen models

​
Configuration

  • No API Keys Required: Ollama runs entirely locally
  • Model Management: Use ollama pull <model> to download models
  • Service URL: Default is http://localhost:11434 (configurable)
Ollama runs as a local service on port 11434. No API key required for complete privacy!

​
Configuration

​
model
str
default:"llama2"
The Ollama model to use. Must be pulled locally first with ollama pull.
​
base_url
str
default:"http://localhost:11434/v1"
Base URL for the Ollama API endpoint.

​
InputParams

This service uses the same input parameters as OpenAILLMService. See OpenAI LLM for details.

​
Usage

​
Basic Setup

from pipecat.services.ollama import OLLamaLLMService

llm = OLLamaLLMService(
    model="llama2",
)

​
With Custom Model and URL

from pipecat.services.ollama import OLLamaLLMService

llm = OLLamaLLMService(
    model="mistral",
    base_url="http://localhost:11434/v1",
)

​
Notes

  • No API key is required. The service automatically uses a placeholder key ("ollama") for OpenAI client compatibility.
  • The Ollama service must be running locally before starting your pipeline. Start it with ollama serve if it is not already running.
  • Model capabilities (function calling, vision, etc.) depend on the specific model you pull. Check the Ollama model library for details.