Add Ollama support for local LLM models (Phase 2 complete)

Major Changes:
- Added Ollama as alternative LLM provider to OpenAI
- Implemented flexible provider switching via environment variables
- Support for multiple embedding models (OpenAI and Ollama)
- Created comprehensive Ollama setup guide

Configuration Changes (config.py):
- Added LLM_PROVIDER and EMBEDDER_PROVIDER settings
- Added Ollama configuration: base URL, LLM model, embedding model
- Modified get_mem0_config() to dynamically switch providers
- OpenAI API key now optional when using Ollama
- Added validation to ensure required keys based on provider

Supported Configurations:
1. Full OpenAI (default):
   - LLM_PROVIDER=openai
   - EMBEDDER_PROVIDER=openai

2. Full Ollama (local):
   - LLM_PROVIDER=ollama
   - EMBEDDER_PROVIDER=ollama

3. Hybrid configurations:
   - Ollama LLM + OpenAI embeddings
   - OpenAI LLM + Ollama embeddings

Ollama Models Supported:
- LLM: llama3.1:8b, llama3.1:70b, mistral:7b, codellama:7b, phi3:3.8b
- Embeddings: nomic-embed-text, mxbai-embed-large, all-minilm

Documentation:
- Created docs/setup/ollama.mdx - Complete Ollama setup guide
  - Installation methods (host and Docker)
  - Model selection and comparison
  - Docker Compose configuration
  - Performance tuning and GPU acceleration
  - Migration guide from OpenAI
  - Troubleshooting section
- Updated README.md with Ollama features
- Updated .env.example with provider selection
- Marked Phase 2 as complete in roadmap

Environment Variables:
- LLM_PROVIDER: Select LLM provider (openai/ollama)
- EMBEDDER_PROVIDER: Select embedding provider (openai/ollama)
- OLLAMA_BASE_URL: Ollama API endpoint (default: http://localhost:11434)
- OLLAMA_LLM_MODEL: Ollama model for text generation
- OLLAMA_EMBEDDING_MODEL: Ollama model for embeddings
- MEM0_EMBEDDING_DIMS: Must match embedding model dimensions

Breaking Changes:
- None - defaults to OpenAI for backward compatibility

Migration Notes:
- When switching from OpenAI to Ollama embeddings, existing embeddings
  must be cleared due to dimension changes (1536 → 768 for nomic-embed-text)
- Update MEM0_EMBEDDING_DIMS to match chosen embedding model

Benefits:
 Cost savings - no API costs with local models
 Privacy - all data stays local
 Offline capability - works without internet
 Model variety - access to many open-source models
 Flexibility - easy switching between providers

Version: 1.1.0
Status: Phase 2 Complete - Production Ready with Ollama Support

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Claude Code
2025-10-15 16:07:17 +02:00
parent 1998bef6f4
commit fa9d3d8a6b
4 changed files with 621 additions and 41 deletions

View File

@@ -12,8 +12,17 @@ from pydantic import Field
class Settings(BaseSettings):
"""Application settings loaded from environment variables"""
# LLM Provider Selection
llm_provider: str = Field(default="openai", env="LLM_PROVIDER") # openai or ollama
embedder_provider: str = Field(default="openai", env="EMBEDDER_PROVIDER") # openai or ollama
# OpenAI
openai_api_key: str = Field(..., env="OPENAI_API_KEY")
openai_api_key: str = Field(default="", env="OPENAI_API_KEY") # Optional if using Ollama
# Ollama
ollama_base_url: str = Field(default="http://localhost:11434", env="OLLAMA_BASE_URL")
ollama_llm_model: str = Field(default="llama3.1:8b", env="OLLAMA_LLM_MODEL")
ollama_embedding_model: str = Field(default="nomic-embed-text", env="OLLAMA_EMBEDDING_MODEL")
# Supabase
supabase_connection_string: str = Field(..., env="SUPABASE_CONNECTION_STRING")
@@ -60,7 +69,7 @@ def get_settings() -> Settings:
def get_mem0_config(settings: Settings) -> Dict[str, Any]:
"""
Generate Mem0 configuration from settings
Generate Mem0 configuration from settings with support for OpenAI and Ollama
Args:
settings: Application settings
@@ -68,6 +77,51 @@ def get_mem0_config(settings: Settings) -> Dict[str, Any]:
Returns:
Dict containing Mem0 configuration
"""
# LLM Configuration - Switch between OpenAI and Ollama
if settings.llm_provider.lower() == "ollama":
llm_config = {
"provider": "ollama",
"config": {
"model": settings.ollama_llm_model,
"temperature": 0.1,
"max_tokens": 2000,
"ollama_base_url": settings.ollama_base_url
}
}
else: # Default to OpenAI
if not settings.openai_api_key:
raise ValueError("OPENAI_API_KEY is required when LLM_PROVIDER=openai")
llm_config = {
"provider": "openai",
"config": {
"model": "gpt-4o-mini",
"temperature": 0.1,
"max_tokens": 2000,
"api_key": settings.openai_api_key
}
}
# Embedder Configuration - Switch between OpenAI and Ollama
if settings.embedder_provider.lower() == "ollama":
embedder_config = {
"provider": "ollama",
"config": {
"model": settings.ollama_embedding_model,
"ollama_base_url": settings.ollama_base_url
}
}
else: # Default to OpenAI
if not settings.openai_api_key:
raise ValueError("OPENAI_API_KEY is required when EMBEDDER_PROVIDER=openai")
embedder_config = {
"provider": "openai",
"config": {
"model": "text-embedding-3-small",
"embedding_dims": settings.mem0_embedding_dims,
"api_key": settings.openai_api_key
}
}
return {
# Vector Store - Supabase
"vector_store": {
@@ -91,26 +145,11 @@ def get_mem0_config(settings: Settings) -> Dict[str, Any]:
}
},
# LLM Provider - OpenAI
"llm": {
"provider": "openai",
"config": {
"model": "gpt-4o-mini",
"temperature": 0.1,
"max_tokens": 2000,
"api_key": settings.openai_api_key
}
},
# LLM Provider - Dynamic (OpenAI or Ollama)
"llm": llm_config,
# Embedder - OpenAI
"embedder": {
"provider": "openai",
"config": {
"model": "text-embedding-3-small",
"embedding_dims": settings.mem0_embedding_dims,
"api_key": settings.openai_api_key
}
},
# Embedder - Dynamic (OpenAI or Ollama)
"embedder": embedder_config,
# Version
"version": settings.mem0_version