feat: Add Ollama integration for AI-based speaker detection
- Added Requirement 14 for intelligent speaker detection using Ollama/llama3:8B - Updated design document with OllamaService component and integration details - Added 4 new correctness properties for Ollama functionality - Added tasks 15-17 for implementing Ollama integration - Updated technology stack to include httpx and Ollama - Added error handling guidelines for Ollama integration - Configured fallback to pattern-based detection when Ollama unavailable This enhancement allows users to paste raw dialogue text without manual role markers. The system will use AI to intelligently detect speakers, making the tool more user-friendly.
This commit is contained in:
@@ -75,6 +75,8 @@ The application workflow consists of three main phases:
|
||||
- kokoro library (hexgrad/Kokoro-82M)
|
||||
- soundfile for audio processing
|
||||
- espeak-ng for grapheme-to-phoneme conversion
|
||||
- httpx for Ollama API communication
|
||||
- Ollama with llama3:8B for AI-based speaker detection
|
||||
|
||||
**Infrastructure:**
|
||||
- Docker for containerization
|
||||
@@ -224,6 +226,7 @@ class TTSRequest(BaseModel):
|
||||
def detect_roles(self, text: str) -> List[RoleInfo]:
|
||||
"""
|
||||
Analyzes text to identify distinct speaker roles.
|
||||
First attempts AI-based detection using Ollama, then falls back to pattern matching.
|
||||
Supports patterns:
|
||||
- "RoleName:" at line start
|
||||
- "[RoleName]" preceding dialogue
|
||||
@@ -231,6 +234,13 @@ def detect_roles(self, text: str) -> List[RoleInfo]:
|
||||
"""
|
||||
pass
|
||||
|
||||
def _detect_with_ollama(self, text: str) -> Set[str]:
|
||||
"""
|
||||
Uses Ollama with llama3:8B to intelligently detect speakers.
|
||||
Returns set of speaker names or empty set if Ollama unavailable.
|
||||
"""
|
||||
pass
|
||||
|
||||
def _extract_colon_pattern(self, text: str) -> Set[str]:
|
||||
"""Extracts roles from "Name:" pattern"""
|
||||
pass
|
||||
@@ -240,7 +250,52 @@ def _extract_bracket_pattern(self, text: str) -> Set[str]:
|
||||
pass
|
||||
```
|
||||
|
||||
#### 3. TTS Service (`services/tts_service.py`)
|
||||
#### 3. Ollama Integration Service (`services/ollama_service.py`)
|
||||
|
||||
**Class: OllamaService**
|
||||
|
||||
**Attributes:**
|
||||
- `base_url: str` - Ollama API base URL (from environment)
|
||||
- `model: str = "llama3:8b"` - Model to use for speaker detection
|
||||
- `timeout: int = 30` - Request timeout in seconds
|
||||
|
||||
**Methods:**
|
||||
```python
|
||||
def __init__(self, base_url: str = "http://localhost:11434"):
|
||||
"""Initializes Ollama service with base URL"""
|
||||
pass
|
||||
|
||||
async def detect_speakers(self, text: str) -> List[str]:
|
||||
"""
|
||||
Sends text to Ollama for speaker detection.
|
||||
Returns list of detected speaker names.
|
||||
Raises exception if Ollama is unavailable or request fails.
|
||||
"""
|
||||
pass
|
||||
|
||||
def _build_prompt(self, text: str) -> str:
|
||||
"""
|
||||
Constructs prompt for speaker detection.
|
||||
Instructs model to return JSON array of speaker names.
|
||||
"""
|
||||
pass
|
||||
|
||||
def _parse_response(self, response: str) -> List[str]:
|
||||
"""
|
||||
Parses Ollama response to extract speaker names.
|
||||
Handles both JSON and plain text responses.
|
||||
"""
|
||||
pass
|
||||
|
||||
async def is_available(self) -> bool:
|
||||
"""
|
||||
Checks if Ollama service is available.
|
||||
Returns True if service responds to health check.
|
||||
"""
|
||||
pass
|
||||
```
|
||||
|
||||
#### 4. TTS Service (`services/tts_service.py`)
|
||||
|
||||
**Class: TTSService**
|
||||
|
||||
@@ -507,6 +562,22 @@ ROLE_PATTERNS = {
|
||||
*For any* synthesized audio, the output should have a sample rate of 24 kHz.
|
||||
**Validates: Requirements 10.4**
|
||||
|
||||
### Property 28: Ollama speaker detection
|
||||
*For any* text without explicit role markers, when Ollama is available, the backend should attempt AI-based speaker detection before falling back to pattern matching.
|
||||
**Validates: Requirements 14.1**
|
||||
|
||||
### Property 29: Ollama JSON response parsing
|
||||
*For any* valid JSON response from Ollama containing speaker names, the backend should successfully parse and extract the speaker list.
|
||||
**Validates: Requirements 14.3**
|
||||
|
||||
### Property 30: Ollama fallback behavior
|
||||
*For any* text, when Ollama is unavailable or fails, the backend should fall back to pattern-based role detection without error.
|
||||
**Validates: Requirements 14.4**
|
||||
|
||||
### Property 31: Role deduplication across detection methods
|
||||
*For any* text where both AI detection and pattern detection find roles, the backend should merge results and return each unique role exactly once.
|
||||
**Validates: Requirements 14.5**
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Frontend Error Handling
|
||||
@@ -544,6 +615,13 @@ ROLE_PATTERNS = {
|
||||
- Return 500 Internal Server Error with error details
|
||||
- Log errors for debugging
|
||||
|
||||
**Ollama Integration Errors:**
|
||||
- Handle Ollama service unavailable gracefully
|
||||
- Fall back to pattern-based detection on Ollama failure
|
||||
- Log Ollama errors for debugging without failing request
|
||||
- Handle malformed JSON responses from Ollama
|
||||
- Implement timeout for Ollama requests (30 seconds)
|
||||
|
||||
**Resource Management:**
|
||||
- Handle out-of-memory scenarios for very long text
|
||||
- Implement timeouts for long-running synthesis
|
||||
|
||||
Reference in New Issue
Block a user