feat: Add Ollama integration for AI-based speaker detection

- Added Requirement 14 for intelligent speaker detection using Ollama/llama3:8B - Updated design document with OllamaService component and integration details - Added 4 new correctness properties for Ollama functionality - Added tasks 15-17 for implementing Ollama integration - Updated technology stack to include httpx and Ollama - Added error handling guidelines for Ollama integration - Configured fallback to pattern-based detection when Ollama unavailable This enhancement allows users to paste raw dialogue text without manual role markers. The system will use AI to intelligently detect speakers, making the tool more user-friendly.
2025-12-09 10:31:38 +00:00
parent ff2c8f8404
commit 20e6cd1f9d
3 changed files with 157 additions and 4 deletions
--- a/.kiro/specs/text-to-audiobook/design.md
+++ b/.kiro/specs/text-to-audiobook/design.md
@@ -75,6 +75,8 @@ The application workflow consists of three main phases:
 - kokoro library (hexgrad/Kokoro-82M)
 - soundfile for audio processing
 - espeak-ng for grapheme-to-phoneme conversion
+- httpx for Ollama API communication
+- Ollama with llama3:8B for AI-based speaker detection

 **Infrastructure:**
 - Docker for containerization
@@ -224,6 +226,7 @@ class TTSRequest(BaseModel):
 def detect_roles(self, text: str) -> List[RoleInfo]:
    """
    Analyzes text to identify distinct speaker roles.
+    First attempts AI-based detection using Ollama, then falls back to pattern matching.
    Supports patterns:
    - "RoleName:" at line start
    - "[RoleName]" preceding dialogue
@@ -231,6 +234,13 @@ def detect_roles(self, text: str) -> List[RoleInfo]:
    """
    pass

+def _detect_with_ollama(self, text: str) -> Set[str]:
+    """
+    Uses Ollama with llama3:8B to intelligently detect speakers.
+    Returns set of speaker names or empty set if Ollama unavailable.
+    """
+    pass
+
 def _extract_colon_pattern(self, text: str) -> Set[str]:
    """Extracts roles from "Name:" pattern"""
    pass
@@ -240,7 +250,52 @@ def _extract_bracket_pattern(self, text: str) -> Set[str]:
    pass
 ```

-#### 3. TTS Service (`services/tts_service.py`)
+#### 3. Ollama Integration Service (`services/ollama_service.py`)
+
+**Class: OllamaService**
+
+**Attributes:**
+- `base_url: str` - Ollama API base URL (from environment)
+- `model: str = "llama3:8b"` - Model to use for speaker detection
+- `timeout: int = 30` - Request timeout in seconds
+
+**Methods:**
+```python
+def __init__(self, base_url: str = "http://localhost:11434"):
+    """Initializes Ollama service with base URL"""
+    pass
+
+async def detect_speakers(self, text: str) -> List[str]:
+    """
+    Sends text to Ollama for speaker detection.
+    Returns list of detected speaker names.
+    Raises exception if Ollama is unavailable or request fails.
+    """
+    pass
+
+def _build_prompt(self, text: str) -> str:
+    """
+    Constructs prompt for speaker detection.
+    Instructs model to return JSON array of speaker names.
+    """
+    pass
+
+def _parse_response(self, response: str) -> List[str]:
+    """
+    Parses Ollama response to extract speaker names.
+    Handles both JSON and plain text responses.
+    """
+    pass
+
+async def is_available(self) -> bool:
+    """
+    Checks if Ollama service is available.
+    Returns True if service responds to health check.
+    """
+    pass
+```
+
+#### 4. TTS Service (`services/tts_service.py`)

 **Class: TTSService**

@@ -507,6 +562,22 @@ ROLE_PATTERNS = {
 *For any* synthesized audio, the output should have a sample rate of 24 kHz.
 **Validates: Requirements 10.4**

+### Property 28: Ollama speaker detection
+*For any* text without explicit role markers, when Ollama is available, the backend should attempt AI-based speaker detection before falling back to pattern matching.
+**Validates: Requirements 14.1**
+
+### Property 29: Ollama JSON response parsing
+*For any* valid JSON response from Ollama containing speaker names, the backend should successfully parse and extract the speaker list.
+**Validates: Requirements 14.3**
+
+### Property 30: Ollama fallback behavior
+*For any* text, when Ollama is unavailable or fails, the backend should fall back to pattern-based role detection without error.
+**Validates: Requirements 14.4**
+
+### Property 31: Role deduplication across detection methods
+*For any* text where both AI detection and pattern detection find roles, the backend should merge results and return each unique role exactly once.
+**Validates: Requirements 14.5**
+
 ## Error Handling

 ### Frontend Error Handling
@@ -544,6 +615,13 @@ ROLE_PATTERNS = {
 - Return 500 Internal Server Error with error details
 - Log errors for debugging

+**Ollama Integration Errors:**
+- Handle Ollama service unavailable gracefully
+- Fall back to pattern-based detection on Ollama failure
+- Log Ollama errors for debugging without failing request
+- Handle malformed JSON responses from Ollama
+- Implement timeout for Ollama requests (30 seconds)
+
 **Resource Management:**
 - Handle out-of-memory scenarios for very long text
 - Implement timeouts for long-running synthesis
--- a/.kiro/specs/text-to-audiobook/requirements.md
+++ b/.kiro/specs/text-to-audiobook/requirements.md
@@ -179,3 +179,16 @@ The Text to Audiobook system is a production-ready React TypeScript single-page
 3. THE README SHALL document how to run the frontend with example commands
 4. THE README SHALL include steps to test the full workflow from text input to audio generation
 5. THE README MAY include screenshots or diagrams illustrating the application flow
+
+### Requirement 14
+
+**User Story:** As a user, I want the system to intelligently detect speakers in unmarked text using AI, so that I can convert any dialogue or narrative without manually adding role markers.
+
+#### Acceptance Criteria
+
+1. WHEN the Backend receives text without explicit role markers THEN the Backend SHALL use Ollama with llama3:8B to analyze the text and identify distinct speakers
+2. WHEN Ollama analyzes text THEN the Backend SHALL send a structured prompt requesting speaker identification in JSON format
+3. WHEN Ollama returns speaker names THEN the Backend SHALL parse the JSON response and create role objects
+4. WHEN Ollama is unavailable or fails THEN the Backend SHALL fall back to pattern-based role detection
+5. WHEN both AI detection and pattern detection find roles THEN the Backend SHALL merge and deduplicate the results
+6. THE Backend SHALL configure Ollama connection using environment variable OLLAMA_BASE_URL with default http://localhost:11434
--- a/.kiro/specs/text-to-audiobook/tasks.md
+++ b/.kiro/specs/text-to-audiobook/tasks.md
@@ -289,13 +289,75 @@
  - Include troubleshooting section
  - _Requirements: 13.1, 13.2, 13.3, 13.4, 13.5_

- [ ] 15. Final integration testing and polish
+- [ ] 15. Implement Ollama integration service
+  - Create OllamaService class in services/ollama_service.py
+  - Implement connection to Ollama API using httpx
+  - Configure OLLAMA_BASE_URL environment variable (default: http://localhost:11434)
+  - Implement detect_speakers() method with structured prompt
+  - Implement prompt builder for speaker detection
+  - Implement JSON response parser
+  - Implement is_available() health check method
+  - Add timeout handling (30 seconds)
+  - _Requirements: 14.1, 14.2, 14.3, 14.6_
+
+- [ ]* 15.1 Write property test for Ollama speaker detection
+  - **Property 28: Ollama speaker detection**
+  - **Validates: Requirements 14.1**
+
+- [ ]* 15.2 Write property test for JSON response parsing
+  - **Property 29: Ollama JSON response parsing**
+  - **Validates: Requirements 14.3**
+
+- [ ]* 15.3 Write unit tests for Ollama service
+  - Test connection to Ollama
+  - Test prompt construction
+  - Test JSON parsing with various formats
+  - Test timeout handling
+  - Test error handling for unavailable service
+  - _Requirements: 14.1, 14.2, 14.3_
+
+- [ ] 16. Integrate Ollama into role detection
+  - Update RoleDetector to use OllamaService
+  - Implement _detect_with_ollama() method
+  - Add fallback logic: try Ollama first, then pattern matching
+  - Implement role merging and deduplication across both methods
+  - Handle Ollama unavailable gracefully
+  - Log detection method used (AI vs pattern)
+  - _Requirements: 14.1, 14.4, 14.5_
+
+- [ ]* 16.1 Write property test for Ollama fallback
+  - **Property 30: Ollama fallback behavior**
+  - **Validates: Requirements 14.4**
+
+- [ ]* 16.2 Write property test for role deduplication
+  - **Property 31: Role deduplication across detection methods**
+  - **Validates: Requirements 14.5**
+
+- [ ]* 16.3 Write unit tests for integrated role detection
+  - Test with Ollama available
+  - Test with Ollama unavailable
+  - Test role merging from both sources
+  - Test deduplication logic
+  - _Requirements: 14.4, 14.5_
+
+- [ ] 17. Update documentation for Ollama integration
+  - Document Ollama setup in README
+  - Add instructions for installing and running Ollama
+  - Document llama3:8B model requirement
+  - Add OLLAMA_BASE_URL environment variable to .env.example
+  - Update backend README with Ollama configuration
+  - Add troubleshooting section for Ollama issues
+  - _Requirements: 14.6_
+
+- [ ] 18. Final integration testing and polish
  - Test complete end-to-end workflow
  - Test with various text samples (short, long, multiple roles, no roles)
+  - Test with unmarked dialogue text (relying on AI detection)
+  - Test with marked text (pattern detection)
+  - Test with Ollama available and unavailable
  - Test all language options (a, b, f, j)
  - Test error scenarios (network failures, invalid inputs)
  - Verify audio quality and playback
  - Test Docker deployment
  - Fix any bugs discovered during testing
-  - _Requirements: All_
-,.
+  - _Requirements: All_