Task 1: Set up project structure and dependencies
- Created backend directory with FastAPI project structure - Created frontend directory with React + TypeScript + Vite - Configured Python virtual environment and installed dependencies (fastapi, uvicorn, kokoro, soundfile, python-multipart) - Verified espeak-ng system dependency is installed - Configured Node.js project with TypeScript, React, and Vite - Set up Tailwind CSS for styling - Set up environment variable configuration for both projects - Created comprehensive README documentation Requirements: 9.6, 10.2, 11.1
This commit is contained in:
771
.kiro/specs/text-to-audiobook/design.md
Normal file
771
.kiro/specs/text-to-audiobook/design.md
Normal file
@@ -0,0 +1,771 @@
|
||||
# Design Document: Text to Audiobook
|
||||
|
||||
## Overview
|
||||
|
||||
The Text to Audiobook system is a full-stack application that converts long-form text into multi-voice audiobooks using Kokoro TTS. The architecture follows a client-server model with a React TypeScript frontend and a Python FastAPI backend running in Docker. The system automatically detects speaker roles in text, allows voice assignment per role, and generates high-quality audio using the Kokoro-82M model.
|
||||
|
||||
The application workflow consists of three main phases:
|
||||
1. **Text Input & Role Detection**: User provides text, system identifies distinct speakers
|
||||
2. **Voice Assignment**: User maps each detected role to a specific Kokoro voice
|
||||
3. **Audio Generation & Playback**: System synthesizes multi-voice audio and provides playback/download
|
||||
|
||||
## Architecture
|
||||
|
||||
### High-Level Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Browser │
|
||||
│ ┌───────────────────────────────────────────────────────┐ │
|
||||
│ │ React Frontend (TypeScript + Vite) │ │
|
||||
│ │ - Text Input Component │ │
|
||||
│ │ - Role Assignment Component │ │
|
||||
│ │ - Audio Player Component │ │
|
||||
│ │ - API Client (tts.ts) │ │
|
||||
│ └───────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
│ HTTP/REST
|
||||
│
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Docker Container │
|
||||
│ ┌───────────────────────────────────────────────────────┐ │
|
||||
│ │ Python Backend (FastAPI) │ │
|
||||
│ │ ┌─────────────────────────────────────────────────┐ │ │
|
||||
│ │ │ API Layer │ │ │
|
||||
│ │ │ - /api/health │ │ │
|
||||
│ │ │ - /api/tts/roles │ │ │
|
||||
│ │ │ - /api/tts │ │ │
|
||||
│ │ └─────────────────────────────────────────────────┘ │ │
|
||||
│ │ ┌─────────────────────────────────────────────────┐ │ │
|
||||
│ │ │ Role Detection Service │ │ │
|
||||
│ │ │ - Pattern matching for role markers │ │ │
|
||||
│ │ │ - Role extraction and deduplication │ │ │
|
||||
│ │ └─────────────────────────────────────────────────┘ │ │
|
||||
│ │ ┌─────────────────────────────────────────────────┐ │ │
|
||||
│ │ │ TTS Service │ │ │
|
||||
│ │ │ - Kokoro KPipeline management │ │ │
|
||||
│ │ │ - Text segmentation │ │ │
|
||||
│ │ │ - Voice synthesis per role │ │ │
|
||||
│ │ │ - Audio concatenation │ │ │
|
||||
│ │ └─────────────────────────────────────────────────┘ │ │
|
||||
│ │ ┌─────────────────────────────────────────────────┐ │ │
|
||||
│ │ │ Kokoro TTS Engine │ │ │
|
||||
│ │ │ - hexgrad/Kokoro-82M model │ │ │
|
||||
│ │ │ - espeak-ng for G2P │ │ │
|
||||
│ │ │ - soundfile for audio I/O │ │ │
|
||||
│ │ └─────────────────────────────────────────────────┘ │ │
|
||||
│ └───────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Technology Stack
|
||||
|
||||
**Frontend:**
|
||||
- React 18+ with TypeScript
|
||||
- Vite for build tooling
|
||||
- Native fetch API for HTTP requests
|
||||
- HTML5 Audio API for playback
|
||||
- CSS/Tailwind for responsive styling
|
||||
|
||||
**Backend:**
|
||||
- Python 3.10+
|
||||
- FastAPI for REST API
|
||||
- Uvicorn as ASGI server
|
||||
- kokoro library (hexgrad/Kokoro-82M)
|
||||
- soundfile for audio processing
|
||||
- espeak-ng for grapheme-to-phoneme conversion
|
||||
|
||||
**Infrastructure:**
|
||||
- Docker for containerization
|
||||
- Docker Compose for orchestration (optional)
|
||||
|
||||
## Components and Interfaces
|
||||
|
||||
### Frontend Components
|
||||
|
||||
#### 1. App Component (`App.tsx`)
|
||||
Main application container managing the overall workflow state.
|
||||
|
||||
**State:**
|
||||
- `text: string` - User input text
|
||||
- `roles: Role[]` - Detected roles from backend
|
||||
- `voiceAssignments: Map<string, string>` - Role ID to Voice ID mapping
|
||||
- `selectedLanguage: string` - Language code (default: 'a')
|
||||
- `speed: number` - Playback speed (default: 1.0)
|
||||
- `audioUrl: string | null` - Generated audio blob URL
|
||||
- `isDetectingRoles: boolean` - Loading state for role detection
|
||||
- `isGenerating: boolean` - Loading state for audio generation
|
||||
- `error: string | null` - Error message if any
|
||||
|
||||
**Methods:**
|
||||
- `handleTextChange(text: string)` - Updates text state
|
||||
- `handleDetectRoles()` - Calls API to detect roles
|
||||
- `handleVoiceAssignment(roleId: string, voiceId: string)` - Updates voice mapping
|
||||
- `handleGenerate()` - Calls API to generate audiobook
|
||||
- `handleDownload()` - Triggers download of generated audio
|
||||
|
||||
#### 2. TextInputSection Component
|
||||
Displays the text input area and "Next: Assign voices" button.
|
||||
|
||||
**Props:**
|
||||
- `text: string`
|
||||
- `onChange: (text: string) => void`
|
||||
- `onNext: () => void`
|
||||
- `isLoading: boolean`
|
||||
- `disabled: boolean`
|
||||
|
||||
#### 3. VoiceAssignmentSection Component
|
||||
Displays role-to-voice mapping interface with language and speed controls.
|
||||
|
||||
**Props:**
|
||||
- `roles: Role[]`
|
||||
- `voiceAssignments: Map<string, string>`
|
||||
- `onVoiceChange: (roleId: string, voiceId: string) => void`
|
||||
- `language: string`
|
||||
- `onLanguageChange: (lang: string) => void`
|
||||
- `speed: number`
|
||||
- `onSpeedChange: (speed: number) => void`
|
||||
- `onGenerate: () => void`
|
||||
- `isGenerating: boolean`
|
||||
- `allRolesAssigned: boolean`
|
||||
|
||||
#### 4. AudioPlayerSection Component
|
||||
Displays audio player and download button after generation.
|
||||
|
||||
**Props:**
|
||||
- `audioUrl: string`
|
||||
- `onDownload: () => void`
|
||||
|
||||
#### 5. API Client (`src/api/tts.ts`)
|
||||
|
||||
**Interfaces:**
|
||||
```typescript
|
||||
export interface Role {
|
||||
id: string;
|
||||
label: string;
|
||||
}
|
||||
|
||||
export interface RoleWithVoice extends Role {
|
||||
voice: string;
|
||||
}
|
||||
|
||||
export interface RolesResponse {
|
||||
roles: Role[];
|
||||
}
|
||||
|
||||
export interface GenerateRequest {
|
||||
text: string;
|
||||
lang: string;
|
||||
roles: { id: string; voice: string }[];
|
||||
speed?: number;
|
||||
}
|
||||
```
|
||||
|
||||
**Functions:**
|
||||
```typescript
|
||||
export async function detectRoles(text: string): Promise<RolesResponse>
|
||||
export async function generateAudiobook(payload: GenerateRequest): Promise<Blob>
|
||||
export async function checkHealth(): Promise<{ status: string }>
|
||||
```
|
||||
|
||||
### Backend Components
|
||||
|
||||
#### 1. API Layer (`main.py`)
|
||||
|
||||
**Endpoints:**
|
||||
|
||||
```python
|
||||
@app.get("/api/health")
|
||||
async def health_check() -> dict:
|
||||
"""Returns service health status"""
|
||||
return {"status": "ok"}
|
||||
|
||||
@app.post("/api/tts/roles")
|
||||
async def detect_roles(request: RoleDetectionRequest) -> RolesResponse:
|
||||
"""Analyzes text and returns detected roles"""
|
||||
pass
|
||||
|
||||
@app.post("/api/tts")
|
||||
async def generate_audiobook(request: TTSRequest) -> StreamingResponse:
|
||||
"""Generates multi-voice audiobook and streams audio"""
|
||||
pass
|
||||
```
|
||||
|
||||
**Request/Response Models:**
|
||||
```python
|
||||
class RoleDetectionRequest(BaseModel):
|
||||
text: str
|
||||
|
||||
class RoleInfo(BaseModel):
|
||||
id: str
|
||||
label: str
|
||||
|
||||
class RolesResponse(BaseModel):
|
||||
roles: List[RoleInfo]
|
||||
|
||||
class RoleVoiceMapping(BaseModel):
|
||||
id: str
|
||||
voice: str
|
||||
|
||||
class TTSRequest(BaseModel):
|
||||
text: str
|
||||
lang: str
|
||||
roles: List[RoleVoiceMapping]
|
||||
speed: Optional[float] = 1.0
|
||||
```
|
||||
|
||||
#### 2. Role Detection Service (`services/role_detector.py`)
|
||||
|
||||
**Class: RoleDetector**
|
||||
|
||||
**Methods:**
|
||||
```python
|
||||
def detect_roles(self, text: str) -> List[RoleInfo]:
|
||||
"""
|
||||
Analyzes text to identify distinct speaker roles.
|
||||
Supports patterns:
|
||||
- "RoleName:" at line start
|
||||
- "[RoleName]" preceding dialogue
|
||||
Returns unique roles or default "Narrator" if none found.
|
||||
"""
|
||||
pass
|
||||
|
||||
def _extract_colon_pattern(self, text: str) -> Set[str]:
|
||||
"""Extracts roles from "Name:" pattern"""
|
||||
pass
|
||||
|
||||
def _extract_bracket_pattern(self, text: str) -> Set[str]:
|
||||
"""Extracts roles from "[Name]" pattern"""
|
||||
pass
|
||||
```
|
||||
|
||||
#### 3. TTS Service (`services/tts_service.py`)
|
||||
|
||||
**Class: TTSService**
|
||||
|
||||
**Attributes:**
|
||||
- `pipelines: Dict[str, KPipeline]` - Cached pipelines per language
|
||||
- `default_voice: str = "af_bella"`
|
||||
|
||||
**Methods:**
|
||||
```python
|
||||
def __init__(self):
|
||||
"""Initializes TTS service"""
|
||||
pass
|
||||
|
||||
def get_pipeline(self, lang_code: str) -> KPipeline:
|
||||
"""
|
||||
Returns or creates KPipeline for given language.
|
||||
Caches pipelines to avoid reloading.
|
||||
"""
|
||||
pass
|
||||
|
||||
def segment_text(self, text: str, role_mappings: Dict[str, str]) -> List[Tuple[str, str]]:
|
||||
"""
|
||||
Segments text into (role_id, text_content) tuples.
|
||||
Identifies role markers and assigns segments accordingly.
|
||||
"""
|
||||
pass
|
||||
|
||||
def synthesize_segment(self, pipeline: KPipeline, text: str, voice: str, speed: float) -> np.ndarray:
|
||||
"""
|
||||
Synthesizes a single text segment with specified voice.
|
||||
Returns audio samples as numpy array.
|
||||
"""
|
||||
pass
|
||||
|
||||
def concatenate_audio(self, segments: List[np.ndarray]) -> np.ndarray:
|
||||
"""
|
||||
Concatenates multiple audio segments into single array.
|
||||
Ensures consistent sample rate (24kHz).
|
||||
"""
|
||||
pass
|
||||
|
||||
async def generate_audiobook(
|
||||
self,
|
||||
text: str,
|
||||
lang: str,
|
||||
role_voice_map: Dict[str, str],
|
||||
speed: float = 1.0
|
||||
) -> bytes:
|
||||
"""
|
||||
Main orchestration method.
|
||||
Segments text, synthesizes each segment, concatenates, and returns audio bytes.
|
||||
"""
|
||||
pass
|
||||
```
|
||||
|
||||
## Data Models
|
||||
|
||||
### Frontend Data Models
|
||||
|
||||
**Role:**
|
||||
```typescript
|
||||
interface Role {
|
||||
id: string; // Unique identifier (e.g., "Narrator", "Alice")
|
||||
label: string; // Display name for UI
|
||||
}
|
||||
```
|
||||
|
||||
**Voice Option:**
|
||||
```typescript
|
||||
interface VoiceOption {
|
||||
id: string; // Kokoro voice ID (e.g., "af_bella")
|
||||
label: string; // Display name (e.g., "Bella (Female, American)")
|
||||
gender: 'male' | 'female';
|
||||
accent: string; // e.g., "American", "British"
|
||||
}
|
||||
```
|
||||
|
||||
**Language Option:**
|
||||
```typescript
|
||||
interface LanguageOption {
|
||||
code: string; // Kokoro language code (e.g., "a", "b", "f", "j")
|
||||
label: string; // Display name (e.g., "American English")
|
||||
}
|
||||
```
|
||||
|
||||
**Application State:**
|
||||
```typescript
|
||||
interface AppState {
|
||||
phase: 'input' | 'assignment' | 'generated';
|
||||
text: string;
|
||||
roles: Role[];
|
||||
voiceAssignments: Map<string, string>;
|
||||
language: string;
|
||||
speed: number;
|
||||
audioUrl: string | null;
|
||||
isLoading: boolean;
|
||||
error: string | null;
|
||||
}
|
||||
```
|
||||
|
||||
### Backend Data Models
|
||||
|
||||
**Text Segment:**
|
||||
```python
|
||||
@dataclass
|
||||
class TextSegment:
|
||||
role_id: str
|
||||
text: str
|
||||
voice: str
|
||||
```
|
||||
|
||||
**Audio Chunk:**
|
||||
```python
|
||||
@dataclass
|
||||
class AudioChunk:
|
||||
samples: np.ndarray
|
||||
sample_rate: int = 24000
|
||||
```
|
||||
|
||||
### Constants
|
||||
|
||||
**Supported Voices:**
|
||||
```typescript
|
||||
const KOKORO_VOICES: VoiceOption[] = [
|
||||
{ id: 'af_bella', label: 'Bella (Female, American)', gender: 'female', accent: 'American' },
|
||||
{ id: 'af_sarah', label: 'Sarah (Female, American)', gender: 'female', accent: 'American' },
|
||||
{ id: 'am_adam', label: 'Adam (Male, American)', gender: 'male', accent: 'American' },
|
||||
{ id: 'am_michael', label: 'Michael (Male, American)', gender: 'male', accent: 'American' },
|
||||
{ id: 'bf_emma', label: 'Emma (Female, British)', gender: 'female', accent: 'British' },
|
||||
{ id: 'bf_isabella', label: 'Isabella (Female, British)', gender: 'female', accent: 'British' },
|
||||
{ id: 'bm_george', label: 'George (Male, British)', gender: 'male', accent: 'British' },
|
||||
{ id: 'bm_lewis', label: 'Lewis (Male, British)', gender: 'male', accent: 'British' },
|
||||
];
|
||||
```
|
||||
|
||||
**Supported Languages:**
|
||||
```typescript
|
||||
const LANGUAGES: LanguageOption[] = [
|
||||
{ code: 'a', label: 'American English' },
|
||||
{ code: 'b', label: 'British English' },
|
||||
{ code: 'f', label: 'French' },
|
||||
{ code: 'j', label: 'Japanese' },
|
||||
];
|
||||
```
|
||||
|
||||
**Role Detection Patterns:**
|
||||
```python
|
||||
ROLE_PATTERNS = {
|
||||
'colon': r'^([A-Z][a-zA-Z\s]+):\s', # "Alice: Hello there"
|
||||
'bracket': r'\[([A-Z][a-zA-Z\s]+)\]', # "[Narrator] The story begins"
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
## Correctness Properties
|
||||
|
||||
*A property is a characteristic or behavior that should hold true across all valid executions of a system—essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.*
|
||||
|
||||
### Property 1: Text persistence invariant
|
||||
*For any* text entered by the user, the text content should remain unchanged as the application transitions between input, assignment, and generated phases.
|
||||
**Validates: Requirements 1.5**
|
||||
|
||||
### Property 2: Role extraction from colon pattern
|
||||
*For any* text containing lines starting with "RoleName:", the role detection should extract RoleName as a distinct role.
|
||||
**Validates: Requirements 2.2**
|
||||
|
||||
### Property 3: Role extraction from bracket pattern
|
||||
*For any* text containing "[RoleName]" patterns, the role detection should extract RoleName as a distinct role.
|
||||
**Validates: Requirements 2.3**
|
||||
|
||||
### Property 4: Role deduplication
|
||||
*For any* text with multiple instances of the same role marker, the role detection should return each unique role exactly once.
|
||||
**Validates: Requirements 2.5**
|
||||
|
||||
### Property 5: Role response structure
|
||||
*For any* detected roles, each role object in the response should contain both id and label fields.
|
||||
**Validates: Requirements 2.6**
|
||||
|
||||
### Property 6: Voice assignment storage
|
||||
*For any* role and voice selection, the frontend should correctly store and retrieve the role-to-voice mapping.
|
||||
**Validates: Requirements 3.4**
|
||||
|
||||
### Property 7: Generate button disabled when incomplete
|
||||
*For any* state where not all roles have assigned voices, the "Generate Audiobook" button should be disabled.
|
||||
**Validates: Requirements 3.5**
|
||||
|
||||
### Property 8: Generate button enabled when complete
|
||||
*For any* state where all roles have assigned voices, the "Generate Audiobook" button should be enabled.
|
||||
**Validates: Requirements 3.6**
|
||||
|
||||
### Property 9: Language selection storage
|
||||
*For any* language selection, the frontend should store the selected language code for the generation request.
|
||||
**Validates: Requirements 4.4**
|
||||
|
||||
### Property 10: Speed range constraints
|
||||
*For any* speed slider interaction, the value should be constrained between 0.5x and 1.5x.
|
||||
**Validates: Requirements 5.2**
|
||||
|
||||
### Property 11: Speed value storage
|
||||
*For any* speed adjustment, the frontend should store the speed value for the generation request.
|
||||
**Validates: Requirements 5.4**
|
||||
|
||||
### Property 12: Generation request completeness
|
||||
*For any* valid application state, the generation request should contain all required fields: text, language code, role-to-voice mappings, and speed.
|
||||
**Validates: Requirements 6.1**
|
||||
|
||||
### Property 13: Voice assignment per segment
|
||||
*For any* text segment with a role marker, the backend should use the Kokoro voice assigned to that role for synthesis.
|
||||
**Validates: Requirements 6.3**
|
||||
|
||||
### Property 14: Text chunking for long content
|
||||
*For any* long text input, the backend should split it into manageable chunks per paragraph or sentence.
|
||||
**Validates: Requirements 6.5**
|
||||
|
||||
### Property 15: Audio concatenation at 24 kHz
|
||||
*For any* set of synthesized chunks, the backend should concatenate them into a single audio stream at 24 kHz sample rate.
|
||||
**Validates: Requirements 6.6**
|
||||
|
||||
### Property 16: Audio response content type
|
||||
*For any* successful synthesis, the HTTP response should include Content-Type header set to audio/mpeg or audio/wav.
|
||||
**Validates: Requirements 6.7**
|
||||
|
||||
### Property 17: Audio response filename
|
||||
*For any* audio response, the HTTP response should include Content-Disposition header with filename "audiobook.mp3" or "audiobook.wav".
|
||||
**Validates: Requirements 6.8**
|
||||
|
||||
### Property 18: Blob URL creation
|
||||
*For any* audio response from the backend, the frontend should convert it to a Blob and create a valid object URL.
|
||||
**Validates: Requirements 7.1**
|
||||
|
||||
### Property 19: Download functionality
|
||||
*For any* generated audio blob, clicking the download button should trigger file save with the correct filename.
|
||||
**Validates: Requirements 7.5**
|
||||
|
||||
### Property 20: Loading state during role detection
|
||||
*For any* role detection request, the frontend should display a loading state indicator until the response is received.
|
||||
**Validates: Requirements 8.1**
|
||||
|
||||
### Property 21: Loading state during generation
|
||||
*For any* audiobook generation request, the frontend should display loading or progress feedback until completion.
|
||||
**Validates: Requirements 8.2**
|
||||
|
||||
### Property 22: Long text warning
|
||||
*For any* text input exceeding a threshold length, the frontend should display a message indicating "Generation may take several minutes for very long content".
|
||||
**Validates: Requirements 8.3**
|
||||
|
||||
### Property 23: Error message display
|
||||
*For any* failed API request, the frontend should display a clear error message to the user.
|
||||
**Validates: Requirements 8.4**
|
||||
|
||||
### Property 24: Role response structure validation
|
||||
*For any* response from /api/tts/roles, the JSON should contain a roles array with objects having id and label fields.
|
||||
**Validates: Requirements 9.3**
|
||||
|
||||
### Property 25: Request validation
|
||||
*For any* request to /api/tts missing required fields (text, lang, or roles), the backend should reject it with an appropriate error.
|
||||
**Validates: Requirements 9.5**
|
||||
|
||||
### Property 26: Pipeline initialization per language
|
||||
*For any* supported language code (a, b, f, j), the backend should successfully load a KPipeline for that language.
|
||||
**Validates: Requirements 10.3**
|
||||
|
||||
### Property 27: Audio sample rate
|
||||
*For any* synthesized audio, the output should have a sample rate of 24 kHz.
|
||||
**Validates: Requirements 10.4**
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Frontend Error Handling
|
||||
|
||||
**Network Errors:**
|
||||
- Catch fetch errors and display user-friendly messages
|
||||
- Implement retry logic for transient failures (optional)
|
||||
- Show specific error messages for timeout, connection refused, etc.
|
||||
|
||||
**Validation Errors:**
|
||||
- Validate text is non-empty before API calls
|
||||
- Validate all roles have assigned voices before generation
|
||||
- Show inline validation messages near relevant controls
|
||||
|
||||
**API Error Responses:**
|
||||
- Parse error responses from backend (4xx, 5xx)
|
||||
- Display error details to user
|
||||
- Log errors to console for debugging
|
||||
|
||||
**Audio Playback Errors:**
|
||||
- Handle cases where audio blob is invalid
|
||||
- Show error if browser doesn't support audio format
|
||||
- Provide fallback message if playback fails
|
||||
|
||||
### Backend Error Handling
|
||||
|
||||
**Request Validation:**
|
||||
- Validate all required fields are present
|
||||
- Return 400 Bad Request with descriptive error message
|
||||
- Validate language codes and voice IDs are supported
|
||||
|
||||
**TTS Processing Errors:**
|
||||
- Catch exceptions during pipeline initialization
|
||||
- Handle synthesis failures gracefully
|
||||
- Return 500 Internal Server Error with error details
|
||||
- Log errors for debugging
|
||||
|
||||
**Resource Management:**
|
||||
- Handle out-of-memory scenarios for very long text
|
||||
- Implement timeouts for long-running synthesis
|
||||
- Clean up temporary audio files/buffers
|
||||
|
||||
**Model Loading Errors:**
|
||||
- Catch errors when loading Kokoro models
|
||||
- Return clear error if models are missing
|
||||
- Provide helpful error messages for setup issues
|
||||
|
||||
### Error Response Format
|
||||
|
||||
```python
|
||||
class ErrorResponse(BaseModel):
|
||||
error: str
|
||||
message: str
|
||||
details: Optional[dict] = None
|
||||
```
|
||||
|
||||
Example:
|
||||
```json
|
||||
{
|
||||
"error": "ValidationError",
|
||||
"message": "Missing required field: lang",
|
||||
"details": {
|
||||
"field": "lang",
|
||||
"required": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Unit Testing
|
||||
|
||||
**Frontend Unit Tests:**
|
||||
- Test individual components in isolation
|
||||
- Mock API calls using jest/vitest mocks
|
||||
- Test state management logic
|
||||
- Test utility functions (e.g., blob URL creation)
|
||||
- Verify UI elements render correctly
|
||||
- Test form validation logic
|
||||
|
||||
**Backend Unit Tests:**
|
||||
- Test role detection patterns independently
|
||||
- Test text segmentation logic
|
||||
- Test audio concatenation
|
||||
- Test API endpoint request/response handling
|
||||
- Mock Kokoro TTS calls to test logic without actual synthesis
|
||||
|
||||
**Example Unit Tests:**
|
||||
- Empty text validation prevents API call
|
||||
- Health endpoint returns correct status
|
||||
- Role detector extracts "Alice:" pattern correctly
|
||||
- Role detector extracts "[Narrator]" pattern correctly
|
||||
- Default "Narrator" role returned when no patterns found
|
||||
- Voice assignment updates state correctly
|
||||
- Download button triggers file save
|
||||
|
||||
### Property-Based Testing
|
||||
|
||||
Property-based tests will verify universal properties across many randomly generated inputs using a PBT library.
|
||||
|
||||
**Frontend Property Tests (using fast-check for TypeScript):**
|
||||
|
||||
We will use **fast-check** as the property-based testing library for the React TypeScript frontend.
|
||||
|
||||
**Backend Property Tests (using Hypothesis for Python):**
|
||||
|
||||
We will use **Hypothesis** as the property-based testing library for the Python FastAPI backend.
|
||||
|
||||
**Configuration:**
|
||||
- Each property-based test should run a minimum of 100 iterations
|
||||
- Tests should generate diverse inputs to explore edge cases
|
||||
- Each test must include a comment referencing the design property it validates
|
||||
|
||||
**Property Test Examples:**
|
||||
|
||||
*Frontend:*
|
||||
- Property 1: Text persistence - generate random text, verify it persists across phase transitions
|
||||
- Property 6: Voice assignment storage - generate random role/voice pairs, verify storage/retrieval
|
||||
- Property 10: Speed range - generate random speed values, verify clamping to 0.5-1.5 range
|
||||
|
||||
*Backend:*
|
||||
- Property 2: Colon pattern extraction - generate random text with "Name:" patterns, verify extraction
|
||||
- Property 3: Bracket pattern extraction - generate random text with "[Name]" patterns, verify extraction
|
||||
- Property 4: Role deduplication - generate text with duplicate roles, verify uniqueness
|
||||
- Property 14: Text chunking - generate long text, verify chunks are reasonable size
|
||||
- Property 27: Audio sample rate - generate random text, verify output is 24 kHz
|
||||
|
||||
### Integration Testing
|
||||
|
||||
**End-to-End Flow Tests:**
|
||||
- Test complete workflow: text input → role detection → voice assignment → generation → playback
|
||||
- Test with sample texts containing multiple roles
|
||||
- Verify audio file is generated and playable
|
||||
- Test error scenarios (network failures, invalid inputs)
|
||||
|
||||
**API Integration Tests:**
|
||||
- Test all API endpoints with real requests
|
||||
- Verify request/response formats match specifications
|
||||
- Test error responses for invalid inputs
|
||||
- Test with different languages and voices
|
||||
|
||||
### Manual Testing Checklist
|
||||
|
||||
- [ ] Enter text and verify role detection works
|
||||
- [ ] Assign voices to all roles
|
||||
- [ ] Generate audiobook and verify audio plays
|
||||
- [ ] Download audiobook and verify file is valid
|
||||
- [ ] Test with very long text (multiple pages)
|
||||
- [ ] Test with text containing no role markers
|
||||
- [ ] Test with different languages (English, French, Japanese)
|
||||
- [ ] Test on mobile devices for responsiveness
|
||||
- [ ] Test error scenarios (empty text, network failure)
|
||||
- [ ] Verify loading states appear during processing
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
### Frontend Implementation
|
||||
|
||||
**State Management:**
|
||||
- Use React hooks (useState, useEffect) for local state
|
||||
- Consider useReducer for complex state transitions
|
||||
- Store voice assignments in a Map for efficient lookups
|
||||
|
||||
**API Client:**
|
||||
- Use native fetch API
|
||||
- Implement error handling and timeout logic
|
||||
- Use environment variable for base URL configuration
|
||||
|
||||
**Audio Handling:**
|
||||
- Use Blob API for audio data
|
||||
- Create object URLs with URL.createObjectURL()
|
||||
- Revoke object URLs when component unmounts to prevent memory leaks
|
||||
- Use HTML5 audio element with controls attribute
|
||||
|
||||
**Styling:**
|
||||
- Use Tailwind CSS for responsive design
|
||||
- Ensure mobile-friendly touch targets (min 44x44px)
|
||||
- Use loading spinners for async operations
|
||||
- Implement accessible form controls
|
||||
|
||||
### Backend Implementation
|
||||
|
||||
**Kokoro Integration:**
|
||||
- Initialize KPipeline instances lazily (on first use)
|
||||
- Cache pipelines per language to avoid reloading
|
||||
- Use appropriate voice IDs from Kokoro documentation
|
||||
- Handle model download/loading errors gracefully
|
||||
|
||||
**Text Processing:**
|
||||
- Use regex for role pattern matching
|
||||
- Split text on paragraph boundaries (\n\n) or sentence boundaries
|
||||
- Preserve whitespace and formatting where appropriate
|
||||
- Handle edge cases (empty lines, special characters)
|
||||
|
||||
**Audio Processing:**
|
||||
- Use soundfile library for audio I/O
|
||||
- Concatenate numpy arrays for audio chunks
|
||||
- Ensure consistent sample rate (24 kHz) across all chunks
|
||||
- Convert to MP3 or WAV format for streaming
|
||||
|
||||
**Performance Optimization:**
|
||||
- Consider async processing for long texts
|
||||
- Implement streaming response for large audio files
|
||||
- Use connection pooling for database (if needed)
|
||||
- Monitor memory usage for very long texts
|
||||
|
||||
**Docker Configuration:**
|
||||
- Use multi-stage build to reduce image size
|
||||
- Install system dependencies (espeak-ng) in base layer
|
||||
- Copy only necessary files to container
|
||||
- Use .dockerignore to exclude unnecessary files
|
||||
- Set appropriate environment variables
|
||||
- Use health check endpoint for container orchestration
|
||||
|
||||
### Security Considerations
|
||||
|
||||
**Input Validation:**
|
||||
- Sanitize user input to prevent injection attacks
|
||||
- Limit text length to prevent DoS (e.g., max 1MB)
|
||||
- Validate language codes and voice IDs against whitelist
|
||||
- Rate limit API endpoints to prevent abuse
|
||||
|
||||
**CORS Configuration:**
|
||||
- Configure CORS to allow frontend origin
|
||||
- Restrict allowed methods and headers
|
||||
- Don't expose sensitive headers
|
||||
|
||||
**Error Messages:**
|
||||
- Don't expose internal implementation details in errors
|
||||
- Log detailed errors server-side only
|
||||
- Return generic error messages to client
|
||||
|
||||
### Deployment Considerations
|
||||
|
||||
**Environment Variables:**
|
||||
- Frontend: VITE_API_BASE_URL
|
||||
- Backend: PORT, MODEL_PATH (optional), LOG_LEVEL
|
||||
|
||||
**Resource Requirements:**
|
||||
- Backend: Minimum 2GB RAM for model loading
|
||||
- Storage: ~500MB for Kokoro models
|
||||
- CPU: Multi-core recommended for faster synthesis
|
||||
|
||||
**Monitoring:**
|
||||
- Log API requests and response times
|
||||
- Monitor memory usage during synthesis
|
||||
- Track error rates and types
|
||||
- Set up health check monitoring
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
**Potential Improvements:**
|
||||
- Add support for more languages and voices
|
||||
- Implement progress tracking for long generations
|
||||
- Add ability to preview individual role voices
|
||||
- Support for SSML markup for advanced control
|
||||
- Batch processing for multiple texts
|
||||
- User accounts and saved voice preferences
|
||||
- Audio editing (trim, adjust volume per role)
|
||||
- Export to different audio formats (FLAC, OGG)
|
||||
- Background processing with job queue
|
||||
- WebSocket for real-time progress updates
|
||||
181
.kiro/specs/text-to-audiobook/requirements.md
Normal file
181
.kiro/specs/text-to-audiobook/requirements.md
Normal file
@@ -0,0 +1,181 @@
|
||||
# Requirements Document
|
||||
|
||||
## Introduction
|
||||
|
||||
The Text to Audiobook system is a production-ready React TypeScript single-page application that converts long-form text into multi-voice audio using Kokoro TTS (hexgrad/Kokoro-82M). The system automatically detects distinct roles/speakers in the input text, allows users to assign different voices to each role, and generates a complete audiobook with appropriate voice assignments. The application consists of a React frontend and a containerized backend service that handles TTS processing locally without relying on external cloud services.
|
||||
|
||||
## Glossary
|
||||
|
||||
- **Kokoro TTS**: A text-to-speech engine using the hexgrad/Kokoro-82M model for high-quality voice synthesis
|
||||
- **Frontend**: The React TypeScript single-page application that provides the user interface
|
||||
- **Backend**: The containerized Node/Express or FastAPI service that processes TTS requests
|
||||
- **Role**: A distinct speaker or character identified in the input text (e.g., Narrator, Alice, Bob)
|
||||
- **Voice ID**: A Kokoro voice identifier (e.g., af_bella, af_sarah, am_adam) assigned to a role
|
||||
- **Language Code**: A single-character code representing the language for synthesis (a=American English, b=British English, f=French, j=Japanese)
|
||||
- **KPipeline**: The Kokoro library's pipeline object for text-to-speech synthesis
|
||||
- **Segment**: A portion of text assigned to a specific role for synthesis
|
||||
- **Role Marker**: Text patterns that identify speakers (e.g., "Alice:", "[Narrator]")
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement 1
|
||||
|
||||
**User Story:** As a user, I want to input long-form text into the application, so that I can convert it into an audiobook.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the user navigates to the application THEN the Frontend SHALL display a large multiline text area for text input
|
||||
2. WHEN the user enters text into the text area THEN the Frontend SHALL accept and store the full content without character limits
|
||||
3. WHEN the text area is empty and the user attempts to proceed THEN the Frontend SHALL prevent progression and display a validation message
|
||||
4. WHEN the user has entered text THEN the Frontend SHALL enable a "Next: Assign voices" button
|
||||
5. THE Frontend SHALL maintain the entered text throughout the voice assignment and generation workflow
|
||||
|
||||
### Requirement 2
|
||||
|
||||
**User Story:** As a user, I want the system to automatically detect distinct roles in my text, so that I can assign different voices to different speakers.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the user submits text for role detection THEN the Backend SHALL analyze the text to identify distinct speaker roles
|
||||
2. WHEN the text contains lines starting with "RoleName:" THEN the Backend SHALL extract RoleName as a distinct role
|
||||
3. WHEN the text contains patterns like "[RoleName]" preceding dialogue THEN the Backend SHALL extract RoleName as a distinct role
|
||||
4. WHEN no clear role markers are detected in the text THEN the Backend SHALL return a single default role named "Narrator"
|
||||
5. WHEN multiple instances of the same role appear THEN the Backend SHALL return each unique role exactly once
|
||||
6. WHEN role detection completes THEN the Backend SHALL return a list of role objects containing id and label fields
|
||||
|
||||
### Requirement 3
|
||||
|
||||
**User Story:** As a user, I want to assign a specific Kokoro voice to each detected role, so that different characters sound distinct in the audiobook.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN roles are detected THEN the Frontend SHALL display a voice assignment section with the title "Assign voices to roles"
|
||||
2. WHEN displaying each role THEN the Frontend SHALL show the role label and a dropdown selector
|
||||
3. WHEN the user opens a voice dropdown THEN the Frontend SHALL populate it with available Kokoro voice IDs including af_bella, af_sarah, am_adam, and bf_emma
|
||||
4. WHEN the user selects a voice for a role THEN the Frontend SHALL store the role-to-voice mapping
|
||||
5. WHEN not all roles have assigned voices THEN the Frontend SHALL keep the "Generate Audiobook" button disabled
|
||||
6. WHEN all roles have assigned voices THEN the Frontend SHALL enable the "Generate Audiobook" button
|
||||
|
||||
### Requirement 4
|
||||
|
||||
**User Story:** As a user, I want to select the language for speech synthesis, so that I can generate audiobooks in different languages.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the voice assignment interface is displayed THEN the Frontend SHALL show a language selector
|
||||
2. WHEN the user opens the language selector THEN the Frontend SHALL display options for American English (a), British English (b), French (f), and Japanese (j)
|
||||
3. WHEN no language is explicitly selected THEN the Frontend SHALL default to American English (a)
|
||||
4. WHEN the user selects a language THEN the Frontend SHALL store the selected language code for the generation request
|
||||
|
||||
### Requirement 5
|
||||
|
||||
**User Story:** As a user, I want to control the playback speed of the generated audio, so that I can adjust the narration pace to my preference.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the voice assignment interface is displayed THEN the Frontend SHALL show a speed control slider
|
||||
2. WHEN the user interacts with the speed slider THEN the Frontend SHALL allow selection of values between 0.5x and 1.5x
|
||||
3. WHEN no speed is explicitly set THEN the Frontend SHALL default to 1.0x speed
|
||||
4. WHEN the user adjusts the speed THEN the Frontend SHALL store the speed value for the generation request
|
||||
|
||||
### Requirement 6
|
||||
|
||||
**User Story:** As a user, I want to generate the audiobook with my selected voices, so that I can listen to the multi-voice narration.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the user clicks "Generate Audiobook" THEN the Frontend SHALL send a request to the Backend with text, language code, role-to-voice mappings, and speed
|
||||
2. WHEN the Backend receives a generation request THEN the Backend SHALL parse the text and assign each segment to its corresponding role based on role markers
|
||||
3. WHEN a text segment matches a role marker THEN the Backend SHALL use the Kokoro voice assigned to that role for synthesis
|
||||
4. WHEN a segment references a role not in the mapping THEN the Backend SHALL use a default narrator voice (af_bella)
|
||||
5. WHEN synthesizing text THEN the Backend SHALL split long content into manageable chunks per paragraph or sentence
|
||||
6. WHEN multiple chunks are synthesized THEN the Backend SHALL concatenate them into a single audio stream at 24 kHz
|
||||
7. WHEN synthesis completes THEN the Backend SHALL stream the audio back with Content-Type audio/mpeg or audio/wav
|
||||
8. WHEN streaming audio THEN the Backend SHALL include Content-Disposition header with filename "audiobook.mp3" or "audiobook.wav"
|
||||
|
||||
### Requirement 7
|
||||
|
||||
**User Story:** As a user, I want to play and download the generated audiobook, so that I can listen to it immediately or save it for later.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the audiobook generation completes THEN the Frontend SHALL convert the response to a Blob and create an object URL
|
||||
2. WHEN the audio is ready THEN the Frontend SHALL display an HTML5 audio player with the generated audio
|
||||
3. WHEN the audio player is displayed THEN the Frontend SHALL show standard playback controls (play, pause, seek, volume)
|
||||
4. WHEN the audio is ready THEN the Frontend SHALL display a "Download audiobook" button
|
||||
5. WHEN the user clicks the download button THEN the Frontend SHALL save the file as "audiobook.mp3" or "audiobook.wav"
|
||||
|
||||
### Requirement 8
|
||||
|
||||
**User Story:** As a user, I want clear feedback during processing, so that I understand the system status and can anticipate wait times.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the user initiates role detection THEN the Frontend SHALL display a loading state indicator
|
||||
2. WHEN the user initiates audiobook generation THEN the Frontend SHALL display loading or progress feedback
|
||||
3. WHEN processing long text THEN the Frontend SHALL display a message indicating "Generation may take several minutes for very long content"
|
||||
4. WHEN an API request fails THEN the Frontend SHALL display a clear error message to the user
|
||||
5. WHEN the Backend is processing THEN the Backend SHALL maintain responsive behavior without blocking
|
||||
|
||||
### Requirement 9
|
||||
|
||||
**User Story:** As a developer, I want a well-defined API contract between frontend and backend, so that the services can communicate reliably.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Backend SHALL expose a GET endpoint at /api/health that returns status "ok"
|
||||
2. THE Backend SHALL expose a POST endpoint at /api/tts/roles that accepts text and returns detected roles
|
||||
3. WHEN /api/tts/roles receives a request THEN the Backend SHALL return JSON with a roles array containing id and label fields
|
||||
4. THE Backend SHALL expose a POST endpoint at /api/tts that accepts text, lang, roles array, and optional speed
|
||||
5. WHEN /api/tts receives a request THEN the Backend SHALL validate all required fields are present
|
||||
6. THE Frontend SHALL use environment variable VITE_API_BASE_URL for the API base URL
|
||||
7. THE Frontend SHALL implement TypeScript interfaces for Role, RoleWithVoice, and RolesResponse
|
||||
|
||||
### Requirement 10
|
||||
|
||||
**User Story:** As a developer, I want the backend to use Kokoro TTS (hexgrad/Kokoro-82M) for synthesis, so that we have high-quality local voice generation.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Backend SHALL use the kokoro Python library with the hexgrad/Kokoro-82M model
|
||||
2. THE Backend SHALL install dependencies including kokoro, soundfile, and espeak-ng
|
||||
3. WHEN initializing TTS THEN the Backend SHALL load a KPipeline with the appropriate language code
|
||||
4. WHEN synthesizing text THEN the Backend SHALL use the KPipeline to generate audio at 24 kHz
|
||||
5. THE Backend SHALL support language codes a (American English), b (British English), f (French), and j (Japanese)
|
||||
6. THE Backend SHALL support Kokoro voice IDs including af_bella, af_sarah, am_adam, and bf_emma
|
||||
|
||||
### Requirement 11
|
||||
|
||||
**User Story:** As a developer, I want the backend containerized with Docker, so that it can be deployed consistently across environments.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Backend SHALL include a Dockerfile that installs Python and all dependencies
|
||||
2. WHEN building the Docker image THEN the Dockerfile SHALL install kokoro, soundfile, espeak-ng, and model files
|
||||
3. THE Dockerfile SHALL copy the backend code into the container
|
||||
4. THE Dockerfile SHALL expose port 8000 or 3001
|
||||
5. WHEN the container starts THEN the Dockerfile SHALL start the server using Uvicorn or equivalent
|
||||
6. THE Backend MAY include a docker-compose.yml for simplified orchestration
|
||||
|
||||
### Requirement 12
|
||||
|
||||
**User Story:** As a user, I want the interface to be responsive and usable on mobile devices, so that I can use the application on any device.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the application is viewed on mobile THEN the Frontend SHALL display all elements in a readable layout
|
||||
2. WHEN the user interacts with controls on mobile THEN the Frontend SHALL respond appropriately to touch inputs
|
||||
3. THE Frontend SHALL use responsive CSS or Tailwind to adapt to different screen sizes
|
||||
4. WHEN text areas and dropdowns are displayed THEN the Frontend SHALL ensure they are appropriately sized for the viewport
|
||||
|
||||
### Requirement 13
|
||||
|
||||
**User Story:** As a developer, I want comprehensive documentation, so that I can set up and run the application easily.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE project SHALL include a README explaining how to download and configure Kokoro model files
|
||||
2. THE README SHALL document how to run the backend including Docker commands
|
||||
3. THE README SHALL document how to run the frontend with example commands
|
||||
4. THE README SHALL include steps to test the full workflow from text input to audio generation
|
||||
5. THE README MAY include screenshots or diagrams illustrating the application flow
|
||||
300
.kiro/specs/text-to-audiobook/tasks.md
Normal file
300
.kiro/specs/text-to-audiobook/tasks.md
Normal file
@@ -0,0 +1,300 @@
|
||||
# Implementation Plan
|
||||
|
||||
- [-] 1. Set up project structure and dependencies
|
||||
- Create backend directory with FastAPI project structure
|
||||
- Create frontend directory with React + TypeScript + Vite
|
||||
- Configure Python virtual environment and install dependencies (fastapi, uvicorn, kokoro, soundfile, python-multipart)
|
||||
- Install espeak-ng system dependency
|
||||
- Configure Node.js project with TypeScript, React, and Vite
|
||||
- Set up environment variable configuration for both projects
|
||||
- _Requirements: 9.6, 10.2, 11.1_
|
||||
|
||||
- [ ] 2. Implement backend API foundation
|
||||
- Create FastAPI application with CORS middleware
|
||||
- Define Pydantic models for requests and responses (RoleDetectionRequest, RolesResponse, TTSRequest, ErrorResponse)
|
||||
- Implement /api/health endpoint
|
||||
- Set up error handling middleware
|
||||
- Configure logging
|
||||
- _Requirements: 9.1, 9.2, 9.4_
|
||||
|
||||
- [ ]* 2.1 Write unit tests for API foundation
|
||||
- Test health endpoint returns correct status
|
||||
- Test CORS headers are set correctly
|
||||
- Test error response format
|
||||
- _Requirements: 9.1_
|
||||
|
||||
- [ ] 3. Implement role detection service
|
||||
- Create RoleDetector class with pattern matching logic
|
||||
- Implement colon pattern extraction ("RoleName:")
|
||||
- Implement bracket pattern extraction ("[RoleName]")
|
||||
- Implement role deduplication logic
|
||||
- Implement fallback to default "Narrator" role
|
||||
- Create /api/tts/roles endpoint using RoleDetector
|
||||
- _Requirements: 2.1, 2.2, 2.3, 2.4, 2.5, 2.6_
|
||||
|
||||
- [ ]* 3.1 Write property test for colon pattern extraction
|
||||
- **Property 2: Role extraction from colon pattern**
|
||||
- **Validates: Requirements 2.2**
|
||||
|
||||
- [ ]* 3.2 Write property test for bracket pattern extraction
|
||||
- **Property 3: Role extraction from bracket pattern**
|
||||
- **Validates: Requirements 2.3**
|
||||
|
||||
- [ ]* 3.3 Write property test for role deduplication
|
||||
- **Property 4: Role deduplication**
|
||||
- **Validates: Requirements 2.5**
|
||||
|
||||
- [ ]* 3.4 Write property test for role response structure
|
||||
- **Property 5: Role response structure**
|
||||
- **Validates: Requirements 2.6**
|
||||
|
||||
- [ ]* 3.5 Write unit tests for role detection edge cases
|
||||
- Test empty text returns default "Narrator" role
|
||||
- Test text with no role markers returns "Narrator"
|
||||
- Test mixed patterns (both colon and bracket)
|
||||
- _Requirements: 2.4_
|
||||
|
||||
- [ ] 4. Implement TTS service with Kokoro integration
|
||||
- Create TTSService class
|
||||
- Implement KPipeline initialization and caching per language
|
||||
- Implement text segmentation logic to identify role markers and split text
|
||||
- Implement segment synthesis with voice assignment
|
||||
- Implement audio concatenation at 24 kHz
|
||||
- Handle fallback to default voice for unmapped roles
|
||||
- _Requirements: 10.1, 10.3, 10.4, 6.2, 6.3, 6.4, 6.5, 6.6_
|
||||
|
||||
- [ ]* 4.1 Write property test for pipeline initialization
|
||||
- **Property 26: Pipeline initialization per language**
|
||||
- **Validates: Requirements 10.3**
|
||||
|
||||
- [ ]* 4.2 Write property test for voice assignment per segment
|
||||
- **Property 13: Voice assignment per segment**
|
||||
- **Validates: Requirements 6.3**
|
||||
|
||||
- [ ]* 4.3 Write property test for text chunking
|
||||
- **Property 14: Text chunking for long content**
|
||||
- **Validates: Requirements 6.5**
|
||||
|
||||
- [ ]* 4.4 Write property test for audio concatenation
|
||||
- **Property 15: Audio concatenation at 24 kHz**
|
||||
- **Validates: Requirements 6.6**
|
||||
|
||||
- [ ]* 4.5 Write property test for audio sample rate
|
||||
- **Property 27: Audio sample rate**
|
||||
- **Validates: Requirements 10.4**
|
||||
|
||||
- [ ]* 4.6 Write unit tests for TTS service
|
||||
- Test default voice fallback for unmapped roles
|
||||
- Test synthesis with different language codes (a, b, f, j)
|
||||
- Test synthesis with different voice IDs
|
||||
- _Requirements: 6.4, 10.5, 10.6_
|
||||
|
||||
- [ ] 5. Implement audio generation endpoint
|
||||
- Create /api/tts POST endpoint
|
||||
- Validate request fields (text, lang, roles, speed)
|
||||
- Integrate TTSService to generate audio
|
||||
- Convert audio to bytes (WAV or MP3 format)
|
||||
- Stream response with correct Content-Type header
|
||||
- Add Content-Disposition header with filename
|
||||
- _Requirements: 6.1, 6.7, 6.8, 9.5_
|
||||
|
||||
- [ ]* 5.1 Write property test for request validation
|
||||
- **Property 25: Request validation**
|
||||
- **Validates: Requirements 9.5**
|
||||
|
||||
- [ ]* 5.2 Write property test for audio response headers
|
||||
- **Property 16: Audio response content type**
|
||||
- **Property 17: Audio response filename**
|
||||
- **Validates: Requirements 6.7, 6.8**
|
||||
|
||||
- [ ]* 5.3 Write unit tests for generation endpoint
|
||||
- Test successful generation returns audio
|
||||
- Test missing required fields returns 400 error
|
||||
- Test invalid language code returns error
|
||||
- Test invalid voice ID returns error
|
||||
- _Requirements: 9.5_
|
||||
|
||||
- [ ] 6. Create backend Dockerfile and docker-compose
|
||||
- Write Dockerfile with Python base image
|
||||
- Install system dependencies (espeak-ng)
|
||||
- Install Python dependencies
|
||||
- Copy backend code
|
||||
- Expose port 8000
|
||||
- Set CMD to start Uvicorn server
|
||||
- Create docker-compose.yml for easy orchestration
|
||||
- _Requirements: 11.1, 11.2, 11.3, 11.4, 11.5, 11.6_
|
||||
|
||||
- [ ]* 6.1 Test Docker container builds and runs
|
||||
- Build Docker image successfully
|
||||
- Start container and verify health endpoint responds
|
||||
- _Requirements: 11.5_
|
||||
|
||||
- [ ] 7. Set up React frontend project structure
|
||||
- Initialize Vite project with React and TypeScript
|
||||
- Configure TypeScript with strict mode
|
||||
- Set up Tailwind CSS for styling
|
||||
- Create component directory structure
|
||||
- Configure environment variables (.env file with VITE_API_BASE_URL)
|
||||
- _Requirements: 9.6, 12.3_
|
||||
|
||||
- [ ] 8. Implement API client module
|
||||
- Create src/api/tts.ts with TypeScript interfaces (Role, RoleWithVoice, RolesResponse, GenerateRequest)
|
||||
- Implement detectRoles() function
|
||||
- Implement generateAudiobook() function
|
||||
- Implement checkHealth() function
|
||||
- Add error handling for network failures
|
||||
- Use VITE_API_BASE_URL environment variable
|
||||
- _Requirements: 9.6, 9.7_
|
||||
|
||||
- [ ]* 8.1 Write unit tests for API client
|
||||
- Test API client reads VITE_API_BASE_URL
|
||||
- Test error handling for network failures
|
||||
- Mock fetch calls and verify request format
|
||||
- _Requirements: 9.6_
|
||||
|
||||
- [ ] 9. Implement text input component
|
||||
- Create TextInputSection component
|
||||
- Add large multiline textarea
|
||||
- Add "Next: Assign voices" button
|
||||
- Implement text validation (non-empty)
|
||||
- Show validation message for empty text
|
||||
- Disable button when text is empty
|
||||
- _Requirements: 1.1, 1.2, 1.3, 1.4_
|
||||
|
||||
- [ ]* 9.1 Write property test for text acceptance
|
||||
- **Property 1: Text persistence invariant** (partial - input phase)
|
||||
- **Validates: Requirements 1.2**
|
||||
|
||||
- [ ]* 9.2 Write unit tests for text input component
|
||||
- Test textarea renders correctly
|
||||
- Test button is disabled when text is empty
|
||||
- Test button is enabled when text is present
|
||||
- Test validation message appears for empty text
|
||||
- _Requirements: 1.1, 1.3, 1.4_
|
||||
|
||||
- [ ] 10. Implement voice assignment component
|
||||
- Create VoiceAssignmentSection component
|
||||
- Display "Assign voices to roles" title
|
||||
- Render role list with labels and dropdowns
|
||||
- Populate dropdowns with Kokoro voice options (af_bella, af_sarah, am_adam, bf_emma, etc.)
|
||||
- Implement voice selection handler
|
||||
- Add language selector with options (American English, British English, French, Japanese)
|
||||
- Add speed slider (0.5x - 1.5x range)
|
||||
- Add "Generate Audiobook" button
|
||||
- Disable generate button until all roles have voices
|
||||
- _Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 4.1, 4.2, 4.3, 4.4, 5.1, 5.2, 5.3_
|
||||
|
||||
- [ ]* 10.1 Write property test for voice assignment storage
|
||||
- **Property 6: Voice assignment storage**
|
||||
- **Validates: Requirements 3.4**
|
||||
|
||||
- [ ]* 10.2 Write property test for generate button state
|
||||
- **Property 7: Generate button disabled when incomplete**
|
||||
- **Property 8: Generate button enabled when complete**
|
||||
- **Validates: Requirements 3.5, 3.6**
|
||||
|
||||
- [ ]* 10.3 Write property test for language selection
|
||||
- **Property 9: Language selection storage**
|
||||
- **Validates: Requirements 4.4**
|
||||
|
||||
- [ ]* 10.4 Write property test for speed range
|
||||
- **Property 10: Speed range constraints**
|
||||
- **Validates: Requirements 5.2**
|
||||
|
||||
- [ ]* 10.5 Write property test for speed storage
|
||||
- **Property 11: Speed value storage**
|
||||
- **Validates: Requirements 5.4**
|
||||
|
||||
- [ ]* 10.6 Write unit tests for voice assignment component
|
||||
- Test section title displays correctly
|
||||
- Test dropdowns contain required voices
|
||||
- Test language selector contains required options
|
||||
- Test default language is American English
|
||||
- Test default speed is 1.0x
|
||||
- Test speed slider enforces min/max bounds
|
||||
- _Requirements: 3.1, 3.3, 4.2, 4.3, 5.3_
|
||||
|
||||
- [ ] 11. Implement audio player component
|
||||
- Create AudioPlayerSection component
|
||||
- Display HTML5 audio element with controls
|
||||
- Add "Download audiobook" button
|
||||
- Implement download functionality using blob URL
|
||||
- _Requirements: 7.2, 7.3, 7.4, 7.5_
|
||||
|
||||
- [ ]* 11.1 Write property test for blob URL creation
|
||||
- **Property 18: Blob URL creation**
|
||||
- **Validates: Requirements 7.1**
|
||||
|
||||
- [ ]* 11.2 Write property test for download functionality
|
||||
- **Property 19: Download functionality**
|
||||
- **Validates: Requirements 7.5**
|
||||
|
||||
- [ ]* 11.3 Write unit tests for audio player component
|
||||
- Test audio element renders with controls
|
||||
- Test download button appears
|
||||
- Test download button triggers file save
|
||||
- _Requirements: 7.2, 7.3, 7.4_
|
||||
|
||||
- [ ] 12. Implement main App component with workflow orchestration
|
||||
- Create App component with state management
|
||||
- Implement phase transitions (input → assignment → generated)
|
||||
- Wire up text input to role detection
|
||||
- Wire up voice assignment to audio generation
|
||||
- Implement loading states for async operations
|
||||
- Implement error handling and display
|
||||
- Add warning message for long text
|
||||
- Ensure text persists across phases
|
||||
- _Requirements: 1.5, 6.1, 8.1, 8.2, 8.3, 8.4_
|
||||
|
||||
- [ ]* 12.1 Write property test for text persistence
|
||||
- **Property 1: Text persistence invariant** (complete workflow)
|
||||
- **Validates: Requirements 1.5**
|
||||
|
||||
- [ ]* 12.2 Write property test for generation request completeness
|
||||
- **Property 12: Generation request completeness**
|
||||
- **Validates: Requirements 6.1**
|
||||
|
||||
- [ ]* 12.3 Write property test for loading states
|
||||
- **Property 20: Loading state during role detection**
|
||||
- **Property 21: Loading state during generation**
|
||||
- **Validates: Requirements 8.1, 8.2**
|
||||
|
||||
- [ ]* 12.4 Write property test for long text warning
|
||||
- **Property 22: Long text warning**
|
||||
- **Validates: Requirements 8.3**
|
||||
|
||||
- [ ]* 12.5 Write property test for error display
|
||||
- **Property 23: Error message display**
|
||||
- **Validates: Requirements 8.4**
|
||||
|
||||
- [ ]* 12.6 Write unit tests for App component
|
||||
- Test phase transitions work correctly
|
||||
- Test error states display messages
|
||||
- Test loading states appear during API calls
|
||||
- _Requirements: 8.1, 8.2, 8.4_
|
||||
|
||||
- [ ] 13. Add responsive styling and mobile support
|
||||
- Apply Tailwind CSS classes for responsive layout
|
||||
- Ensure text areas and controls are mobile-friendly
|
||||
- Test on different screen sizes
|
||||
- Ensure touch targets are appropriately sized
|
||||
- _Requirements: 12.1, 12.2, 12.3, 12.4_
|
||||
|
||||
- [ ] 14. Create comprehensive README documentation
|
||||
- Document Kokoro model setup and configuration
|
||||
- Provide backend setup instructions (local and Docker)
|
||||
- Provide frontend setup instructions
|
||||
- Include example commands for running both services
|
||||
- Add workflow testing steps
|
||||
- Include troubleshooting section
|
||||
- _Requirements: 13.1, 13.2, 13.3, 13.4, 13.5_
|
||||
|
||||
- [ ] 15. Final integration testing and polish
|
||||
- Test complete end-to-end workflow
|
||||
- Test with various text samples (short, long, multiple roles, no roles)
|
||||
- Test all language options (a, b, f, j)
|
||||
- Test error scenarios (network failures, invalid inputs)
|
||||
- Verify audio quality and playback
|
||||
- Test Docker deployment
|
||||
- Fix any bugs discovered during testing
|
||||
- _Requirements: All_
|
||||
Reference in New Issue
Block a user