Admin/audiobook
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
2f200aa61baaf9d5bbd1a08b3d169658dcf7b06a
audiobook/.kiro/specs/text-to-audiobook/tasks.md
Paul Bailey 706ada866c feat: modernize UI with gradient design, glass morphism, and enhanced visual appeal 
- Add purple/indigo gradient background throughout app
- Implement glass morphism effects on all cards
- Add gradient buttons with smooth hover animations
- Integrate emoji icons for better visual communication
- Enhance typography with improved font stack
- Add loading spinners and smooth transitions
- Improve color scheme with modern purple/indigo palette
- Add glowing shadows and depth effects
- Maintain responsive design and touch-friendly interactions
2025-12-09 12:29:40 +00:00

18 KiB
Raw Blame History

Implementation Plan

  • 1. Set up project structure and dependencies

    • Create backend directory with FastAPI project structure
    • Create frontend directory with React + TypeScript + Vite
    • Configure Python virtual environment and install dependencies (fastapi, uvicorn, kokoro, soundfile, python-multipart)
    • Install espeak-ng system dependency
    • Configure Node.js project with TypeScript, React, and Vite
    • Set up environment variable configuration for both projects
    • Requirements: 9.6, 10.2, 11.1

  • 2. Implement backend API foundation

    • Create FastAPI application with CORS middleware
    • Define Pydantic models for requests and responses (RoleDetectionRequest, RolesResponse, TTSRequest, ErrorResponse)
    • Implement /api/health endpoint
    • Set up error handling middleware
    • Configure logging
    • Requirements: 9.1, 9.2, 9.4

  • * 2.1 Write unit tests for API foundation

    • Test health endpoint returns correct status
    • Test CORS headers are set correctly
    • Test error response format
    • Requirements: 9.1

  • 3. Implement role detection service

    • Create RoleDetector class with pattern matching logic
    • Implement colon pattern extraction ("RoleName:")
    • Implement bracket pattern extraction ("[RoleName]")
    • Implement role deduplication logic
    • Implement fallback to default "Narrator" role
    • Create /api/tts/roles endpoint using RoleDetector
    • Requirements: 2.1, 2.2, 2.3, 2.4, 2.5, 2.6

  • * 3.1 Write property test for colon pattern extraction

    • Property 2: Role extraction from colon pattern
    • Validates: Requirements 2.2

  • * 3.2 Write property test for bracket pattern extraction

    • Property 3: Role extraction from bracket pattern
    • Validates: Requirements 2.3

  • * 3.3 Write property test for role deduplication

    • Property 4: Role deduplication
    • Validates: Requirements 2.5

  • * 3.4 Write property test for role response structure

    • Property 5: Role response structure
    • Validates: Requirements 2.6

  • * 3.5 Write unit tests for role detection edge cases

    • Test empty text returns default "Narrator" role
    • Test text with no role markers returns "Narrator"
    • Test mixed patterns (both colon and bracket)
    • Requirements: 2.4

  • 4. Implement TTS service with Kokoro integration

    • Create TTSService class
    • Implement KPipeline initialization and caching per language
    • Implement text segmentation logic to identify role markers and split text
    • Implement segment synthesis with voice assignment
    • Implement audio concatenation at 24 kHz
    • Handle fallback to default voice for unmapped roles
    • Requirements: 10.1, 10.3, 10.4, 6.2, 6.3, 6.4, 6.5, 6.6

  • * 4.1 Write property test for pipeline initialization

    • Property 26: Pipeline initialization per language
    • Validates: Requirements 10.3

  • * 4.2 Write property test for voice assignment per segment

    • Property 13: Voice assignment per segment
    • Validates: Requirements 6.3

  • * 4.3 Write property test for text chunking

    • Property 14: Text chunking for long content
    • Validates: Requirements 6.5

  • * 4.4 Write property test for audio concatenation

    • Property 15: Audio concatenation at 24 kHz
    • Validates: Requirements 6.6

  • * 4.5 Write property test for audio sample rate

    • Property 27: Audio sample rate
    • Validates: Requirements 10.4

  • * 4.6 Write unit tests for TTS service

    • Test default voice fallback for unmapped roles
    • Test synthesis with different language codes (a, b, f, j)
    • Test synthesis with different voice IDs
    • Requirements: 6.4, 10.5, 10.6

  • 5. Implement audio generation endpoint

    • Create /api/tts POST endpoint
    • Validate request fields (text, lang, roles, speed)
    • Integrate TTSService to generate audio
    • Convert audio to bytes (WAV or MP3 format)
    • Stream response with correct Content-Type header
    • Add Content-Disposition header with filename
    • Requirements: 6.1, 6.7, 6.8, 9.5

  • * 5.1 Write property test for request validation

    • Property 25: Request validation
    • Validates: Requirements 9.5

  • * 5.2 Write property test for audio response headers

    • Property 16: Audio response content type
    • Property 17: Audio response filename
    • Validates: Requirements 6.7, 6.8

  • * 5.3 Write unit tests for generation endpoint

    • Test successful generation returns audio
    • Test missing required fields returns 400 error
    • Test invalid language code returns error
    • Test invalid voice ID returns error
    • Requirements: 9.5

  • 6. Create backend Dockerfile and docker-compose

    • Write Dockerfile with Python base image
    • Install system dependencies (espeak-ng)
    • Install Python dependencies
    • Copy backend code
    • Expose port 8000
    • Set CMD to start Uvicorn server
    • Create docker-compose.yml for easy orchestration
    • Requirements: 11.1, 11.2, 11.3, 11.4, 11.5, 11.6

  • * 6.1 Test Docker container builds and runs

    • Build Docker image successfully
    • Start container and verify health endpoint responds
    • Requirements: 11.5

  • 7. Set up React frontend project structure

    • Initialize Vite project with React and TypeScript
    • Configure TypeScript with strict mode
    • Set up Tailwind CSS for styling
    • Create component directory structure
    • Configure environment variables (.env file with VITE_API_BASE_URL)
    • Requirements: 9.6, 12.3

  • 8. Implement API client module

    • Create src/api/tts.ts with TypeScript interfaces (Role, RoleWithVoice, RolesResponse, GenerateRequest)
    • Implement detectRoles() function
    • Implement generateAudiobook() function
    • Implement checkHealth() function
    • Add error handling for network failures
    • Use VITE_API_BASE_URL environment variable
    • Requirements: 9.6, 9.7

  • * 8.1 Write unit tests for API client

    • Test API client reads VITE_API_BASE_URL
    • Test error handling for network failures
    • Mock fetch calls and verify request format
    • Requirements: 9.6

  • 9. Implement text input component

    • Create TextInputSection component
    • Add large multiline textarea
    • Add "Next: Assign voices" button
    • Implement text validation (non-empty)
    • Show validation message for empty text
    • Disable button when text is empty
    • Requirements: 1.1, 1.2, 1.3, 1.4

  • * 9.1 Write property test for text acceptance

    • Property 1: Text persistence invariant (partial - input phase)
    • Validates: Requirements 1.2

  • * 9.2 Write unit tests for text input component

    • Test textarea renders correctly
    • Test button is disabled when text is empty
    • Test button is enabled when text is present
    • Test validation message appears for empty text
    • Requirements: 1.1, 1.3, 1.4

  • 10. Implement voice assignment component

    • Create VoiceAssignmentSection component
    • Display "Assign voices to roles" title
    • Render role list with labels and dropdowns
    • Populate dropdowns with Kokoro voice options (af_bella, af_sarah, am_adam, bf_emma, etc.)
    • Implement voice selection handler
    • Add language selector with options (American English, British English, French, Japanese)
    • Add speed slider (0.5x - 1.5x range)
    • Add "Generate Audiobook" button
    • Disable generate button until all roles have voices
    • Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 4.1, 4.2, 4.3, 4.4, 5.1, 5.2, 5.3

  • * 10.1 Write property test for voice assignment storage

    • Property 6: Voice assignment storage
    • Validates: Requirements 3.4

  • * 10.2 Write property test for generate button state

    • Property 7: Generate button disabled when incomplete
    • Property 8: Generate button enabled when complete
    • Validates: Requirements 3.5, 3.6

  • * 10.3 Write property test for language selection

    • Property 9: Language selection storage
    • Validates: Requirements 4.4

  • * 10.4 Write property test for speed range

    • Property 10: Speed range constraints
    • Validates: Requirements 5.2

  • * 10.5 Write property test for speed storage

    • Property 11: Speed value storage
    • Validates: Requirements 5.4

  • * 10.6 Write unit tests for voice assignment component

    • Test section title displays correctly
    • Test dropdowns contain required voices
    • Test language selector contains required options
    • Test default language is American English
    • Test default speed is 1.0x
    • Test speed slider enforces min/max bounds
    • Requirements: 3.1, 3.3, 4.2, 4.3, 5.3

  • 11. Implement audio player component

    • Create AudioPlayerSection component
    • Display HTML5 audio element with controls
    • Add "Download audiobook" button
    • Implement download functionality using blob URL
    • Requirements: 7.2, 7.3, 7.4, 7.5

  • * 11.1 Write property test for blob URL creation

    • Property 18: Blob URL creation
    • Validates: Requirements 7.1

  • * 11.2 Write property test for download functionality

    • Property 19: Download functionality
    • Validates: Requirements 7.5

  • * 11.3 Write unit tests for audio player component

    • Test audio element renders with controls
    • Test download button appears
    • Test download button triggers file save
    • Requirements: 7.2, 7.3, 7.4

  • 12. Implement main App component with workflow orchestration

    • Create App component with state management
    • Implement phase transitions (input → assignment → generated)
    • Wire up text input to role detection
    • Wire up voice assignment to audio generation
    • Implement loading states for async operations
    • Implement error handling and display
    • Add warning message for long text
    • Ensure text persists across phases
    • Requirements: 1.5, 6.1, 8.1, 8.2, 8.3, 8.4

  • * 12.1 Write property test for text persistence

    • Property 1: Text persistence invariant (complete workflow)
    • Validates: Requirements 1.5

  • * 12.2 Write property test for generation request completeness

    • Property 12: Generation request completeness
    • Validates: Requirements 6.1

  • * 12.3 Write property test for loading states

    • Property 20: Loading state during role detection
    • Property 21: Loading state during generation
    • Validates: Requirements 8.1, 8.2

  • * 12.4 Write property test for long text warning

    • Property 22: Long text warning
    • Validates: Requirements 8.3

  • * 12.5 Write property test for error display

    • Property 23: Error message display
    • Validates: Requirements 8.4

  • * 12.6 Write unit tests for App component

    • Test phase transitions work correctly
    • Test error states display messages
    • Test loading states appear during API calls
    • Requirements: 8.1, 8.2, 8.4

  • 13. Add responsive styling and mobile support

    • Apply Tailwind CSS classes for responsive layout
    • Ensure text areas and controls are mobile-friendly
    • Test on different screen sizes
    • Ensure touch targets are appropriately sized
    • Requirements: 12.1, 12.2, 12.3, 12.4

  • 14. Create comprehensive README documentation

    • Document Kokoro model setup and configuration
    • Provide backend setup instructions (local and Docker)
    • Provide frontend setup instructions
    • Include example commands for running both services
    • Add workflow testing steps
    • Include troubleshooting section
    • Requirements: 13.1, 13.2, 13.3, 13.4, 13.5

  • 15. Implement Ollama integration service

    • Create OllamaService class in services/ollama_service.py
    • Implement connection to Ollama API using httpx
    • Configure OLLAMA_BASE_URL environment variable (default: http://localhost:11434)
    • Implement detect_speakers() method with structured prompt
    • Implement prompt builder for speaker detection
    • Implement JSON response parser
    • Implement is_available() health check method
    • Add timeout handling (30 seconds)
    • Requirements: 14.1, 14.2, 14.3, 14.6

  • * 15.1 Write property test for Ollama speaker detection

    • Property 28: Ollama speaker detection
    • Validates: Requirements 14.1

  • * 15.2 Write property test for JSON response parsing

    • Property 29: Ollama JSON response parsing
    • Validates: Requirements 14.3

  • * 15.3 Write unit tests for Ollama service

    • Test connection to Ollama
    • Test prompt construction
    • Test JSON parsing with various formats
    • Test timeout handling
    • Test error handling for unavailable service
    • Requirements: 14.1, 14.2, 14.3

  • 16. Integrate Ollama into role detection

    • Update RoleDetector to use OllamaService
    • Implement _detect_with_ollama() method
    • Add fallback logic: try Ollama first, then pattern matching
    • Implement role merging and deduplication across both methods
    • Handle Ollama unavailable gracefully
    • Log detection method used (AI vs pattern)
    • Requirements: 14.1, 14.4, 14.5

  • * 16.1 Write property test for Ollama fallback

    • Property 30: Ollama fallback behavior
    • Validates: Requirements 14.4

  • * 16.2 Write property test for role deduplication

    • Property 31: Role deduplication across detection methods
    • Validates: Requirements 14.5

  • * 16.3 Write unit tests for integrated role detection

    • Test with Ollama available
    • Test with Ollama unavailable
    • Test role merging from both sources
    • Test deduplication logic
    • Requirements: 14.4, 14.5

  • 17. Update documentation for Ollama integration

    • Document Ollama setup in README
    • Add instructions for installing and running Ollama
    • Document llama3:8B model requirement
    • Add OLLAMA_BASE_URL environment variable to .env.example
    • Update backend README with Ollama configuration
    • Add troubleshooting section for Ollama issues
    • Requirements: 14.6

  • 18. Implement script generation endpoint

    • Create /api/tts/script POST endpoint in main.py
    • Define ScriptRequest and ScriptResponse Pydantic models
    • Define ScriptSegment model with id, speaker, and text fields
    • Integrate with TTSService to generate script segments using AI
    • Return segments with unique IDs for frontend tracking
    • Handle errors gracefully when AI segmentation fails
    • Requirements: 15.1, 15.2

  • * 18.1 Write property test for script generation

    • Property 32: Script generation completeness
    • Validates: Requirements 15.2

  • * 18.2 Write property test for script segment structure

    • Property 33: Script segment structure
    • Validates: Requirements 15.4

  • * 18.3 Write unit tests for script endpoint

    • Test successful script generation
    • Test with various text lengths
    • Test error handling
    • Requirements: 15.2

  • 19. Update TTS generation endpoint to accept edited scripts

    • Modify TTSRequest to accept optional segments array
    • Update generate_audiobook logic to use segments if provided
    • Fall back to text segmentation if no segments provided
    • Ensure backward compatibility with existing API calls
    • Requirements: 15.7

  • * 19.1 Write property test for edited script persistence

    • Property 35: Edited script persistence
    • Validates: Requirements 15.7

  • * 19.2 Write unit tests for script-based generation

    • Test generation with edited segments
    • Test fallback to text-based generation
    • Test segment speaker assignments are respected
    • Requirements: 15.7

  • 20. Implement script generation API client function

    • Add generateScript() function to src/api/tts.ts
    • Define ScriptSegment, ScriptRequest, and ScriptResponse TypeScript interfaces
    • Implement error handling for script generation
    • Update GenerateRequest interface to support segments
    • Requirements: 15.1, 15.2

  • * 20.1 Write unit tests for script API client

    • Test generateScript() function
    • Test error handling
    • Mock API responses
    • Requirements: 15.1

  • 21. Implement ScriptReviewSection component

    • Create ScriptReviewSection component
    • Display list of script segments with text boxes
    • Add speaker dropdown for each segment
    • Implement segment editing handlers
    • Add "Generate Audiobook" button
    • Add "Skip Review" button for quick generation
    • Style for readability and mobile responsiveness
    • Requirements: 15.3, 15.4, 15.5, 15.6, 15.8

  • * 21.1 Write property test for script segment editability

    • Property 34: Script segment editability
    • Validates: Requirements 15.5

  • * 21.2 Write unit tests for ScriptReviewSection

    • Test component renders segments correctly
    • Test speaker dropdown contains all roles
    • Test segment editing updates state
    • Test generate button is enabled
    • Test skip review button works
    • Requirements: 15.3, 15.4, 15.5, 15.8

  • 22. Update App component for script review workflow

    • Add 'review' phase to application state
    • Add scriptSegments state variable
    • Add isGeneratingScript loading state
    • Implement handleGenerateScript() method
    • Implement handleSkipReview() method
    • Implement handleSegmentChange() method
    • Update handleGenerate() to use edited segments
    • Wire up ScriptReviewSection component
    • Update phase transitions: input → assignment → review → generated
    • Requirements: 15.1, 15.3, 15.6, 15.7, 15.8

  • * 22.1 Write unit tests for script workflow

    • Test phase transitions include review phase
    • Test script generation is called after voice assignment
    • Test segment changes are tracked
    • Test skip review bypasses review phase
    • Requirements: 15.1, 15.8

  • 23. Update VoiceAssignmentSection to trigger script generation

    • Change "Generate Audiobook" button to "Next: Review Script"
    • Add "Skip Review & Generate" button as alternative
    • Update button handlers to call appropriate methods
    • Requirements: 15.1, 15.8

  • 24. Add script review documentation

    • Document script review feature in README
    • Add screenshots of script review interface
    • Explain how to edit speaker assignments
    • Document skip review option
    • Add troubleshooting for script generation issues
    • Requirements: 15.3, 15.4, 15.5, 15.8

  • 25. Final integration testing and polish

    • Test complete end-to-end workflow with script review
    • Test script generation with various text samples
    • Test editing speaker assignments in script
    • Test skip review functionality
    • Test with unmarked dialogue text (relying on AI detection)
    • Test with marked text (pattern detection)
    • Test with Ollama available and unavailable
    • Test all language options (a, b, f, j)
    • Test error scenarios (network failures, invalid inputs)
    • Verify audio quality and playback with edited scripts
    • Test Docker deployment
    • Fix any bugs discovered during testing
    • Requirements: All
Reference in New Issue View Git Blame Copy Permalink