audiobook/.kiro/specs/text-to-audiobook/requirements.md

Requirements Document

Introduction

The Text to Audiobook system is a production-ready React TypeScript single-page application that converts long-form text into multi-voice audio using Kokoro TTS (hexgrad/Kokoro-82M). The system automatically detects distinct roles/speakers in the input text, allows users to assign different voices to each role, and generates a complete audiobook with appropriate voice assignments. The application consists of a React frontend and a containerized backend service that handles TTS processing locally without relying on external cloud services.

Glossary

• Kokoro TTS: A text-to-speech engine using the hexgrad/Kokoro-82M model for high-quality voice synthesis
• Frontend: The React TypeScript single-page application that provides the user interface
• Backend: The containerized Node/Express or FastAPI service that processes TTS requests
• Role: A distinct speaker or character identified in the input text (e.g., Narrator, Alice, Bob)
• Voice ID: A Kokoro voice identifier (e.g., af_bella, af_sarah, am_adam) assigned to a role
• Language Code: A single-character code representing the language for synthesis (a=American English, b=British English, f=French, j=Japanese)
• KPipeline: The Kokoro library's pipeline object for text-to-speech synthesis
• Segment: A portion of text assigned to a specific role for synthesis
• Role Marker: Text patterns that identify speakers (e.g., "Alice:", "[Narrator]")
• Script: The complete breakdown of text into segments with speaker assignments, displayed for user review and editing

Requirements

Requirement 1

User Story: As a user, I want to input long-form text into the application, so that I can convert it into an audiobook.

Acceptance Criteria

1. WHEN the user navigates to the application THEN the Frontend SHALL display a large multiline text area for text input
2. WHEN the user enters text into the text area THEN the Frontend SHALL accept and store the full content without character limits
3. WHEN the text area is empty and the user attempts to proceed THEN the Frontend SHALL prevent progression and display a validation message
4. WHEN the user has entered text THEN the Frontend SHALL enable a "Next: Assign voices" button
5. THE Frontend SHALL maintain the entered text throughout the voice assignment and generation workflow

Requirement 2

User Story: As a user, I want the system to automatically detect distinct roles in my text, so that I can assign different voices to different speakers.

Acceptance Criteria

1. WHEN the user submits text for role detection THEN the Backend SHALL analyze the text to identify distinct speaker roles
2. WHEN the text contains lines starting with "RoleName:" THEN the Backend SHALL extract RoleName as a distinct role
3. WHEN the text contains patterns like "[RoleName]" preceding dialogue THEN the Backend SHALL extract RoleName as a distinct role
4. WHEN no clear role markers are detected in the text THEN the Backend SHALL return a single default role named "Narrator"
5. WHEN multiple instances of the same role appear THEN the Backend SHALL return each unique role exactly once
6. WHEN role detection completes THEN the Backend SHALL return a list of role objects containing id and label fields

Requirement 3

User Story: As a user, I want to assign a specific Kokoro voice to each detected role, so that different characters sound distinct in the audiobook.

Acceptance Criteria

1. WHEN roles are detected THEN the Frontend SHALL display a voice assignment section with the title "Assign voices to roles"
2. WHEN displaying each role THEN the Frontend SHALL show the role label and a dropdown selector
3. WHEN the user opens a voice dropdown THEN the Frontend SHALL populate it with available Kokoro voice IDs including af_bella, af_sarah, am_adam, and bf_emma
4. WHEN the user selects a voice for a role THEN the Frontend SHALL store the role-to-voice mapping
5. WHEN not all roles have assigned voices THEN the Frontend SHALL keep the "Generate Audiobook" button disabled
6. WHEN all roles have assigned voices THEN the Frontend SHALL enable the "Generate Audiobook" button

Requirement 4

User Story: As a user, I want to select the language for speech synthesis, so that I can generate audiobooks in different languages.

Acceptance Criteria

1. WHEN the voice assignment interface is displayed THEN the Frontend SHALL show a language selector
2. WHEN the user opens the language selector THEN the Frontend SHALL display options for American English (a), British English (b), French (f), and Japanese (j)
3. WHEN no language is explicitly selected THEN the Frontend SHALL default to American English (a)
4. WHEN the user selects a language THEN the Frontend SHALL store the selected language code for the generation request

Requirement 5

User Story: As a user, I want to control the playback speed of the generated audio, so that I can adjust the narration pace to my preference.

Acceptance Criteria

1. WHEN the voice assignment interface is displayed THEN the Frontend SHALL show a speed control slider
2. WHEN the user interacts with the speed slider THEN the Frontend SHALL allow selection of values between 0.5x and 1.5x
3. WHEN no speed is explicitly set THEN the Frontend SHALL default to 1.0x speed
4. WHEN the user adjusts the speed THEN the Frontend SHALL store the speed value for the generation request

Requirement 6

User Story: As a user, I want to generate the audiobook with my selected voices, so that I can listen to the multi-voice narration.

Acceptance Criteria

1. WHEN the user clicks "Generate Audiobook" THEN the Frontend SHALL send a request to the Backend with text, language code, role-to-voice mappings, and speed
2. WHEN the Backend receives a generation request THEN the Backend SHALL parse the text and assign each segment to its corresponding role based on role markers
3. WHEN a text segment matches a role marker THEN the Backend SHALL use the Kokoro voice assigned to that role for synthesis
4. WHEN a segment references a role not in the mapping THEN the Backend SHALL use a default narrator voice (af_bella)
5. WHEN synthesizing text THEN the Backend SHALL split long content into manageable chunks per paragraph or sentence
6. WHEN multiple chunks are synthesized THEN the Backend SHALL concatenate them into a single audio stream at 24 kHz
7. WHEN synthesis completes THEN the Backend SHALL stream the audio back with Content-Type audio/mpeg or audio/wav
8. WHEN streaming audio THEN the Backend SHALL include Content-Disposition header with filename "audiobook.mp3" or "audiobook.wav"

Requirement 7

User Story: As a user, I want to play and download the generated audiobook, so that I can listen to it immediately or save it for later.

Acceptance Criteria

1. WHEN the audiobook generation completes THEN the Frontend SHALL convert the response to a Blob and create an object URL
2. WHEN the audio is ready THEN the Frontend SHALL display an HTML5 audio player with the generated audio
3. WHEN the audio player is displayed THEN the Frontend SHALL show standard playback controls (play, pause, seek, volume)
4. WHEN the audio is ready THEN the Frontend SHALL display a "Download audiobook" button
5. WHEN the user clicks the download button THEN the Frontend SHALL save the file as "audiobook.mp3" or "audiobook.wav"

Requirement 8

User Story: As a user, I want clear feedback during processing, so that I understand the system status and can anticipate wait times.

Acceptance Criteria

1. WHEN the user initiates role detection THEN the Frontend SHALL display a loading state indicator
2. WHEN the user initiates audiobook generation THEN the Frontend SHALL display loading or progress feedback
3. WHEN processing long text THEN the Frontend SHALL display a message indicating "Generation may take several minutes for very long content"
4. WHEN an API request fails THEN the Frontend SHALL display a clear error message to the user
5. WHEN the Backend is processing THEN the Backend SHALL maintain responsive behavior without blocking

Requirement 9

User Story: As a developer, I want a well-defined API contract between frontend and backend, so that the services can communicate reliably.

Acceptance Criteria

1. THE Backend SHALL expose a GET endpoint at /api/health that returns status "ok"
2. THE Backend SHALL expose a POST endpoint at /api/tts/roles that accepts text and returns detected roles
3. WHEN /api/tts/roles receives a request THEN the Backend SHALL return JSON with a roles array containing id and label fields
4. THE Backend SHALL expose a POST endpoint at /api/tts that accepts text, lang, roles array, and optional speed
5. WHEN /api/tts receives a request THEN the Backend SHALL validate all required fields are present
6. THE Frontend SHALL use environment variable VITE_API_BASE_URL for the API base URL
7. THE Frontend SHALL implement TypeScript interfaces for Role, RoleWithVoice, and RolesResponse

Requirement 10

User Story: As a developer, I want the backend to use Kokoro TTS (hexgrad/Kokoro-82M) for synthesis, so that we have high-quality local voice generation.

Acceptance Criteria

1. THE Backend SHALL use the kokoro Python library with the hexgrad/Kokoro-82M model
2. THE Backend SHALL install dependencies including kokoro, soundfile, and espeak-ng
3. WHEN initializing TTS THEN the Backend SHALL load a KPipeline with the appropriate language code
4. WHEN synthesizing text THEN the Backend SHALL use the KPipeline to generate audio at 24 kHz
5. THE Backend SHALL support language codes a (American English), b (British English), f (French), and j (Japanese)
6. THE Backend SHALL support Kokoro voice IDs including af_bella, af_sarah, am_adam, and bf_emma

Requirement 11

User Story: As a developer, I want the backend containerized with Docker, so that it can be deployed consistently across environments.

Acceptance Criteria

1. THE Backend SHALL include a Dockerfile that installs Python and all dependencies
2. WHEN building the Docker image THEN the Dockerfile SHALL install kokoro, soundfile, espeak-ng, and model files
3. THE Dockerfile SHALL copy the backend code into the container
4. THE Dockerfile SHALL expose port 8000 or 3001
5. WHEN the container starts THEN the Dockerfile SHALL start the server using Uvicorn or equivalent
6. THE Backend MAY include a docker-compose.yml for simplified orchestration

Requirement 12

User Story: As a user, I want the interface to be responsive and usable on mobile devices, so that I can use the application on any device.

Acceptance Criteria

1. WHEN the application is viewed on mobile THEN the Frontend SHALL display all elements in a readable layout
2. WHEN the user interacts with controls on mobile THEN the Frontend SHALL respond appropriately to touch inputs
3. THE Frontend SHALL use responsive CSS or Tailwind to adapt to different screen sizes
4. WHEN text areas and dropdowns are displayed THEN the Frontend SHALL ensure they are appropriately sized for the viewport

Requirement 13

User Story: As a developer, I want comprehensive documentation, so that I can set up and run the application easily.

Acceptance Criteria

1. THE project SHALL include a README explaining how to download and configure Kokoro model files
2. THE README SHALL document how to run the backend including Docker commands
3. THE README SHALL document how to run the frontend with example commands
4. THE README SHALL include steps to test the full workflow from text input to audio generation
5. THE README MAY include screenshots or diagrams illustrating the application flow

Requirement 14

User Story: As a user, I want the system to intelligently detect speakers in unmarked text using AI, so that I can convert any dialogue or narrative without manually adding role markers.

Acceptance Criteria

1. WHEN the Backend receives text without explicit role markers THEN the Backend SHALL use Ollama with llama3:8B to analyze the text and identify distinct speakers
2. WHEN Ollama analyzes text THEN the Backend SHALL send a structured prompt requesting speaker identification in JSON format
3. WHEN Ollama returns speaker names THEN the Backend SHALL parse the JSON response and create role objects
4. WHEN Ollama is unavailable or fails THEN the Backend SHALL fall back to pattern-based role detection
5. WHEN both AI detection and pattern detection find roles THEN the Backend SHALL merge and deduplicate the results
6. THE Backend SHALL configure Ollama connection using environment variable OLLAMA_BASE_URL with default http://localhost:11434

Requirement 15

User Story: As a user, I want to review and edit the AI-generated script showing which character speaks each line, so that I can correct any mistakes before generating the audiobook.

Acceptance Criteria

1. WHEN the user completes voice assignment THEN the Frontend SHALL request a script preview from the Backend before generation
2. WHEN the Backend receives a script preview request THEN the Backend SHALL use AI to segment the text and return a list of segments with speaker assignments
3. WHEN the script preview is received THEN the Frontend SHALL display a new "Review Script" phase showing each segment with its assigned speaker
4. WHEN displaying script segments THEN the Frontend SHALL show each segment in a text box with a dropdown to select the speaker
5. WHEN the user changes a speaker assignment for a segment THEN the Frontend SHALL update the segment mapping
6. WHEN the user is satisfied with the script THEN the Frontend SHALL enable a "Generate Audiobook" button
7. WHEN the user clicks "Generate Audiobook" THEN the Frontend SHALL send the edited script segments to the Backend for synthesis
8. THE Frontend SHALL allow users to skip the review step with a "Skip Review" option that proceeds directly to generation