Task 1: Set up project structure and dependencies
- Created backend directory with FastAPI project structure - Created frontend directory with React + TypeScript + Vite - Configured Python virtual environment and installed dependencies (fastapi, uvicorn, kokoro, soundfile, python-multipart) - Verified espeak-ng system dependency is installed - Configured Node.js project with TypeScript, React, and Vite - Set up Tailwind CSS for styling - Set up environment variable configuration for both projects - Created comprehensive README documentation Requirements: 9.6, 10.2, 11.1
This commit is contained in:
181
.kiro/specs/text-to-audiobook/requirements.md
Normal file
181
.kiro/specs/text-to-audiobook/requirements.md
Normal file
@@ -0,0 +1,181 @@
|
||||
# Requirements Document
|
||||
|
||||
## Introduction
|
||||
|
||||
The Text to Audiobook system is a production-ready React TypeScript single-page application that converts long-form text into multi-voice audio using Kokoro TTS (hexgrad/Kokoro-82M). The system automatically detects distinct roles/speakers in the input text, allows users to assign different voices to each role, and generates a complete audiobook with appropriate voice assignments. The application consists of a React frontend and a containerized backend service that handles TTS processing locally without relying on external cloud services.
|
||||
|
||||
## Glossary
|
||||
|
||||
- **Kokoro TTS**: A text-to-speech engine using the hexgrad/Kokoro-82M model for high-quality voice synthesis
|
||||
- **Frontend**: The React TypeScript single-page application that provides the user interface
|
||||
- **Backend**: The containerized Node/Express or FastAPI service that processes TTS requests
|
||||
- **Role**: A distinct speaker or character identified in the input text (e.g., Narrator, Alice, Bob)
|
||||
- **Voice ID**: A Kokoro voice identifier (e.g., af_bella, af_sarah, am_adam) assigned to a role
|
||||
- **Language Code**: A single-character code representing the language for synthesis (a=American English, b=British English, f=French, j=Japanese)
|
||||
- **KPipeline**: The Kokoro library's pipeline object for text-to-speech synthesis
|
||||
- **Segment**: A portion of text assigned to a specific role for synthesis
|
||||
- **Role Marker**: Text patterns that identify speakers (e.g., "Alice:", "[Narrator]")
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement 1
|
||||
|
||||
**User Story:** As a user, I want to input long-form text into the application, so that I can convert it into an audiobook.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the user navigates to the application THEN the Frontend SHALL display a large multiline text area for text input
|
||||
2. WHEN the user enters text into the text area THEN the Frontend SHALL accept and store the full content without character limits
|
||||
3. WHEN the text area is empty and the user attempts to proceed THEN the Frontend SHALL prevent progression and display a validation message
|
||||
4. WHEN the user has entered text THEN the Frontend SHALL enable a "Next: Assign voices" button
|
||||
5. THE Frontend SHALL maintain the entered text throughout the voice assignment and generation workflow
|
||||
|
||||
### Requirement 2
|
||||
|
||||
**User Story:** As a user, I want the system to automatically detect distinct roles in my text, so that I can assign different voices to different speakers.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the user submits text for role detection THEN the Backend SHALL analyze the text to identify distinct speaker roles
|
||||
2. WHEN the text contains lines starting with "RoleName:" THEN the Backend SHALL extract RoleName as a distinct role
|
||||
3. WHEN the text contains patterns like "[RoleName]" preceding dialogue THEN the Backend SHALL extract RoleName as a distinct role
|
||||
4. WHEN no clear role markers are detected in the text THEN the Backend SHALL return a single default role named "Narrator"
|
||||
5. WHEN multiple instances of the same role appear THEN the Backend SHALL return each unique role exactly once
|
||||
6. WHEN role detection completes THEN the Backend SHALL return a list of role objects containing id and label fields
|
||||
|
||||
### Requirement 3
|
||||
|
||||
**User Story:** As a user, I want to assign a specific Kokoro voice to each detected role, so that different characters sound distinct in the audiobook.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN roles are detected THEN the Frontend SHALL display a voice assignment section with the title "Assign voices to roles"
|
||||
2. WHEN displaying each role THEN the Frontend SHALL show the role label and a dropdown selector
|
||||
3. WHEN the user opens a voice dropdown THEN the Frontend SHALL populate it with available Kokoro voice IDs including af_bella, af_sarah, am_adam, and bf_emma
|
||||
4. WHEN the user selects a voice for a role THEN the Frontend SHALL store the role-to-voice mapping
|
||||
5. WHEN not all roles have assigned voices THEN the Frontend SHALL keep the "Generate Audiobook" button disabled
|
||||
6. WHEN all roles have assigned voices THEN the Frontend SHALL enable the "Generate Audiobook" button
|
||||
|
||||
### Requirement 4
|
||||
|
||||
**User Story:** As a user, I want to select the language for speech synthesis, so that I can generate audiobooks in different languages.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the voice assignment interface is displayed THEN the Frontend SHALL show a language selector
|
||||
2. WHEN the user opens the language selector THEN the Frontend SHALL display options for American English (a), British English (b), French (f), and Japanese (j)
|
||||
3. WHEN no language is explicitly selected THEN the Frontend SHALL default to American English (a)
|
||||
4. WHEN the user selects a language THEN the Frontend SHALL store the selected language code for the generation request
|
||||
|
||||
### Requirement 5
|
||||
|
||||
**User Story:** As a user, I want to control the playback speed of the generated audio, so that I can adjust the narration pace to my preference.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the voice assignment interface is displayed THEN the Frontend SHALL show a speed control slider
|
||||
2. WHEN the user interacts with the speed slider THEN the Frontend SHALL allow selection of values between 0.5x and 1.5x
|
||||
3. WHEN no speed is explicitly set THEN the Frontend SHALL default to 1.0x speed
|
||||
4. WHEN the user adjusts the speed THEN the Frontend SHALL store the speed value for the generation request
|
||||
|
||||
### Requirement 6
|
||||
|
||||
**User Story:** As a user, I want to generate the audiobook with my selected voices, so that I can listen to the multi-voice narration.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the user clicks "Generate Audiobook" THEN the Frontend SHALL send a request to the Backend with text, language code, role-to-voice mappings, and speed
|
||||
2. WHEN the Backend receives a generation request THEN the Backend SHALL parse the text and assign each segment to its corresponding role based on role markers
|
||||
3. WHEN a text segment matches a role marker THEN the Backend SHALL use the Kokoro voice assigned to that role for synthesis
|
||||
4. WHEN a segment references a role not in the mapping THEN the Backend SHALL use a default narrator voice (af_bella)
|
||||
5. WHEN synthesizing text THEN the Backend SHALL split long content into manageable chunks per paragraph or sentence
|
||||
6. WHEN multiple chunks are synthesized THEN the Backend SHALL concatenate them into a single audio stream at 24 kHz
|
||||
7. WHEN synthesis completes THEN the Backend SHALL stream the audio back with Content-Type audio/mpeg or audio/wav
|
||||
8. WHEN streaming audio THEN the Backend SHALL include Content-Disposition header with filename "audiobook.mp3" or "audiobook.wav"
|
||||
|
||||
### Requirement 7
|
||||
|
||||
**User Story:** As a user, I want to play and download the generated audiobook, so that I can listen to it immediately or save it for later.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the audiobook generation completes THEN the Frontend SHALL convert the response to a Blob and create an object URL
|
||||
2. WHEN the audio is ready THEN the Frontend SHALL display an HTML5 audio player with the generated audio
|
||||
3. WHEN the audio player is displayed THEN the Frontend SHALL show standard playback controls (play, pause, seek, volume)
|
||||
4. WHEN the audio is ready THEN the Frontend SHALL display a "Download audiobook" button
|
||||
5. WHEN the user clicks the download button THEN the Frontend SHALL save the file as "audiobook.mp3" or "audiobook.wav"
|
||||
|
||||
### Requirement 8
|
||||
|
||||
**User Story:** As a user, I want clear feedback during processing, so that I understand the system status and can anticipate wait times.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the user initiates role detection THEN the Frontend SHALL display a loading state indicator
|
||||
2. WHEN the user initiates audiobook generation THEN the Frontend SHALL display loading or progress feedback
|
||||
3. WHEN processing long text THEN the Frontend SHALL display a message indicating "Generation may take several minutes for very long content"
|
||||
4. WHEN an API request fails THEN the Frontend SHALL display a clear error message to the user
|
||||
5. WHEN the Backend is processing THEN the Backend SHALL maintain responsive behavior without blocking
|
||||
|
||||
### Requirement 9
|
||||
|
||||
**User Story:** As a developer, I want a well-defined API contract between frontend and backend, so that the services can communicate reliably.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Backend SHALL expose a GET endpoint at /api/health that returns status "ok"
|
||||
2. THE Backend SHALL expose a POST endpoint at /api/tts/roles that accepts text and returns detected roles
|
||||
3. WHEN /api/tts/roles receives a request THEN the Backend SHALL return JSON with a roles array containing id and label fields
|
||||
4. THE Backend SHALL expose a POST endpoint at /api/tts that accepts text, lang, roles array, and optional speed
|
||||
5. WHEN /api/tts receives a request THEN the Backend SHALL validate all required fields are present
|
||||
6. THE Frontend SHALL use environment variable VITE_API_BASE_URL for the API base URL
|
||||
7. THE Frontend SHALL implement TypeScript interfaces for Role, RoleWithVoice, and RolesResponse
|
||||
|
||||
### Requirement 10
|
||||
|
||||
**User Story:** As a developer, I want the backend to use Kokoro TTS (hexgrad/Kokoro-82M) for synthesis, so that we have high-quality local voice generation.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Backend SHALL use the kokoro Python library with the hexgrad/Kokoro-82M model
|
||||
2. THE Backend SHALL install dependencies including kokoro, soundfile, and espeak-ng
|
||||
3. WHEN initializing TTS THEN the Backend SHALL load a KPipeline with the appropriate language code
|
||||
4. WHEN synthesizing text THEN the Backend SHALL use the KPipeline to generate audio at 24 kHz
|
||||
5. THE Backend SHALL support language codes a (American English), b (British English), f (French), and j (Japanese)
|
||||
6. THE Backend SHALL support Kokoro voice IDs including af_bella, af_sarah, am_adam, and bf_emma
|
||||
|
||||
### Requirement 11
|
||||
|
||||
**User Story:** As a developer, I want the backend containerized with Docker, so that it can be deployed consistently across environments.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Backend SHALL include a Dockerfile that installs Python and all dependencies
|
||||
2. WHEN building the Docker image THEN the Dockerfile SHALL install kokoro, soundfile, espeak-ng, and model files
|
||||
3. THE Dockerfile SHALL copy the backend code into the container
|
||||
4. THE Dockerfile SHALL expose port 8000 or 3001
|
||||
5. WHEN the container starts THEN the Dockerfile SHALL start the server using Uvicorn or equivalent
|
||||
6. THE Backend MAY include a docker-compose.yml for simplified orchestration
|
||||
|
||||
### Requirement 12
|
||||
|
||||
**User Story:** As a user, I want the interface to be responsive and usable on mobile devices, so that I can use the application on any device.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN the application is viewed on mobile THEN the Frontend SHALL display all elements in a readable layout
|
||||
2. WHEN the user interacts with controls on mobile THEN the Frontend SHALL respond appropriately to touch inputs
|
||||
3. THE Frontend SHALL use responsive CSS or Tailwind to adapt to different screen sizes
|
||||
4. WHEN text areas and dropdowns are displayed THEN the Frontend SHALL ensure they are appropriately sized for the viewport
|
||||
|
||||
### Requirement 13
|
||||
|
||||
**User Story:** As a developer, I want comprehensive documentation, so that I can set up and run the application easily.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE project SHALL include a README explaining how to download and configure Kokoro model files
|
||||
2. THE README SHALL document how to run the backend including Docker commands
|
||||
3. THE README SHALL document how to run the frontend with example commands
|
||||
4. THE README SHALL include steps to test the full workflow from text input to audio generation
|
||||
5. THE README MAY include screenshots or diagrams illustrating the application flow
|
||||
Reference in New Issue
Block a user