Text to Audiobook

A full-stack application that converts long-form text into multi-voice audiobooks using Kokoro TTS (hexgrad/Kokoro-82M). The system automatically detects distinct roles/speakers in text, allows voice assignment per role, and generates high-quality audio.

Features

• 🎭 Automatic role/speaker detection from text
• 🎤 Multi-voice synthesis with Kokoro TTS
• 🌍 Support for multiple languages (English, French, Japanese)
• ⚡ Fast local processing (no cloud dependencies)
• 📱 Responsive web interface
• 🎵 Audio playback and download

Architecture

• Frontend: React + TypeScript + Vite + Tailwind CSS
• Backend: Python + FastAPI + Kokoro TTS
• Deployment: Docker containerization

Table of Contents

• Quick Start
• Kokoro Model Setup
• Backend Setup
  • Local Development
  • Docker Deployment
• Frontend Setup
• Usage Workflow
• Testing the Application
• Troubleshooting
• Project Structure
• API Documentation

Quick Start

The fastest way to get started is using Docker:

# 1. Start the backend
cd backend
docker-compose up -d

# 2. Start the frontend (in a new terminal)
cd frontend
npm install
npm run dev

# 3. Open http://localhost:5173 in your browser

Kokoro Model Setup

The Kokoro TTS model (hexgrad/Kokoro-82M) is automatically downloaded when you first run the backend. The model files are cached by the kokoro Python library.

Model Configuration

Supported Languages:

• a - American English
• b - British English
• f - French
• j - Japanese

Available Voices:

• af_bella - Bella (Female, American)
• af_sarah - Sarah (Female, American)
• am_adam - Adam (Male, American)
• am_michael - Michael (Male, American)
• bf_emma - Emma (Female, British)
• bf_isabella - Isabella (Female, British)
• bm_george - George (Male, British)
• bm_lewis - Lewis (Male, British)

Model Storage

The Kokoro library automatically downloads and caches model files in:

• Linux/macOS: ~/.cache/kokoro/
• Windows: %USERPROFILE%\.cache\kokoro\

The first synthesis request will trigger model download (~500MB). Subsequent requests will use the cached model.

Backend Setup

Prerequisites

• Python 3.10 or higher
• espeak-ng (system dependency for grapheme-to-phoneme conversion)
• Docker (optional, for containerized deployment)

Local Development

1. Install espeak-ng

macOS:

brew install espeak-ng

Linux (Ubuntu/Debian):

sudo apt-get update
sudo apt-get install espeak-ng

Linux (Fedora/RHEL):

sudo dnf install espeak-ng

Windows: Download and install from espeak-ng releases

2. Set up Python environment

cd backend

# Create virtual environment
python -m venv venv

# Activate virtual environment
# macOS/Linux:
source venv/bin/activate
# Windows:
venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

3. Configure environment variables

cp .env.example .env

Edit .env if needed:

PORT=8000
LOG_LEVEL=info

4. Run the backend server

uvicorn main:app --reload --host 0.0.0.0 --port 8000

The API will be available at:

• API Base: http://localhost:8000
• API Docs: http://localhost:8000/docs
• Health Check: http://localhost:8000/api/health

Docker Deployment

Option 1: Using Docker Compose (Recommended)

cd backend

# Build and start the container
docker-compose up -d

# View logs
docker-compose logs -f

# Stop the container
docker-compose down

Option 2: Using Docker directly

cd backend

# Build the image
docker build -t text-to-audiobook-backend .

# Run the container
docker run -d \
  --name text-to-audiobook-backend \
  -p 8000:8000 \
  -e PORT=8000 \
  -e LOG_LEVEL=info \
  text-to-audiobook-backend

# View logs
docker logs -f text-to-audiobook-backend

# Stop the container
docker stop text-to-audiobook-backend
docker rm text-to-audiobook-backend

Docker Configuration

The Docker setup includes:

• Python 3.10 slim base image
• Automatic installation of espeak-ng
• Health check endpoint monitoring
• Automatic restart on failure
• Volume mounting for development (optional)

Resource Requirements:

• Memory: Minimum 2GB RAM (4GB recommended)
• Storage: ~500MB for Kokoro models
• CPU: Multi-core recommended for faster synthesis

Frontend Setup

Prerequisites

• Node.js 20.19+ or 22.12+
• npm or yarn

Installation

cd frontend

# Install dependencies
npm install

# Create environment file
cp .env.example .env

Configuration

Edit frontend/.env:

VITE_API_BASE_URL=http://localhost:8000

For production, update this to your backend URL.

Development

npm run dev

The application will be available at http://localhost:5173

Production Build

# Build for production
npm run build

# Preview production build
npm run preview

The built files will be in frontend/dist/ and can be served by any static file server.

Usage Workflow

1. Enter Text

Paste your text into the text area. Use role markers to identify different speakers:

Colon Pattern:

Narrator: Once upon a time, in a land far away...
Alice: Hello! How are you today?
Bob: I'm doing great, thanks for asking!

Bracket Pattern:

[Narrator] The story begins on a cold winter morning.
[Alice] I can't believe it's snowing again!
[Bob] I know, it's beautiful though.

Mixed Patterns:

Narrator: The two friends met at the park.
Alice: [excitedly] Did you hear the news?
Bob: No, what happened?

2. Assign Voices

• The system automatically detects roles from your text
• Select a voice for each detected role from the dropdown
• Choose the language (American English, British English, French, or Japanese)
• Adjust playback speed (0.5x to 1.5x)

3. Generate Audiobook

• Click "Generate Audiobook" (enabled when all roles have voices)
• Wait for synthesis to complete (may take several minutes for long texts)
• Progress indicator shows generation status

4. Listen & Download

• Use the audio player to listen to your audiobook
• Click "Download audiobook" to save the file
• File is saved as audiobook.wav

Testing the Application

1. Health Check

Verify the backend is running:

curl http://localhost:8000/api/health
# Expected: {"status":"ok"}

2. Role Detection Test

Test role detection with sample text:

curl -X POST http://localhost:8000/api/tts/roles \
  -H "Content-Type: application/json" \
  -d '{"text":"Alice: Hello there!\nBob: Hi Alice!"}'

Expected response:

{
  "roles": [
    {"id": "Alice", "label": "Alice"},
    {"id": "Bob", "label": "Bob"}
  ]
}

3. End-to-End Workflow Test

1. Open http://localhost:5173
2. Enter sample text:

   Narrator: This is a test of the audiobook system.
   Alice: Hello, this is Alice speaking.
   Bob: And this is Bob.
   

3. Click "Next: Assign voices"
4. Assign different voices to each role
5. Click "Generate Audiobook"
6. Verify audio plays correctly
7. Test download functionality

4. Test Different Languages

Try generating audiobooks in different languages:

• American English: Default, works with all af_* and am_* voices
• British English: Use with bf_* and bm_* voices
• French: Test with French text
• Japanese: Test with Japanese text

5. Test Edge Cases

• Empty text: Should show validation error
• No role markers: Should detect single "Narrator" role
• Very long text: Should show warning message
• Special characters: Should handle gracefully

Running Automated Tests

Backend Tests:

cd backend
source venv/bin/activate  # Activate virtual environment
pytest test_*.py -v

Frontend Tests:

cd frontend
npm test

Troubleshooting

Backend Issues

Issue: "espeak-ng not found"

Solution:

# macOS
brew install espeak-ng

# Linux
sudo apt-get install espeak-ng

# Verify installation
espeak-ng --version

Issue: "Module 'kokoro' not found"

Solution:

cd backend
source venv/bin/activate
pip install -r requirements.txt

Issue: "Model download fails"

Symptoms: First synthesis request times out or fails

Solution:

• Ensure stable internet connection
• Check firewall settings
• Manually download models:

  python -c "from kokoro import KPipeline; KPipeline(lang_code='a')"
  

• Check disk space (~500MB required)

Issue: "Out of memory during synthesis"

Solution:

• Reduce text length (split into smaller chunks)
• Increase Docker memory limit:

  docker run -m 4g ...  # Allocate 4GB RAM
  

• Close other applications

Issue: "Port 8000 already in use"

Solution:

# Find process using port 8000
lsof -i :8000  # macOS/Linux
netstat -ano | findstr :8000  # Windows

# Kill the process or use different port
uvicorn main:app --port 8001

Frontend Issues

Issue: "Cannot connect to backend"

Solution:

1. Verify backend is running: curl http://localhost:8000/api/health
2. Check VITE_API_BASE_URL in frontend/.env
3. Check CORS settings in backend
4. Verify no firewall blocking

Issue: "Audio won't play"

Solution:

• Check browser console for errors
• Verify audio format is supported (WAV should work in all browsers)
• Try different browser
• Check browser audio permissions

Issue: "Download doesn't work"

Solution:

• Check browser download settings
• Verify popup blocker isn't blocking download
• Check browser console for errors
• Try right-click → "Save As" on audio player

Docker Issues

Issue: "Docker build fails"

Solution:

# Clear Docker cache
docker system prune -a

# Rebuild without cache
docker-compose build --no-cache

Issue: "Container exits immediately"

Solution:

# Check logs
docker-compose logs backend

# Common causes:
# - Missing dependencies
# - Port conflict
# - Syntax errors in code

Issue: "Health check failing"

Solution:

# Check if service is actually running
docker exec -it text-to-audiobook-backend curl http://localhost:8000/api/health

# Check logs for startup errors
docker-compose logs backend

Performance Issues

Issue: "Generation is very slow"

Causes & Solutions:

• Long text: Split into smaller sections
• First run: Model download takes time (one-time)
• Limited CPU: Use faster machine or reduce text length
• Docker on macOS: Docker Desktop has performance overhead

Issue: "High memory usage"

Solution:

• Restart backend service to clear cache
• Process shorter texts
• Increase system RAM
• Monitor with: docker stats (Docker) or top/htop (local)

Common Error Messages

Error	Cause	Solution
ValidationError: Missing required field	Invalid API request	Check request format matches API spec
Language code 'x' not supported	Invalid language code	Use: a, b, f, or j
Voice ID 'x' not found	Invalid voice ID	Check available voices list
Text is required	Empty text submitted	Enter text before proceeding
CORS error	Frontend can't reach backend	Check CORS settings and URLs

Getting Help

If you encounter issues not covered here:

1. Check the logs:

   • Backend: docker-compose logs backend or console output
   • Frontend: Browser developer console (F12)

2. Verify versions:

   python --version  # Should be 3.10+
   node --version    # Should be 20.19+ or 22.12+
   espeak-ng --version
   

3. Check the GitHub Issues

4. Review the design document: .kiro/specs/text-to-audiobook/design.md

Project Structure

.
├── backend/                    # FastAPI backend
│   ├── services/
│   │   ├── role_detector.py   # Role detection logic
│   │   └── tts_service.py     # TTS synthesis logic
│   ├── main.py                # API endpoints
│   ├── requirements.txt       # Python dependencies
│   ├── Dockerfile             # Docker configuration
│   ├── docker-compose.yml     # Docker Compose setup
│   ├── .env.example           # Environment template
│   ├── test_*.py              # Test files
│   └── README.md              # Backend documentation
├── frontend/                   # React frontend
│   ├── src/
│   │   ├── api/
│   │   │   └── tts.ts         # API client
│   │   ├── components/
│   │   │   ├── TextInputSection.tsx
│   │   │   ├── VoiceAssignmentSection.tsx
│   │   │   └── AudioPlayerSection.tsx
│   │   ├── App.tsx            # Main application
│   │   ├── main.tsx           # Entry point
│   │   └── index.css          # Global styles
│   ├── package.json           # Node dependencies
│   ├── vite.config.ts         # Vite configuration
│   ├── .env.example           # Environment template
│   └── README.md              # Frontend documentation
├── .kiro/specs/               # Specification documents
│   └── text-to-audiobook/
│       ├── requirements.md    # Requirements
│       ├── design.md          # Design document
│       └── tasks.md           # Implementation tasks
└── README.md                  # This file

API Documentation

Endpoints

GET /api/health

Health check endpoint.

Response:

{
  "status": "ok"
}

POST /api/tts/roles

Detect roles in text.

Request:

{
  "text": "Alice: Hello!\nBob: Hi there!"
}

Response:

{
  "roles": [
    {"id": "Alice", "label": "Alice"},
    {"id": "Bob", "label": "Bob"}
  ]
}

POST /api/tts

Generate audiobook.

Request:

{
  "text": "Alice: Hello!\nBob: Hi there!",
  "lang": "a",
  "roles": [
    {"id": "Alice", "voice": "af_bella"},
    {"id": "Bob", "voice": "am_adam"}
  ],
  "speed": 1.0
}

Response:

• Content-Type: audio/wav
• Content-Disposition: attachment; filename="audiobook.wav"
• Body: Audio file bytes

Interactive API Documentation

Visit http://localhost:8000/docs for interactive Swagger UI documentation.

Additional Resources

• Backend Documentation
• Frontend Documentation
• Requirements Document
• Design Document
• Kokoro TTS Library

License

MIT