audiobook/README.md

# Text to Audiobook

A full-stack application that converts long-form text into multi-voice audiobooks using Kokoro TTS (hexgrad/Kokoro-82M). The system automatically detects distinct roles/speakers in text, allows voice assignment per role, and generates high-quality audio.

## Features

- 🎭 Automatic role/speaker detection from text
- 🎤 Multi-voice synthesis with Kokoro TTS
- 🌍 Support for multiple languages (English, French, Japanese)
- ⚡ Fast local processing (no cloud dependencies)
- 📱 Responsive web interface
- 🎵 Audio playback and download

## Architecture

- **Frontend**: React + TypeScript + Vite + Tailwind CSS
- **Backend**: Python + FastAPI + Kokoro TTS
- **Deployment**: Docker containerization

## Table of Contents

- [Quick Start](#quick-start)
- [Kokoro Model Setup](#kokoro-model-setup)
- [Backend Setup](#backend-setup)
  - [Local Development](#local-development)
  - [Docker Deployment](#docker-deployment)
- [Frontend Setup](#frontend-setup)
- [Usage Workflow](#usage-workflow)
- [Testing the Application](#testing-the-application)
- [Troubleshooting](#troubleshooting)
- [Project Structure](#project-structure)
- [API Documentation](#api-documentation)

## Quick Start

The fastest way to get started is using Docker:

```bash
# 1. Start the backend
cd backend
docker-compose up -d

# 2. Start the frontend (in a new terminal)
cd frontend
npm install
npm run dev

# 3. Open http://localhost:5173 in your browser
```

## Kokoro Model Setup

The Kokoro TTS model (hexgrad/Kokoro-82M) is automatically downloaded when you first run the backend. The model files are cached by the `kokoro` Python library.

### Model Configuration

**Supported Languages:**
- `a` - American English
- `b` - British English
- `f` - French
- `j` - Japanese

**Available Voices:**
- `af_bella` - Bella (Female, American)
- `af_sarah` - Sarah (Female, American)
- `am_adam` - Adam (Male, American)
- `am_michael` - Michael (Male, American)
- `bf_emma` - Emma (Female, British)
- `bf_isabella` - Isabella (Female, British)
- `bm_george` - George (Male, British)
- `bm_lewis` - Lewis (Male, British)

### Model Storage

The Kokoro library automatically downloads and caches model files in:
- **Linux/macOS**: `~/.cache/kokoro/`
- **Windows**: `%USERPROFILE%\.cache\kokoro\`

The first synthesis request will trigger model download (~500MB). Subsequent requests will use the cached model.

## Backend Setup

### Prerequisites

- Python 3.10 or higher
- espeak-ng (system dependency for grapheme-to-phoneme conversion)
- Docker (optional, for containerized deployment)

### Local Development

#### 1. Install espeak-ng

**macOS:**
```bash
brew install espeak-ng
```

**Linux (Ubuntu/Debian):**
```bash
sudo apt-get update
sudo apt-get install espeak-ng
```

**Linux (Fedora/RHEL):**
```bash
sudo dnf install espeak-ng
```

**Windows:**
Download and install from [espeak-ng releases](https://github.com/espeak-ng/espeak-ng/releases)

#### 2. Set up Python environment

```bash
cd backend

# Create virtual environment
python -m venv venv

# Activate virtual environment
# macOS/Linux:
source venv/bin/activate
# Windows:
venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt
```

#### 3. Configure environment variables

```bash
cp .env.example .env
```

Edit `.env` if needed:
```bash
PORT=8000
LOG_LEVEL=info
```

#### 4. Run the backend server

```bash
uvicorn main:app --reload --host 0.0.0.0 --port 8000
```

The API will be available at:
- **API Base**: http://localhost:8000
- **API Docs**: http://localhost:8000/docs
- **Health Check**: http://localhost:8000/api/health

### Docker Deployment

#### Option 1: Using Docker Compose (Recommended)

```bash
cd backend

# Build and start the container
docker-compose up -d

# View logs
docker-compose logs -f

# Stop the container
docker-compose down
```

#### Option 2: Using Docker directly

```bash
cd backend

# Build the image
docker build -t text-to-audiobook-backend .

# Run the container
docker run -d \
  --name text-to-audiobook-backend \
  -p 8000:8000 \
  -e PORT=8000 \
  -e LOG_LEVEL=info \
  text-to-audiobook-backend

# View logs
docker logs -f text-to-audiobook-backend

# Stop the container
docker stop text-to-audiobook-backend
docker rm text-to-audiobook-backend
```

#### Docker Configuration

The Docker setup includes:
- Python 3.10 slim base image
- Automatic installation of espeak-ng
- Health check endpoint monitoring
- Automatic restart on failure
- Volume mounting for development (optional)

**Resource Requirements:**
- **Memory**: Minimum 2GB RAM (4GB recommended)
- **Storage**: ~500MB for Kokoro models
- **CPU**: Multi-core recommended for faster synthesis

## Frontend Setup

### Prerequisites

- Node.js 20.19+ or 22.12+
- npm or yarn

### Installation

```bash
cd frontend

# Install dependencies
npm install

# Create environment file
cp .env.example .env
```

### Configuration

Edit `frontend/.env`:
```bash
VITE_API_BASE_URL=http://localhost:8000
```

For production, update this to your backend URL.

### Development

```bash
npm run dev
```

The application will be available at http://localhost:5173

### Production Build

```bash
# Build for production
npm run build

# Preview production build
npm run preview
```

The built files will be in `frontend/dist/` and can be served by any static file server.

## Usage Workflow

### 1. Enter Text

Paste your text into the text area. Use role markers to identify different speakers:

**Colon Pattern:**
```
Narrator: Once upon a time, in a land far away...
Alice: Hello! How are you today?
Bob: I'm doing great, thanks for asking!
```

**Bracket Pattern:**
```
[Narrator] The story begins on a cold winter morning.
[Alice] I can't believe it's snowing again!
[Bob] I know, it's beautiful though.
```

**Mixed Patterns:**
```
Narrator: The two friends met at the park.
Alice: [excitedly] Did you hear the news?
Bob: No, what happened?
```

### 2. Assign Voices

- The system automatically detects roles from your text
- Select a voice for each detected role from the dropdown
- Choose the language (American English, British English, French, or Japanese)
- Adjust playback speed (0.5x to 1.5x)

### 3. Generate Audiobook

- Click "Generate Audiobook" (enabled when all roles have voices)
- Wait for synthesis to complete (may take several minutes for long texts)
- Progress indicator shows generation status

### 4. Listen & Download

- Use the audio player to listen to your audiobook
- Click "Download audiobook" to save the file
- File is saved as `audiobook.wav`

## Testing the Application

### 1. Health Check

Verify the backend is running:
```bash
curl http://localhost:8000/api/health
# Expected: {"status":"ok"}
```

### 2. Role Detection Test

Test role detection with sample text:
```bash
curl -X POST http://localhost:8000/api/tts/roles \
  -H "Content-Type: application/json" \
  -d '{"text":"Alice: Hello there!\nBob: Hi Alice!"}'
```

Expected response:
```json
{
  "roles": [
    {"id": "Alice", "label": "Alice"},
    {"id": "Bob", "label": "Bob"}
  ]
}
```

### 3. End-to-End Workflow Test

1. Open http://localhost:5173
2. Enter sample text:
   ```
   Narrator: This is a test of the audiobook system.
   Alice: Hello, this is Alice speaking.
   Bob: And this is Bob.
   ```
3. Click "Next: Assign voices"
4. Assign different voices to each role
5. Click "Generate Audiobook"
6. Verify audio plays correctly
7. Test download functionality

### 4. Test Different Languages

Try generating audiobooks in different languages:
- **American English**: Default, works with all `af_*` and `am_*` voices
- **British English**: Use with `bf_*` and `bm_*` voices
- **French**: Test with French text
- **Japanese**: Test with Japanese text

### 5. Test Edge Cases

- **Empty text**: Should show validation error
- **No role markers**: Should detect single "Narrator" role
- **Very long text**: Should show warning message
- **Special characters**: Should handle gracefully

### Running Automated Tests

**Backend Tests:**
```bash
cd backend
source venv/bin/activate  # Activate virtual environment
pytest test_*.py -v
```

**Frontend Tests:**
```bash
cd frontend
npm test
```

## Troubleshooting

### Backend Issues

#### Issue: "espeak-ng not found"

**Solution:**
```bash
# macOS
brew install espeak-ng

# Linux
sudo apt-get install espeak-ng

# Verify installation
espeak-ng --version
```

#### Issue: "Module 'kokoro' not found"

**Solution:**
```bash
cd backend
source venv/bin/activate
pip install -r requirements.txt
```

#### Issue: "Model download fails"

**Symptoms:** First synthesis request times out or fails

**Solution:**
- Ensure stable internet connection
- Check firewall settings
- Manually download models:
  ```bash
  python -c "from kokoro import KPipeline; KPipeline(lang_code='a')"
  ```
- Check disk space (~500MB required)

#### Issue: "Out of memory during synthesis"

**Solution:**
- Reduce text length (split into smaller chunks)
- Increase Docker memory limit:
  ```bash
  docker run -m 4g ...  # Allocate 4GB RAM
  ```
- Close other applications

#### Issue: "Port 8000 already in use"

**Solution:**
```bash
# Find process using port 8000
lsof -i :8000  # macOS/Linux
netstat -ano | findstr :8000  # Windows

# Kill the process or use different port
uvicorn main:app --port 8001
```

### Frontend Issues

#### Issue: "Cannot connect to backend"

**Solution:**
1. Verify backend is running: `curl http://localhost:8000/api/health`
2. Check `VITE_API_BASE_URL` in `frontend/.env`
3. Check CORS settings in backend
4. Verify no firewall blocking

#### Issue: "Audio won't play"

**Solution:**
- Check browser console for errors
- Verify audio format is supported (WAV should work in all browsers)
- Try different browser
- Check browser audio permissions

#### Issue: "Download doesn't work"

**Solution:**
- Check browser download settings
- Verify popup blocker isn't blocking download
- Check browser console for errors
- Try right-click → "Save As" on audio player

### Docker Issues

#### Issue: "Docker build fails"

**Solution:**
```bash
# Clear Docker cache
docker system prune -a

# Rebuild without cache
docker-compose build --no-cache
```

#### Issue: "Container exits immediately"

**Solution:**
```bash
# Check logs
docker-compose logs backend

# Common causes:
# - Missing dependencies
# - Port conflict
# - Syntax errors in code
```

#### Issue: "Health check failing"

**Solution:**
```bash
# Check if service is actually running
docker exec -it text-to-audiobook-backend curl http://localhost:8000/api/health

# Check logs for startup errors
docker-compose logs backend
```

### Performance Issues

#### Issue: "Generation is very slow"

**Causes & Solutions:**
- **Long text**: Split into smaller sections
- **First run**: Model download takes time (one-time)
- **Limited CPU**: Use faster machine or reduce text length
- **Docker on macOS**: Docker Desktop has performance overhead

#### Issue: "High memory usage"

**Solution:**
- Restart backend service to clear cache
- Process shorter texts
- Increase system RAM
- Monitor with: `docker stats` (Docker) or `top`/`htop` (local)

### Common Error Messages

| Error | Cause | Solution |
|-------|-------|----------|
| `ValidationError: Missing required field` | Invalid API request | Check request format matches API spec |
| `Language code 'x' not supported` | Invalid language code | Use: a, b, f, or j |
| `Voice ID 'x' not found` | Invalid voice ID | Check available voices list |
| `Text is required` | Empty text submitted | Enter text before proceeding |
| `CORS error` | Frontend can't reach backend | Check CORS settings and URLs |

### Getting Help

If you encounter issues not covered here:

1. Check the logs:
   - Backend: `docker-compose logs backend` or console output
   - Frontend: Browser developer console (F12)

2. Verify versions:
   ```bash
   python --version  # Should be 3.10+
   node --version    # Should be 20.19+ or 22.12+
   espeak-ng --version
   ```

3. Check the [GitHub Issues](https://github.com/yourusername/text-to-audiobook/issues)

4. Review the design document: `.kiro/specs/text-to-audiobook/design.md`

## Project Structure

```
.
├── backend/                    # FastAPI backend
│   ├── services/
│   │   ├── role_detector.py   # Role detection logic
│   │   └── tts_service.py     # TTS synthesis logic
│   ├── main.py                # API endpoints
│   ├── requirements.txt       # Python dependencies
│   ├── Dockerfile             # Docker configuration
│   ├── docker-compose.yml     # Docker Compose setup
│   ├── .env.example           # Environment template
│   ├── test_*.py              # Test files
│   └── README.md              # Backend documentation
├── frontend/                   # React frontend
│   ├── src/
│   │   ├── api/
│   │   │   └── tts.ts         # API client
│   │   ├── components/
│   │   │   ├── TextInputSection.tsx
│   │   │   ├── VoiceAssignmentSection.tsx
│   │   │   └── AudioPlayerSection.tsx
│   │   ├── App.tsx            # Main application
│   │   ├── main.tsx           # Entry point
│   │   └── index.css          # Global styles
│   ├── package.json           # Node dependencies
│   ├── vite.config.ts         # Vite configuration
│   ├── .env.example           # Environment template
│   └── README.md              # Frontend documentation
├── .kiro/specs/               # Specification documents
│   └── text-to-audiobook/
│       ├── requirements.md    # Requirements
│       ├── design.md          # Design document
│       └── tasks.md           # Implementation tasks
└── README.md                  # This file
```

## API Documentation

### Endpoints

#### GET /api/health
Health check endpoint.

**Response:**
```json
{
  "status": "ok"
}
```

#### POST /api/tts/roles
Detect roles in text.

**Request:**
```json
{
  "text": "Alice: Hello!\nBob: Hi there!"
}
```

**Response:**
```json
{
  "roles": [
    {"id": "Alice", "label": "Alice"},
    {"id": "Bob", "label": "Bob"}
  ]
}
```

#### POST /api/tts
Generate audiobook.

**Request:**
```json
{
  "text": "Alice: Hello!\nBob: Hi there!",
  "lang": "a",
  "roles": [
    {"id": "Alice", "voice": "af_bella"},
    {"id": "Bob", "voice": "am_adam"}
  ],
  "speed": 1.0
}
```

**Response:**
- Content-Type: `audio/wav`
- Content-Disposition: `attachment; filename="audiobook.wav"`
- Body: Audio file bytes

### Interactive API Documentation

Visit http://localhost:8000/docs for interactive Swagger UI documentation.

## Additional Resources

- [Backend Documentation](./backend/README.md)
- [Frontend Documentation](./frontend/README.md)
- [Requirements Document](./.kiro/specs/text-to-audiobook/requirements.md)
- [Design Document](./.kiro/specs/text-to-audiobook/design.md)
- [Kokoro TTS Library](https://github.com/hexgrad/kokoro)

## License

MIT