audiobook/backend

Text to Audiobook Backend

FastAPI backend service for converting text to multi-voice audiobooks using Kokoro TTS (hexgrad/Kokoro-82M).

Overview

The backend provides REST API endpoints for:

• Role/speaker detection in text
• Multi-voice audio synthesis
• Audio streaming and download

Prerequisites

• Python 3.10 or higher
• espeak-ng (system dependency for grapheme-to-phoneme conversion)
• Docker (optional, for containerized deployment)

Local Development Setup

1. Install espeak-ng

macOS:

brew install espeak-ng

Linux (Ubuntu/Debian):

sudo apt-get update
sudo apt-get install espeak-ng

Linux (Fedora/RHEL):

sudo dnf install espeak-ng

Windows: Download from espeak-ng releases

Verify installation:

espeak-ng --version

2. Python Environment Setup

# Create virtual environment
python -m venv venv

# Activate virtual environment
# macOS/Linux:
source venv/bin/activate
# Windows:
venv\Scripts\activate

# Upgrade pip
pip install --upgrade pip

# Install dependencies
pip install -r requirements.txt

3. Environment Configuration

cp .env.example .env

Edit .env if needed:

PORT=8000
LOG_LEVEL=info

4. Run the Server

uvicorn main:app --reload --host 0.0.0.0 --port 8000

The server will start on http://localhost:8000

Available URLs:

• API Base: http://localhost:8000
• Interactive API Docs: http://localhost:8000/docs
• Alternative API Docs: http://localhost:8000/redoc
• Health Check: http://localhost:8000/api/health

Docker Deployment

Using Docker Compose (Recommended)

# Build and start
docker-compose up -d

# View logs
docker-compose logs -f backend

# Stop
docker-compose down

# Rebuild after code changes
docker-compose up -d --build

Using Docker Directly

# Build image
docker build -t text-to-audiobook-backend .

# Run container
docker run -d \
  --name text-to-audiobook-backend \
  -p 8000:8000 \
  -e PORT=8000 \
  -e LOG_LEVEL=info \
  text-to-audiobook-backend

# View logs
docker logs -f text-to-audiobook-backend

# Stop and remove
docker stop text-to-audiobook-backend
docker rm text-to-audiobook-backend

Docker Configuration

The Dockerfile includes:

• Python 3.10 slim base image
• Automatic espeak-ng installation
• Python dependencies from requirements.txt
• Health check monitoring
• Uvicorn ASGI server

Resource Requirements:

• Memory: 2GB minimum (4GB recommended)
• Storage: ~500MB for Kokoro models
• CPU: Multi-core recommended

API Endpoints

GET /api/health

Health check endpoint.

Response:

{
  "status": "ok"
}

Example:

curl http://localhost:8000/api/health

POST /api/tts/roles

Detect distinct roles/speakers in text.

Request Body:

{
  "text": "Alice: Hello there!\nBob: Hi Alice, how are you?\nNarrator: They greeted each other warmly."
}

Response:

{
  "roles": [
    {"id": "Alice", "label": "Alice"},
    {"id": "Bob", "label": "Bob"},
    {"id": "Narrator", "label": "Narrator"}
  ]
}

Supported Role Patterns:

• Colon pattern: RoleName: dialogue text
• Bracket pattern: [RoleName] dialogue text

Example:

curl -X POST http://localhost:8000/api/tts/roles \
  -H "Content-Type: application/json" \
  -d '{"text":"Alice: Hello!\nBob: Hi there!"}'

POST /api/tts

Generate multi-voice audiobook.

Request Body:

{
  "text": "Alice: Hello there!\nBob: Hi Alice!",
  "lang": "a",
  "roles": [
    {"id": "Alice", "voice": "af_bella"},
    {"id": "Bob", "voice": "am_adam"}
  ],
  "speed": 1.0
}

Parameters:

• text (required): Text to synthesize
• lang (required): Language code (a=American English, b=British English, f=French, j=Japanese)
• roles (required): Array of role-to-voice mappings
• speed (optional): Playback speed (0.5-1.5, default: 1.0)

Response:

• Content-Type: audio/wav
• Content-Disposition: attachment; filename="audiobook.wav"
• Body: Audio file bytes

Example:

curl -X POST http://localhost:8000/api/tts \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Alice: Hello!\nBob: Hi!",
    "lang": "a",
    "roles": [
      {"id": "Alice", "voice": "af_bella"},
      {"id": "Bob", "voice": "am_adam"}
    ],
    "speed": 1.0
  }' \
  --output audiobook.wav

Kokoro TTS Configuration

Supported Languages

Code	Language
a	American English
b	British English
f	French
j	Japanese

Available Voices

Voice ID	Gender	Accent	Description
af_bella	Female	American	Bella
af_sarah	Female	American	Sarah
am_adam	Male	American	Adam
am_michael	Male	American	Michael
bf_emma	Female	British	Emma
bf_isabella	Female	British	Isabella
bm_george	Male	British	George
bm_lewis	Male	British	Lewis

Model Storage

Models are automatically downloaded on first use and cached:

• Linux/macOS: ~/.cache/kokoro/
• Windows: %USERPROFILE%\.cache\kokoro\

First synthesis may take longer due to model download (~500MB).

Project Structure

backend/
├── services/
│   ├── __init__.py
│   ├── role_detector.py      # Role detection logic
│   └── tts_service.py         # TTS synthesis service
├── main.py                    # FastAPI application & endpoints
├── requirements.txt           # Python dependencies
├── Dockerfile                 # Docker image configuration
├── docker-compose.yml         # Docker Compose setup
├── .env.example               # Environment template
├── .dockerignore              # Docker ignore patterns
├── test_api.py                # API endpoint tests
├── test_role_detection.py     # Role detection tests
├── test_tts_service.py        # TTS service tests
└── README.md                  # This file

Testing

Run All Tests

# Activate virtual environment
source venv/bin/activate  # macOS/Linux
# or
venv\Scripts\activate  # Windows

# Run tests
pytest test_*.py -v

Run Specific Tests

# Test API endpoints
pytest test_api.py -v

# Test role detection
pytest test_role_detection.py -v

# Test TTS service
pytest test_tts_service.py -v

Test Coverage

pytest --cov=. --cov-report=html test_*.py

Troubleshooting

espeak-ng not found

Error: OSError: espeak-ng not found

Solution:

# Install espeak-ng
brew install espeak-ng  # macOS
sudo apt-get install espeak-ng  # Linux

# Verify
espeak-ng --version

Module not found

Error: ModuleNotFoundError: No module named 'kokoro'

Solution:

source venv/bin/activate
pip install -r requirements.txt

Port already in use

Error: Address already in use

Solution:

# Find process using port 8000
lsof -i :8000  # macOS/Linux
netstat -ano | findstr :8000  # Windows

# Use different port
uvicorn main:app --port 8001

Model download fails

Symptoms: First synthesis times out

Solution:

• Check internet connection
• Check firewall settings
• Manually trigger download:

  python -c "from kokoro import KPipeline; KPipeline(lang_code='a')"
  

Out of memory

Error: Memory error during synthesis

Solution:

• Reduce text length
• Increase system RAM
• For Docker: docker run -m 4g ...

Performance Optimization

Tips for Faster Synthesis

1. Use cached pipelines: The service caches KPipeline instances per language
2. Shorter texts: Break very long texts into chapters
3. Multi-core CPU: Synthesis benefits from multiple cores
4. SSD storage: Faster model loading from SSD

Monitoring

# View logs
docker-compose logs -f backend

# Monitor resource usage
docker stats text-to-audiobook-backend

# Check health
curl http://localhost:8000/api/health

Development

Code Style

# Format code
black .

# Lint
flake8 .

# Type checking
mypy .

Adding New Voices

To add new Kokoro voices, update the voice validation in main.py:

SUPPORTED_VOICES = [
    "af_bella", "af_sarah", "am_adam", "am_michael",
    "bf_emma", "bf_isabella", "bm_george", "bm_lewis",
    # Add new voices here
]

Adding New Languages

To add new languages, update the language validation in main.py:

SUPPORTED_LANGUAGES = ["a", "b", "f", "j"]  # Add new codes

Security Considerations

• Input validation on all endpoints
• Text length limits to prevent DoS
• CORS configured for frontend origin
• No sensitive data in logs
• Rate limiting recommended for production

Production Deployment

Recommendations

1. Use environment variables for configuration
2. Enable HTTPS with reverse proxy (nginx, Caddy)
3. Set up monitoring (health checks, logs)
4. Configure rate limiting
5. Use production ASGI server (Uvicorn with workers)
6. Set resource limits (memory, CPU)

Production Command

uvicorn main:app \
  --host 0.0.0.0 \
  --port 8000 \
  --workers 4 \
  --log-level info \
  --no-access-log

Additional Resources

• Kokoro TTS GitHub
• FastAPI Documentation
• Main Project README
• Design Document

License

MIT