audiobook/PIPER_TTS_INTEGRATION.md

Piper TTS Integration Summary

Overview

Successfully integrated Piper TTS as an alternative text-to-speech engine alongside Kokoro TTS, giving users the ability to choose between two engines with different strengths.

What Was Added

Backend Components

1. PiperTTSService (backend/services/piper_tts_service.py)

   • Handles Piper TTS synthesis with pitch control
   • Supports pitch shifting from -12 to +12 semitones per character
   • Automatic voice model downloading
   • Audio effects processing (pitch and speed)

2. UnifiedTTSService (backend/services/unified_tts_service.py)

   • Manages multiple TTS engines (Kokoro and Piper)
   • Runtime engine switching
   • Unified API for both engines
   • Handles engine-specific features (pitch control for Piper)

3. New API Endpoints (in backend/main.py)

   • GET /api/tts/engines - List available engines and current selection
   • POST /api/tts/engines - Switch active TTS engine
   • GET /api/tts/voices - Get available voices for all engines
   • Updated POST /api/tts to support pitch_adjustments parameter

4. Installation Script (backend/install_piper.sh)

   • Automated Piper TTS installation for macOS/Linux
   • Detects OS and architecture
   • Installs to /usr/local/bin

Frontend Components

1. API Client Functions (frontend/src/api/tts.ts)

   • getEngines() - Fetch available engines
   • setEngine(engine) - Switch active engine
   • getVoices() - Get voices for all engines
   • TypeScript interfaces for engine data

2. UI Components (frontend/src/components/VoiceAssignmentSection.tsx)

   • Engine selector buttons (Kokoro vs Piper)
   • Dynamic voice loading based on selected engine
   • Visual feedback for active engine
   • Pitch control capability notice for Piper

Documentation

1. Backend README (backend/README.md)

   • Multi-engine setup instructions
   • Engine switching examples
   • Pitch control documentation

2. Piper TTS Guide (backend/PIPER_TTS_GUIDE.md)

   • Comprehensive installation guide
   • Available voices documentation
   • Pitch adjustment guidelines
   • Usage examples and best practices
   • Troubleshooting section

3. Integration Summary (this file)

Engine Comparison

Feature	Kokoro TTS	Piper TTS
Quality	High	Medium-High
Speed	Medium	Fast
Voices	54 voices, 9 languages	5+ English voices
Pitch Control	❌ No	✅ Yes (-12 to +12 semitones)
Model Size	~500MB	~10-50MB per voice
Memory Usage	Higher	Lower
Languages	9 languages	English (more available)

How to Use

Backend Setup

1. Install Piper TTS (optional):

   cd backend
   ./install_piper.sh
   

2. Set default engine (optional):

   # In backend/.env
   TTS_ENGINE=piper  # or kokoro (default)
   

3. Install Python dependencies:

   pip install -r requirements.txt
   

Frontend Usage

1. Select Engine: In the Voice Assignment screen, click either "Kokoro TTS" or "Piper TTS" button
2. Choose Voices: Voice dropdown updates automatically with engine-specific voices
3. Generate: Audio is generated using the selected engine

API Usage

Switch Engine:

curl -X POST http://localhost:8000/api/tts/engines \
  -H "Content-Type: application/json" \
  -d '{"engine": "piper"}'

Generate with Pitch Control (Piper only):

curl -X POST http://localhost:8000/api/tts \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Alice: Hello!\nBob: Hi there!",
    "lang": "a",
    "roles": [
      {"id": "Alice", "voice": "en_US-amy-medium"},
      {"id": "Bob", "voice": "en_US-lessac-medium"}
    ],
    "pitch_adjustments": [
      {"id": "Alice", "pitch": 3},
      {"id": "Bob", "pitch": -2}
    ]
  }' \
  --output audiobook.wav

Pitch Control Guidelines

When using Piper TTS, you can adjust pitch per character:

• +12 semitones: One octave higher (very high-pitched)
• +6 semitones: Noticeably higher (child-like)
• +3 semitones: Slightly higher (feminine adjustment)
• 0 semitones: Original voice (no change)
• -3 semitones: Slightly lower (masculine adjustment)
• -6 semitones: Noticeably lower (deep voice)
• -12 semitones: One octave lower (very deep)

Recommended ranges:

• Female characters: +2 to +5
• Male characters: -2 to -5
• Children: +5 to +8
• Elderly/deep voices: -4 to -8

Testing

Run the engine selection test:

cd backend
python test_engine_selection.py

Files Modified/Created

Backend

• ✅ backend/services/piper_tts_service.py (new)
• ✅ backend/services/unified_tts_service.py (new)
• ✅ backend/main.py (modified - added endpoints)
• ✅ backend/requirements.txt (modified - added piper-tts, pydub)
• ✅ backend/.env.example (modified - added TTS_ENGINE)
• ✅ backend/README.md (modified - added multi-engine docs)
• ✅ backend/PIPER_TTS_GUIDE.md (new)
• ✅ backend/install_piper.sh (new)
• ✅ backend/test_engine_selection.py (new)

Frontend

• ✅ frontend/src/api/tts.ts (modified - added engine functions)
• ✅ frontend/src/components/VoiceAssignmentSection.tsx (modified - added engine selector)

Documentation

• ✅ .kiro/specs/text-to-audiobook/tasks.md (modified - added task 26)
• ✅ PIPER_TTS_INTEGRATION.md (new - this file)

Git Commits

1. feat: Add Piper TTS integration with pitch control support (bfdcc94)

   • Backend services and API endpoints
   • Installation script and documentation

2. feat: Add TTS engine selector UI in frontend (c310f11)

   • Frontend UI components
   • API client functions

Next Steps (Optional Enhancements)

1. Pitch Control UI: Add pitch sliders in the frontend for Piper engine
2. Voice Previews: Update preview functionality to work with Piper voices
3. More Piper Voices: Add support for additional Piper voice models
4. Coqui TTS: Integrate Coqui XTTS as a third engine option
5. Voice Cloning: Add voice cloning capabilities with Piper/Coqui

Notes

• Piper TTS requires separate installation (not included in Python packages)
• Voice models are downloaded automatically on first use
• Kokoro remains the default engine for backward compatibility
• Both engines can coexist and be switched at runtime
• Pitch control is only available with Piper TTS

Support

For issues or questions:

• See backend/PIPER_TTS_GUIDE.md for detailed Piper documentation
• See backend/README.md for general backend setup
• Check troubleshooting sections in both guides