diff --git a/PIPER_TTS_INTEGRATION.md b/PIPER_TTS_INTEGRATION.md new file mode 100644 index 0000000..fd57d5e --- /dev/null +++ b/PIPER_TTS_INTEGRATION.md @@ -0,0 +1,209 @@ +# Piper TTS Integration Summary + +## Overview + +Successfully integrated Piper TTS as an alternative text-to-speech engine alongside Kokoro TTS, giving users the ability to choose between two engines with different strengths. + +## What Was Added + +### Backend Components + +1. **PiperTTSService** (`backend/services/piper_tts_service.py`) + - Handles Piper TTS synthesis with pitch control + - Supports pitch shifting from -12 to +12 semitones per character + - Automatic voice model downloading + - Audio effects processing (pitch and speed) + +2. **UnifiedTTSService** (`backend/services/unified_tts_service.py`) + - Manages multiple TTS engines (Kokoro and Piper) + - Runtime engine switching + - Unified API for both engines + - Handles engine-specific features (pitch control for Piper) + +3. **New API Endpoints** (in `backend/main.py`) + - `GET /api/tts/engines` - List available engines and current selection + - `POST /api/tts/engines` - Switch active TTS engine + - `GET /api/tts/voices` - Get available voices for all engines + - Updated `POST /api/tts` to support `pitch_adjustments` parameter + +4. **Installation Script** (`backend/install_piper.sh`) + - Automated Piper TTS installation for macOS/Linux + - Detects OS and architecture + - Installs to `/usr/local/bin` + +### Frontend Components + +1. **API Client Functions** (`frontend/src/api/tts.ts`) + - `getEngines()` - Fetch available engines + - `setEngine(engine)` - Switch active engine + - `getVoices()` - Get voices for all engines + - TypeScript interfaces for engine data + +2. **UI Components** (`frontend/src/components/VoiceAssignmentSection.tsx`) + - Engine selector buttons (Kokoro vs Piper) + - Dynamic voice loading based on selected engine + - Visual feedback for active engine + - Pitch control capability notice for Piper + +### Documentation + +1. **Backend README** (`backend/README.md`) + - Multi-engine setup instructions + - Engine switching examples + - Pitch control documentation + +2. **Piper TTS Guide** (`backend/PIPER_TTS_GUIDE.md`) + - Comprehensive installation guide + - Available voices documentation + - Pitch adjustment guidelines + - Usage examples and best practices + - Troubleshooting section + +3. **Integration Summary** (this file) + +## Engine Comparison + +| Feature | Kokoro TTS | Piper TTS | +|---------|------------|-----------| +| **Quality** | High | Medium-High | +| **Speed** | Medium | Fast | +| **Voices** | 54 voices, 9 languages | 5+ English voices | +| **Pitch Control** | ❌ No | ✅ Yes (-12 to +12 semitones) | +| **Model Size** | ~500MB | ~10-50MB per voice | +| **Memory Usage** | Higher | Lower | +| **Languages** | 9 languages | English (more available) | + +## How to Use + +### Backend Setup + +1. **Install Piper TTS** (optional): + ```bash + cd backend + ./install_piper.sh + ``` + +2. **Set default engine** (optional): + ```bash + # In backend/.env + TTS_ENGINE=piper # or kokoro (default) + ``` + +3. **Install Python dependencies**: + ```bash + pip install -r requirements.txt + ``` + +### Frontend Usage + +1. **Select Engine**: In the Voice Assignment screen, click either "Kokoro TTS" or "Piper TTS" button +2. **Choose Voices**: Voice dropdown updates automatically with engine-specific voices +3. **Generate**: Audio is generated using the selected engine + +### API Usage + +**Switch Engine:** +```bash +curl -X POST http://localhost:8000/api/tts/engines \ + -H "Content-Type: application/json" \ + -d '{"engine": "piper"}' +``` + +**Generate with Pitch Control (Piper only):** +```bash +curl -X POST http://localhost:8000/api/tts \ + -H "Content-Type: application/json" \ + -d '{ + "text": "Alice: Hello!\nBob: Hi there!", + "lang": "a", + "roles": [ + {"id": "Alice", "voice": "en_US-amy-medium"}, + {"id": "Bob", "voice": "en_US-lessac-medium"} + ], + "pitch_adjustments": [ + {"id": "Alice", "pitch": 3}, + {"id": "Bob", "pitch": -2} + ] + }' \ + --output audiobook.wav +``` + +## Pitch Control Guidelines + +When using Piper TTS, you can adjust pitch per character: + +- **+12 semitones**: One octave higher (very high-pitched) +- **+6 semitones**: Noticeably higher (child-like) +- **+3 semitones**: Slightly higher (feminine adjustment) +- **0 semitones**: Original voice (no change) +- **-3 semitones**: Slightly lower (masculine adjustment) +- **-6 semitones**: Noticeably lower (deep voice) +- **-12 semitones**: One octave lower (very deep) + +**Recommended ranges:** +- Female characters: +2 to +5 +- Male characters: -2 to -5 +- Children: +5 to +8 +- Elderly/deep voices: -4 to -8 + +## Testing + +Run the engine selection test: +```bash +cd backend +python test_engine_selection.py +``` + +## Files Modified/Created + +### Backend +- ✅ `backend/services/piper_tts_service.py` (new) +- ✅ `backend/services/unified_tts_service.py` (new) +- ✅ `backend/main.py` (modified - added endpoints) +- ✅ `backend/requirements.txt` (modified - added piper-tts, pydub) +- ✅ `backend/.env.example` (modified - added TTS_ENGINE) +- ✅ `backend/README.md` (modified - added multi-engine docs) +- ✅ `backend/PIPER_TTS_GUIDE.md` (new) +- ✅ `backend/install_piper.sh` (new) +- ✅ `backend/test_engine_selection.py` (new) + +### Frontend +- ✅ `frontend/src/api/tts.ts` (modified - added engine functions) +- ✅ `frontend/src/components/VoiceAssignmentSection.tsx` (modified - added engine selector) + +### Documentation +- ✅ `.kiro/specs/text-to-audiobook/tasks.md` (modified - added task 26) +- ✅ `PIPER_TTS_INTEGRATION.md` (new - this file) + +## Git Commits + +1. **feat: Add Piper TTS integration with pitch control support** (bfdcc94) + - Backend services and API endpoints + - Installation script and documentation + +2. **feat: Add TTS engine selector UI in frontend** (c310f11) + - Frontend UI components + - API client functions + +## Next Steps (Optional Enhancements) + +1. **Pitch Control UI**: Add pitch sliders in the frontend for Piper engine +2. **Voice Previews**: Update preview functionality to work with Piper voices +3. **More Piper Voices**: Add support for additional Piper voice models +4. **Coqui TTS**: Integrate Coqui XTTS as a third engine option +5. **Voice Cloning**: Add voice cloning capabilities with Piper/Coqui + +## Notes + +- Piper TTS requires separate installation (not included in Python packages) +- Voice models are downloaded automatically on first use +- Kokoro remains the default engine for backward compatibility +- Both engines can coexist and be switched at runtime +- Pitch control is only available with Piper TTS + +## Support + +For issues or questions: +- See `backend/PIPER_TTS_GUIDE.md` for detailed Piper documentation +- See `backend/README.md` for general backend setup +- Check troubleshooting sections in both guides diff --git a/backend/test_engine_selection.py b/backend/test_engine_selection.py new file mode 100644 index 0000000..4f1eefc --- /dev/null +++ b/backend/test_engine_selection.py @@ -0,0 +1,58 @@ +""" +Test script for TTS engine selection functionality. +""" +import asyncio +import sys +from services.unified_tts_service import UnifiedTTSService + + +async def test_engine_selection(): + """Test switching between TTS engines.""" + print("Testing TTS Engine Selection\n" + "="*50) + + # Initialize service + print("\n1. Initializing UnifiedTTSService...") + tts = UnifiedTTSService() + print(f" Default engine: {tts.engine_name}") + + # Get available engines + print("\n2. Getting available engines...") + engines = tts.get_available_engines() + print(f" Available: {engines}") + + # Get available voices + print("\n3. Getting available voices...") + voices = tts.get_available_voices() + for engine, voice_list in voices.items(): + print(f" {engine}: {len(voice_list)} voices") + if voice_list: + print(f" Examples: {voice_list[:3]}") + + # Test switching to Piper + print("\n4. Switching to Piper engine...") + try: + tts.set_engine('piper') + print(f" Current engine: {tts.engine_name}") + except Exception as e: + print(f" ⚠️ Error: {e}") + print(" Note: Piper may not be installed. Run ./install_piper.sh") + + # Switch back to Kokoro + print("\n5. Switching back to Kokoro engine...") + tts.set_engine('kokoro') + print(f" Current engine: {tts.engine_name}") + + # Test invalid engine + print("\n6. Testing invalid engine...") + try: + tts.set_engine('invalid') + print(" ❌ Should have raised error!") + except ValueError as e: + print(f" ✓ Correctly rejected: {e}") + + print("\n" + "="*50) + print("✓ Engine selection tests completed!") + + +if __name__ == "__main__": + asyncio.run(test_engine_selection())