docs: Add comprehensive Piper TTS integration documentation
- Created PIPER_TTS_INTEGRATION.md with complete integration summary - Added test_engine_selection.py for testing engine switching - Documented engine comparison, usage examples, and pitch guidelines - Listed all modified/created files and git commits - Included next steps for optional enhancements Task 26 completed: Piper TTS integration with engine selection UI
This commit is contained in:
209
PIPER_TTS_INTEGRATION.md
Normal file
209
PIPER_TTS_INTEGRATION.md
Normal file
@@ -0,0 +1,209 @@
|
||||
# Piper TTS Integration Summary
|
||||
|
||||
## Overview
|
||||
|
||||
Successfully integrated Piper TTS as an alternative text-to-speech engine alongside Kokoro TTS, giving users the ability to choose between two engines with different strengths.
|
||||
|
||||
## What Was Added
|
||||
|
||||
### Backend Components
|
||||
|
||||
1. **PiperTTSService** (`backend/services/piper_tts_service.py`)
|
||||
- Handles Piper TTS synthesis with pitch control
|
||||
- Supports pitch shifting from -12 to +12 semitones per character
|
||||
- Automatic voice model downloading
|
||||
- Audio effects processing (pitch and speed)
|
||||
|
||||
2. **UnifiedTTSService** (`backend/services/unified_tts_service.py`)
|
||||
- Manages multiple TTS engines (Kokoro and Piper)
|
||||
- Runtime engine switching
|
||||
- Unified API for both engines
|
||||
- Handles engine-specific features (pitch control for Piper)
|
||||
|
||||
3. **New API Endpoints** (in `backend/main.py`)
|
||||
- `GET /api/tts/engines` - List available engines and current selection
|
||||
- `POST /api/tts/engines` - Switch active TTS engine
|
||||
- `GET /api/tts/voices` - Get available voices for all engines
|
||||
- Updated `POST /api/tts` to support `pitch_adjustments` parameter
|
||||
|
||||
4. **Installation Script** (`backend/install_piper.sh`)
|
||||
- Automated Piper TTS installation for macOS/Linux
|
||||
- Detects OS and architecture
|
||||
- Installs to `/usr/local/bin`
|
||||
|
||||
### Frontend Components
|
||||
|
||||
1. **API Client Functions** (`frontend/src/api/tts.ts`)
|
||||
- `getEngines()` - Fetch available engines
|
||||
- `setEngine(engine)` - Switch active engine
|
||||
- `getVoices()` - Get voices for all engines
|
||||
- TypeScript interfaces for engine data
|
||||
|
||||
2. **UI Components** (`frontend/src/components/VoiceAssignmentSection.tsx`)
|
||||
- Engine selector buttons (Kokoro vs Piper)
|
||||
- Dynamic voice loading based on selected engine
|
||||
- Visual feedback for active engine
|
||||
- Pitch control capability notice for Piper
|
||||
|
||||
### Documentation
|
||||
|
||||
1. **Backend README** (`backend/README.md`)
|
||||
- Multi-engine setup instructions
|
||||
- Engine switching examples
|
||||
- Pitch control documentation
|
||||
|
||||
2. **Piper TTS Guide** (`backend/PIPER_TTS_GUIDE.md`)
|
||||
- Comprehensive installation guide
|
||||
- Available voices documentation
|
||||
- Pitch adjustment guidelines
|
||||
- Usage examples and best practices
|
||||
- Troubleshooting section
|
||||
|
||||
3. **Integration Summary** (this file)
|
||||
|
||||
## Engine Comparison
|
||||
|
||||
| Feature | Kokoro TTS | Piper TTS |
|
||||
|---------|------------|-----------|
|
||||
| **Quality** | High | Medium-High |
|
||||
| **Speed** | Medium | Fast |
|
||||
| **Voices** | 54 voices, 9 languages | 5+ English voices |
|
||||
| **Pitch Control** | ❌ No | ✅ Yes (-12 to +12 semitones) |
|
||||
| **Model Size** | ~500MB | ~10-50MB per voice |
|
||||
| **Memory Usage** | Higher | Lower |
|
||||
| **Languages** | 9 languages | English (more available) |
|
||||
|
||||
## How to Use
|
||||
|
||||
### Backend Setup
|
||||
|
||||
1. **Install Piper TTS** (optional):
|
||||
```bash
|
||||
cd backend
|
||||
./install_piper.sh
|
||||
```
|
||||
|
||||
2. **Set default engine** (optional):
|
||||
```bash
|
||||
# In backend/.env
|
||||
TTS_ENGINE=piper # or kokoro (default)
|
||||
```
|
||||
|
||||
3. **Install Python dependencies**:
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### Frontend Usage
|
||||
|
||||
1. **Select Engine**: In the Voice Assignment screen, click either "Kokoro TTS" or "Piper TTS" button
|
||||
2. **Choose Voices**: Voice dropdown updates automatically with engine-specific voices
|
||||
3. **Generate**: Audio is generated using the selected engine
|
||||
|
||||
### API Usage
|
||||
|
||||
**Switch Engine:**
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/api/tts/engines \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"engine": "piper"}'
|
||||
```
|
||||
|
||||
**Generate with Pitch Control (Piper only):**
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/api/tts \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"text": "Alice: Hello!\nBob: Hi there!",
|
||||
"lang": "a",
|
||||
"roles": [
|
||||
{"id": "Alice", "voice": "en_US-amy-medium"},
|
||||
{"id": "Bob", "voice": "en_US-lessac-medium"}
|
||||
],
|
||||
"pitch_adjustments": [
|
||||
{"id": "Alice", "pitch": 3},
|
||||
{"id": "Bob", "pitch": -2}
|
||||
]
|
||||
}' \
|
||||
--output audiobook.wav
|
||||
```
|
||||
|
||||
## Pitch Control Guidelines
|
||||
|
||||
When using Piper TTS, you can adjust pitch per character:
|
||||
|
||||
- **+12 semitones**: One octave higher (very high-pitched)
|
||||
- **+6 semitones**: Noticeably higher (child-like)
|
||||
- **+3 semitones**: Slightly higher (feminine adjustment)
|
||||
- **0 semitones**: Original voice (no change)
|
||||
- **-3 semitones**: Slightly lower (masculine adjustment)
|
||||
- **-6 semitones**: Noticeably lower (deep voice)
|
||||
- **-12 semitones**: One octave lower (very deep)
|
||||
|
||||
**Recommended ranges:**
|
||||
- Female characters: +2 to +5
|
||||
- Male characters: -2 to -5
|
||||
- Children: +5 to +8
|
||||
- Elderly/deep voices: -4 to -8
|
||||
|
||||
## Testing
|
||||
|
||||
Run the engine selection test:
|
||||
```bash
|
||||
cd backend
|
||||
python test_engine_selection.py
|
||||
```
|
||||
|
||||
## Files Modified/Created
|
||||
|
||||
### Backend
|
||||
- ✅ `backend/services/piper_tts_service.py` (new)
|
||||
- ✅ `backend/services/unified_tts_service.py` (new)
|
||||
- ✅ `backend/main.py` (modified - added endpoints)
|
||||
- ✅ `backend/requirements.txt` (modified - added piper-tts, pydub)
|
||||
- ✅ `backend/.env.example` (modified - added TTS_ENGINE)
|
||||
- ✅ `backend/README.md` (modified - added multi-engine docs)
|
||||
- ✅ `backend/PIPER_TTS_GUIDE.md` (new)
|
||||
- ✅ `backend/install_piper.sh` (new)
|
||||
- ✅ `backend/test_engine_selection.py` (new)
|
||||
|
||||
### Frontend
|
||||
- ✅ `frontend/src/api/tts.ts` (modified - added engine functions)
|
||||
- ✅ `frontend/src/components/VoiceAssignmentSection.tsx` (modified - added engine selector)
|
||||
|
||||
### Documentation
|
||||
- ✅ `.kiro/specs/text-to-audiobook/tasks.md` (modified - added task 26)
|
||||
- ✅ `PIPER_TTS_INTEGRATION.md` (new - this file)
|
||||
|
||||
## Git Commits
|
||||
|
||||
1. **feat: Add Piper TTS integration with pitch control support** (bfdcc94)
|
||||
- Backend services and API endpoints
|
||||
- Installation script and documentation
|
||||
|
||||
2. **feat: Add TTS engine selector UI in frontend** (c310f11)
|
||||
- Frontend UI components
|
||||
- API client functions
|
||||
|
||||
## Next Steps (Optional Enhancements)
|
||||
|
||||
1. **Pitch Control UI**: Add pitch sliders in the frontend for Piper engine
|
||||
2. **Voice Previews**: Update preview functionality to work with Piper voices
|
||||
3. **More Piper Voices**: Add support for additional Piper voice models
|
||||
4. **Coqui TTS**: Integrate Coqui XTTS as a third engine option
|
||||
5. **Voice Cloning**: Add voice cloning capabilities with Piper/Coqui
|
||||
|
||||
## Notes
|
||||
|
||||
- Piper TTS requires separate installation (not included in Python packages)
|
||||
- Voice models are downloaded automatically on first use
|
||||
- Kokoro remains the default engine for backward compatibility
|
||||
- Both engines can coexist and be switched at runtime
|
||||
- Pitch control is only available with Piper TTS
|
||||
|
||||
## Support
|
||||
|
||||
For issues or questions:
|
||||
- See `backend/PIPER_TTS_GUIDE.md` for detailed Piper documentation
|
||||
- See `backend/README.md` for general backend setup
|
||||
- Check troubleshooting sections in both guides
|
||||
Reference in New Issue
Block a user