feat: Add Piper TTS integration with pitch control support
- Created PiperTTSService with pitch shifting (-12 to +12 semitones) - Created UnifiedTTSService to manage multiple TTS engines - Added engine selection endpoints (GET/POST /api/tts/engines) - Added voices listing endpoint (GET /api/tts/voices) - Added pitch_adjustments parameter to TTS generation - Created install_piper.sh for easy Piper installation - Updated backend README with multi-engine documentation - Created comprehensive PIPER_TTS_GUIDE.md - Added TTS_ENGINE environment variable for engine selection - Updated requirements.txt with piper-tts and pydub This allows users to choose between Kokoro (high quality) and Piper (fast with pitch control) TTS engines at runtime.
This commit is contained in:
292
backend/PIPER_TTS_GUIDE.md
Normal file
292
backend/PIPER_TTS_GUIDE.md
Normal file
@@ -0,0 +1,292 @@
|
||||
# Piper TTS Integration Guide
|
||||
|
||||
This guide covers the Piper TTS engine integration, including installation, configuration, and usage.
|
||||
|
||||
## What is Piper TTS?
|
||||
|
||||
Piper is a fast, local neural text-to-speech system that runs entirely on your machine. It offers:
|
||||
|
||||
- **Fast synthesis**: Faster than Kokoro for most use cases
|
||||
- **Pitch control**: Adjust voice pitch per character (-12 to +12 semitones)
|
||||
- **Multiple voices**: Various English voices with different characteristics
|
||||
- **Low resource usage**: Smaller models and lower memory footprint
|
||||
- **Privacy**: All processing happens locally
|
||||
|
||||
## Installation
|
||||
|
||||
### Automated Installation (Recommended)
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
./install_piper.sh
|
||||
```
|
||||
|
||||
This script will:
|
||||
1. Detect your OS and architecture
|
||||
2. Download the appropriate Piper binary
|
||||
3. Install it to `/usr/local/bin`
|
||||
4. Verify the installation
|
||||
|
||||
### Manual Installation
|
||||
|
||||
1. Visit [Piper Releases](https://github.com/rhasspy/piper/releases/tag/v1.2.0)
|
||||
2. Download the appropriate archive for your platform:
|
||||
- **Linux x86_64**: `piper_amd64.tar.gz`
|
||||
- **Linux ARM64**: `piper_arm64.tar.gz`
|
||||
- **macOS**: `piper_macos_x64.tar.gz`
|
||||
3. Extract the archive
|
||||
4. Copy the `piper` binary to a directory in your PATH
|
||||
|
||||
### Verify Installation
|
||||
|
||||
```bash
|
||||
piper --version
|
||||
```
|
||||
|
||||
You should see output like: `1.2.0`
|
||||
|
||||
## Available Voices
|
||||
|
||||
Piper supports multiple English voices:
|
||||
|
||||
| Voice ID | Language | Gender | Quality | Description |
|
||||
|----------|----------|--------|---------|-------------|
|
||||
| `en_US-lessac-medium` | US English | Male | Medium | Clear, neutral male voice |
|
||||
| `en_US-libritts-high` | US English | Neutral | High | High-quality neutral voice |
|
||||
| `en_US-amy-medium` | US English | Female | Medium | Clear female voice |
|
||||
| `en_GB-alan-medium` | UK English | Male | Medium | British male voice |
|
||||
| `en_GB-alba-medium` | UK English | Female | Medium | British female voice |
|
||||
|
||||
Voice models are automatically downloaded on first use (~10-50MB each).
|
||||
|
||||
## Configuration
|
||||
|
||||
### Set Piper as Default Engine
|
||||
|
||||
**Option 1: Environment Variable**
|
||||
```bash
|
||||
export TTS_ENGINE=piper
|
||||
```
|
||||
|
||||
Add to your `.env` file:
|
||||
```bash
|
||||
TTS_ENGINE=piper
|
||||
```
|
||||
|
||||
**Option 2: Runtime API Call**
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/api/tts/engines \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"engine": "piper"}'
|
||||
```
|
||||
|
||||
### Voice Model Storage
|
||||
|
||||
Voice models are stored in:
|
||||
```
|
||||
backend/piper_models/
|
||||
```
|
||||
|
||||
Models are downloaded automatically on first use. You can pre-download models manually if needed.
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Basic Text-to-Speech
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/api/tts \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"text": "Hello, this is a test of Piper TTS.",
|
||||
"lang": "a",
|
||||
"roles": [
|
||||
{"id": "Narrator", "voice": "en_US-lessac-medium"}
|
||||
],
|
||||
"speed": 1.0
|
||||
}' \
|
||||
--output test.wav
|
||||
```
|
||||
|
||||
### Multi-Voice with Pitch Control
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/api/tts \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"text": "Alice: Hello Bob!\nBob: Hi Alice, how are you?\nNarrator: They greeted each other warmly.",
|
||||
"lang": "a",
|
||||
"roles": [
|
||||
{"id": "Alice", "voice": "en_US-amy-medium"},
|
||||
{"id": "Bob", "voice": "en_US-lessac-medium"},
|
||||
{"id": "Narrator", "voice": "en_GB-alan-medium"}
|
||||
],
|
||||
"speed": 1.0,
|
||||
"pitch_adjustments": [
|
||||
{"id": "Alice", "pitch": 4},
|
||||
{"id": "Bob", "pitch": -3},
|
||||
{"id": "Narrator", "pitch": 0}
|
||||
]
|
||||
}' \
|
||||
--output audiobook.wav
|
||||
```
|
||||
|
||||
### Pitch Adjustment Guidelines
|
||||
|
||||
Pitch is measured in semitones (half-steps on a musical scale):
|
||||
|
||||
- **+12 semitones**: One octave higher (very high-pitched)
|
||||
- **+6 semitones**: Noticeably higher (child-like)
|
||||
- **+3 semitones**: Slightly higher (feminine adjustment)
|
||||
- **0 semitones**: Original voice (no change)
|
||||
- **-3 semitones**: Slightly lower (masculine adjustment)
|
||||
- **-6 semitones**: Noticeably lower (deep voice)
|
||||
- **-12 semitones**: One octave lower (very deep)
|
||||
|
||||
**Recommended ranges:**
|
||||
- Female characters: +2 to +5
|
||||
- Male characters: -2 to -5
|
||||
- Children: +5 to +8
|
||||
- Elderly/deep voices: -4 to -8
|
||||
|
||||
## Python API Usage
|
||||
|
||||
```python
|
||||
from services.unified_tts_service import UnifiedTTSService
|
||||
|
||||
# Initialize with Piper engine
|
||||
tts = UnifiedTTSService(engine="piper")
|
||||
|
||||
# Generate audiobook with pitch control
|
||||
audio_bytes = await tts.generate_audiobook(
|
||||
text="Alice: Hello!\nBob: Hi there!",
|
||||
lang="a",
|
||||
role_voice_map={
|
||||
"Alice": "en_US-amy-medium",
|
||||
"Bob": "en_US-lessac-medium"
|
||||
},
|
||||
speed=1.0,
|
||||
pitch_adjustments={
|
||||
"Alice": 3,
|
||||
"Bob": -2
|
||||
}
|
||||
)
|
||||
|
||||
# Save to file
|
||||
with open("output.wav", "wb") as f:
|
||||
f.write(audio_bytes)
|
||||
```
|
||||
|
||||
## Performance Comparison
|
||||
|
||||
| Feature | Kokoro TTS | Piper TTS |
|
||||
|---------|------------|-----------|
|
||||
| Speed | Medium | Fast |
|
||||
| Quality | High | Medium-High |
|
||||
| Pitch Control | ❌ No | ✅ Yes |
|
||||
| Model Size | ~500MB | ~10-50MB per voice |
|
||||
| Memory Usage | Higher | Lower |
|
||||
| Voice Variety | 10+ voices | 5+ voices (expandable) |
|
||||
| Languages | 4 languages | English (more available) |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Piper Not Found
|
||||
|
||||
**Error:** `piper: command not found`
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Check if piper is in PATH
|
||||
which piper
|
||||
|
||||
# If not found, reinstall
|
||||
./install_piper.sh
|
||||
|
||||
# Or add to PATH manually
|
||||
export PATH="/usr/local/bin:$PATH"
|
||||
```
|
||||
|
||||
### Voice Model Download Fails
|
||||
|
||||
**Error:** Model download timeout or failure
|
||||
|
||||
**Solution:**
|
||||
1. Check internet connection
|
||||
2. Manually download models from [Piper releases](https://github.com/rhasspy/piper/releases)
|
||||
3. Place `.onnx` and `.onnx.json` files in `backend/piper_models/`
|
||||
|
||||
### Audio Quality Issues
|
||||
|
||||
**Problem:** Distorted or robotic audio
|
||||
|
||||
**Solutions:**
|
||||
- Reduce pitch shift amount (stay within ±6 semitones)
|
||||
- Try different voice models
|
||||
- Adjust speed (try 0.9 or 1.1)
|
||||
- Check for text formatting issues
|
||||
|
||||
### Synthesis Timeout
|
||||
|
||||
**Error:** Piper synthesis times out
|
||||
|
||||
**Solutions:**
|
||||
- Break text into smaller chunks
|
||||
- Increase timeout in `piper_tts_service.py`
|
||||
- Check system resources (CPU, memory)
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
### Custom Voice Models
|
||||
|
||||
To add custom Piper voice models:
|
||||
|
||||
1. Download `.onnx` and `.onnx.json` files
|
||||
2. Place in `backend/piper_models/`
|
||||
3. Update `AVAILABLE_VOICES` in `piper_tts_service.py`:
|
||||
|
||||
```python
|
||||
AVAILABLE_VOICES = {
|
||||
"en_US-custom-voice": {"lang": "en", "gender": "female", "quality": "high"},
|
||||
# ... existing voices
|
||||
}
|
||||
```
|
||||
|
||||
### Adjusting Audio Effects
|
||||
|
||||
Modify `_apply_audio_effects()` in `piper_tts_service.py` to customize:
|
||||
- Pitch shifting algorithm
|
||||
- Speed adjustment method
|
||||
- Audio processing pipeline
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Voice Selection**
|
||||
- Use distinct voices for different characters
|
||||
- Match voice gender to character when possible
|
||||
- Use British voices for variety
|
||||
|
||||
2. **Pitch Adjustments**
|
||||
- Start with small adjustments (±2-3 semitones)
|
||||
- Test and iterate to find natural-sounding values
|
||||
- Avoid extreme values (±10+) unless intentional
|
||||
|
||||
3. **Performance**
|
||||
- Pre-download voice models for faster first use
|
||||
- Cache generated audio when possible
|
||||
- Use appropriate chunk sizes for long texts
|
||||
|
||||
4. **Quality**
|
||||
- Use high-quality voice models when available
|
||||
- Adjust speed slightly (0.95-1.05) for more natural pacing
|
||||
- Combine with AI segmentation for best results
|
||||
|
||||
## Resources
|
||||
|
||||
- [Piper GitHub Repository](https://github.com/rhasspy/piper)
|
||||
- [Piper Voice Samples](https://rhasspy.github.io/piper-samples/)
|
||||
- [Voice Model Downloads](https://github.com/rhasspy/piper/releases/tag/v1.2.0)
|
||||
- [Piper Documentation](https://github.com/rhasspy/piper/blob/master/README.md)
|
||||
|
||||
## License
|
||||
|
||||
Piper TTS is licensed under the MIT License.
|
||||
Reference in New Issue
Block a user