fix: Update Piper TTS to support voice preview and improve setup
- Fixed preview endpoint to work with both Kokoro and Piper engines - Improved voice model download with curl fallback and SSL handling - Added better error messages when models are missing - Created PIPER_SETUP.md with manual download instructions - Added test_piper_direct.py for testing Piper installation - Updated README with voice model download requirements Note: Piper voice models require manual download from GitHub releases due to HuggingFace URL structure. See PIPER_SETUP.md for instructions.
This commit is contained in:
98
backend/PIPER_SETUP.md
Normal file
98
backend/PIPER_SETUP.md
Normal file
@@ -0,0 +1,98 @@
|
|||||||
|
# Piper TTS Setup Guide
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
Piper TTS is now integrated but requires manual voice model setup.
|
||||||
|
|
||||||
|
### 1. Install Piper
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd backend
|
||||||
|
./install_piper.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Verify installation:
|
||||||
|
```bash
|
||||||
|
piper --version
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Download Voice Models
|
||||||
|
|
||||||
|
Piper voice models need to be downloaded manually and placed in `backend/piper_models/`.
|
||||||
|
|
||||||
|
**Download voices from:**
|
||||||
|
https://github.com/rhasspy/piper/releases/tag/v1.2.0
|
||||||
|
|
||||||
|
**Recommended voices for English:**
|
||||||
|
|
||||||
|
1. **en_US-lessac-medium** (Male, American)
|
||||||
|
- Download: `en_US-lessac-medium.onnx` and `en_US-lessac-medium.onnx.json`
|
||||||
|
- Place in: `backend/piper_models/`
|
||||||
|
|
||||||
|
2. **en_US-amy-medium** (Female, American)
|
||||||
|
- Download: `en_US-amy-medium.onnx` and `en_US-amy-medium.onnx.json`
|
||||||
|
- Place in: `backend/piper_models/`
|
||||||
|
|
||||||
|
3. **en_GB-alan-medium** (Male, British)
|
||||||
|
- Download: `en_GB-alan-medium.onnx` and `en_GB-alan-medium.onnx.json`
|
||||||
|
- Place in: `backend/piper_models/`
|
||||||
|
|
||||||
|
### 3. Directory Structure
|
||||||
|
|
||||||
|
After downloading, your directory should look like:
|
||||||
|
|
||||||
|
```
|
||||||
|
backend/
|
||||||
|
├── piper_models/
|
||||||
|
│ ├── en_US-lessac-medium.onnx
|
||||||
|
│ ├── en_US-lessac-medium.onnx.json
|
||||||
|
│ ├── en_US-amy-medium.onnx
|
||||||
|
│ ├── en_US-amy-medium.onnx.json
|
||||||
|
│ ├── en_GB-alan-medium.onnx
|
||||||
|
│ └── en_GB-alan-medium.onnx.json
|
||||||
|
└── ...
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Test Piper
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd backend
|
||||||
|
python test_piper_direct.py
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Use in Application
|
||||||
|
|
||||||
|
1. Start the backend server
|
||||||
|
2. In the frontend, click "Piper TTS" button in Voice Assignment screen
|
||||||
|
3. Select Piper voices from the dropdown
|
||||||
|
4. Generate audiobook with pitch control!
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### "Model not found" error
|
||||||
|
|
||||||
|
- Make sure you downloaded both `.onnx` and `.onnx.json` files
|
||||||
|
- Check files are in `backend/piper_models/` directory
|
||||||
|
- Verify filenames match exactly (case-sensitive)
|
||||||
|
|
||||||
|
### "Piper not found" error
|
||||||
|
|
||||||
|
- Run `./install_piper.sh` to install Piper
|
||||||
|
- Or install manually from https://github.com/rhasspy/piper/releases
|
||||||
|
|
||||||
|
### Synthesis fails
|
||||||
|
|
||||||
|
- Check Piper version: `piper --version`
|
||||||
|
- Try running Piper directly: `echo "test" | piper --model piper_models/en_US-lessac-medium.onnx --output-raw > test.wav`
|
||||||
|
- Check backend logs for detailed error messages
|
||||||
|
|
||||||
|
## Alternative: Use Kokoro TTS
|
||||||
|
|
||||||
|
If Piper setup is too complex, you can continue using Kokoro TTS (the default engine) which works out of the box with no additional setup.
|
||||||
|
|
||||||
|
## More Voices
|
||||||
|
|
||||||
|
Browse all available Piper voices:
|
||||||
|
https://rhasspy.github.io/piper-samples/
|
||||||
|
|
||||||
|
Download any voice and place the `.onnx` and `.onnx.json` files in `backend/piper_models/`.
|
||||||
@@ -83,6 +83,15 @@ Or install manually:
|
|||||||
piper --version
|
piper --version
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**Download Voice Models:**
|
||||||
|
|
||||||
|
Piper requires manual voice model download. See [PIPER_SETUP.md](PIPER_SETUP.md) for detailed instructions.
|
||||||
|
|
||||||
|
Quick steps:
|
||||||
|
1. Visit https://github.com/rhasspy/piper/releases/tag/v1.2.0
|
||||||
|
2. Download voice models (e.g., `en_US-lessac-medium.onnx` and `.onnx.json`)
|
||||||
|
3. Place in `backend/piper_models/` directory
|
||||||
|
|
||||||
**Benefits of Piper TTS:**
|
**Benefits of Piper TTS:**
|
||||||
- ✨ Pitch shifting support (-12 to +12 semitones per role)
|
- ✨ Pitch shifting support (-12 to +12 semitones per role)
|
||||||
- ✨ Fast, lightweight neural TTS
|
- ✨ Fast, lightweight neural TTS
|
||||||
|
|||||||
@@ -385,26 +385,42 @@ async def preview_voice(request: VoicePreviewRequest):
|
|||||||
preview_text = request.text or "Hello! This is a preview of this voice. How does it sound?"
|
preview_text = request.text or "Hello! This is a preview of this voice. How does it sound?"
|
||||||
|
|
||||||
try:
|
try:
|
||||||
# Get pipeline for language
|
|
||||||
pipeline = tts_service.get_pipeline(request.lang)
|
|
||||||
|
|
||||||
# Generate preview audio
|
|
||||||
audio = tts_service.synthesize_segment(
|
|
||||||
pipeline=pipeline,
|
|
||||||
text=preview_text,
|
|
||||||
voice=request.voice,
|
|
||||||
speed=request.speed
|
|
||||||
)
|
|
||||||
|
|
||||||
# Convert to WAV bytes
|
|
||||||
import io
|
import io
|
||||||
import soundfile as sf
|
import soundfile as sf
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
# Check which engine is active
|
||||||
|
if tts_service.engine_name == 'kokoro':
|
||||||
|
# Use Kokoro service
|
||||||
|
pipeline = tts_service.kokoro_service.get_pipeline(request.lang)
|
||||||
|
audio = tts_service.kokoro_service.synthesize_segment(
|
||||||
|
pipeline=pipeline,
|
||||||
|
text=preview_text,
|
||||||
|
voice=request.voice,
|
||||||
|
speed=request.speed
|
||||||
|
)
|
||||||
|
sample_rate = tts_service.kokoro_service.SAMPLE_RATE
|
||||||
|
|
||||||
|
elif tts_service.engine_name == 'piper':
|
||||||
|
# Use Piper service with pitch control
|
||||||
|
audio = tts_service.piper_service.synthesize_with_pitch(
|
||||||
|
text=preview_text,
|
||||||
|
voice=request.voice,
|
||||||
|
speed=request.speed,
|
||||||
|
pitch_shift=0 # No pitch shift for preview
|
||||||
|
)
|
||||||
|
sample_rate = tts_service.piper_service.SAMPLE_RATE
|
||||||
|
|
||||||
|
else:
|
||||||
|
raise ValueError(f"Unknown engine: {tts_service.engine_name}")
|
||||||
|
|
||||||
|
# Convert to WAV bytes
|
||||||
buffer = io.BytesIO()
|
buffer = io.BytesIO()
|
||||||
sf.write(buffer, audio, tts_service.SAMPLE_RATE, format='WAV')
|
sf.write(buffer, audio, sample_rate, format='WAV')
|
||||||
buffer.seek(0)
|
buffer.seek(0)
|
||||||
audio_bytes = buffer.read()
|
audio_bytes = buffer.read()
|
||||||
|
|
||||||
logger.info(f"Generated voice preview: {len(audio_bytes)} bytes")
|
logger.info(f"Generated voice preview ({tts_service.engine_name}): {len(audio_bytes)} bytes")
|
||||||
|
|
||||||
return StreamingResponse(
|
return StreamingResponse(
|
||||||
iter([audio_bytes]),
|
iter([audio_bytes]),
|
||||||
|
|||||||
1
backend/piper_models/en_US-lessac-medium.onnx
Normal file
1
backend/piper_models/en_US-lessac-medium.onnx
Normal file
@@ -0,0 +1 @@
|
|||||||
|
Entry not found
|
||||||
@@ -72,20 +72,64 @@ class PiperTTSService:
|
|||||||
|
|
||||||
# Check if model already exists
|
# Check if model already exists
|
||||||
if os.path.exists(model_path) and os.path.exists(config_path):
|
if os.path.exists(model_path) and os.path.exists(config_path):
|
||||||
|
logger.info(f"Voice model {voice} already exists")
|
||||||
return True
|
return True
|
||||||
|
|
||||||
logger.info(f"Downloading Piper voice model: {voice}")
|
logger.info(f"Downloading Piper voice model: {voice}")
|
||||||
|
logger.info(f"Models directory: {self.models_dir}")
|
||||||
|
|
||||||
# Download using piper's built-in download
|
# Try to download using wget/curl as Piper's download might not work
|
||||||
try:
|
try:
|
||||||
subprocess.run(
|
# Piper models are hosted on HuggingFace
|
||||||
["piper", "--model", voice, "--download-dir", self.models_dir],
|
base_url = "https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0"
|
||||||
check=True,
|
|
||||||
timeout=60
|
# Determine language prefix from voice ID
|
||||||
)
|
lang_prefix = voice.split('-')[0] # e.g., "en_US" from "en_US-lessac-medium"
|
||||||
return os.path.exists(model_path)
|
|
||||||
except subprocess.SubprocessError as e:
|
model_url = f"{base_url}/{lang_prefix}/{voice}.onnx"
|
||||||
|
config_url = f"{base_url}/{lang_prefix}/{voice}.onnx.json"
|
||||||
|
|
||||||
|
logger.info(f"Downloading from: {model_url}")
|
||||||
|
|
||||||
|
# Try using curl/wget first (more reliable)
|
||||||
|
try:
|
||||||
|
# Try curl
|
||||||
|
subprocess.run(
|
||||||
|
["curl", "-L", "-o", model_path, model_url],
|
||||||
|
check=True,
|
||||||
|
timeout=120,
|
||||||
|
capture_output=True
|
||||||
|
)
|
||||||
|
subprocess.run(
|
||||||
|
["curl", "-L", "-o", config_path, config_url],
|
||||||
|
check=True,
|
||||||
|
timeout=120,
|
||||||
|
capture_output=True
|
||||||
|
)
|
||||||
|
except (subprocess.SubprocessError, FileNotFoundError):
|
||||||
|
# Fall back to urllib with SSL context
|
||||||
|
import urllib.request
|
||||||
|
import ssl
|
||||||
|
|
||||||
|
# Create SSL context that doesn't verify certificates (for development)
|
||||||
|
ssl_context = ssl.create_default_context()
|
||||||
|
ssl_context.check_hostname = False
|
||||||
|
ssl_context.verify_mode = ssl.CERT_NONE
|
||||||
|
|
||||||
|
urllib.request.urlretrieve(model_url, model_path, context=ssl_context)
|
||||||
|
urllib.request.urlretrieve(config_url, config_path, context=ssl_context)
|
||||||
|
|
||||||
|
if os.path.exists(model_path) and os.path.exists(config_path):
|
||||||
|
logger.info(f"Successfully downloaded voice model: {voice}")
|
||||||
|
return True
|
||||||
|
else:
|
||||||
|
logger.error(f"Download completed but files not found")
|
||||||
|
return False
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
logger.error(f"Failed to download voice model {voice}: {e}")
|
logger.error(f"Failed to download voice model {voice}: {e}")
|
||||||
|
logger.error(f"Please manually download from: https://github.com/rhasspy/piper/releases/tag/v1.2.0")
|
||||||
|
logger.error(f"See backend/PIPER_SETUP.md for detailed instructions")
|
||||||
return False
|
return False
|
||||||
|
|
||||||
def synthesize_with_pitch(
|
def synthesize_with_pitch(
|
||||||
@@ -109,14 +153,25 @@ class PiperTTSService:
|
|||||||
"""
|
"""
|
||||||
# Ensure voice model is available
|
# Ensure voice model is available
|
||||||
if not self._download_voice_model(voice):
|
if not self._download_voice_model(voice):
|
||||||
logger.warning(f"Voice {voice} not available, using default")
|
error_msg = (
|
||||||
voice = self.DEFAULT_VOICE
|
f"Piper voice model '{voice}' not found. "
|
||||||
self._download_voice_model(voice)
|
f"Please download voice models manually. "
|
||||||
|
f"See backend/PIPER_SETUP.md for instructions."
|
||||||
|
)
|
||||||
|
logger.error(error_msg)
|
||||||
|
raise FileNotFoundError(error_msg)
|
||||||
|
|
||||||
model_path = os.path.join(self.models_dir, f"{voice}.onnx")
|
model_path = os.path.join(self.models_dir, f"{voice}.onnx")
|
||||||
|
|
||||||
|
# Check if model file exists
|
||||||
|
if not os.path.exists(model_path):
|
||||||
|
logger.error(f"Model file not found: {model_path}")
|
||||||
|
return np.array([], dtype=np.float32)
|
||||||
|
|
||||||
# Generate audio with Piper
|
# Generate audio with Piper
|
||||||
try:
|
try:
|
||||||
|
logger.info(f"Running Piper with model: {model_path}")
|
||||||
|
|
||||||
# Run piper command
|
# Run piper command
|
||||||
process = subprocess.Popen(
|
process = subprocess.Popen(
|
||||||
["piper", "--model", model_path, "--output-raw"],
|
["piper", "--model", model_path, "--output-raw"],
|
||||||
@@ -129,7 +184,8 @@ class PiperTTSService:
|
|||||||
stdout, stderr = process.communicate(input=text.encode('utf-8'), timeout=30)
|
stdout, stderr = process.communicate(input=text.encode('utf-8'), timeout=30)
|
||||||
|
|
||||||
if process.returncode != 0:
|
if process.returncode != 0:
|
||||||
logger.error(f"Piper synthesis failed: {stderr.decode()}")
|
error_msg = stderr.decode() if stderr else "Unknown error"
|
||||||
|
logger.error(f"Piper synthesis failed (exit code {process.returncode}): {error_msg}")
|
||||||
return np.array([], dtype=np.float32)
|
return np.array([], dtype=np.float32)
|
||||||
|
|
||||||
# Convert raw audio to numpy array
|
# Convert raw audio to numpy array
|
||||||
|
|||||||
94
backend/test_piper_direct.py
Normal file
94
backend/test_piper_direct.py
Normal file
@@ -0,0 +1,94 @@
|
|||||||
|
"""
|
||||||
|
Direct test of Piper TTS functionality.
|
||||||
|
"""
|
||||||
|
import subprocess
|
||||||
|
import os
|
||||||
|
|
||||||
|
def test_piper_installation():
|
||||||
|
"""Test if Piper is installed and accessible."""
|
||||||
|
print("Testing Piper Installation\n" + "="*50)
|
||||||
|
|
||||||
|
# Check if piper command exists
|
||||||
|
try:
|
||||||
|
result = subprocess.run(
|
||||||
|
["piper", "--version"],
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
timeout=5
|
||||||
|
)
|
||||||
|
print(f"✓ Piper found: {result.stdout.strip()}")
|
||||||
|
return True
|
||||||
|
except FileNotFoundError:
|
||||||
|
print("✗ Piper not found in PATH")
|
||||||
|
print(" Install with: ./install_piper.sh")
|
||||||
|
return False
|
||||||
|
except Exception as e:
|
||||||
|
print(f"✗ Error checking Piper: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def test_piper_synthesis():
|
||||||
|
"""Test basic Piper synthesis."""
|
||||||
|
print("\nTesting Piper Synthesis\n" + "="*50)
|
||||||
|
|
||||||
|
# Create models directory
|
||||||
|
models_dir = os.path.join(os.path.dirname(__file__), "piper_models")
|
||||||
|
os.makedirs(models_dir, exist_ok=True)
|
||||||
|
print(f"Models directory: {models_dir}")
|
||||||
|
|
||||||
|
# Try to synthesize with default voice
|
||||||
|
voice = "en_US-lessac-medium"
|
||||||
|
model_path = os.path.join(models_dir, f"{voice}.onnx")
|
||||||
|
|
||||||
|
if not os.path.exists(model_path):
|
||||||
|
print(f"\n⚠️ Model not found: {model_path}")
|
||||||
|
print(f"Attempting to download {voice}...")
|
||||||
|
|
||||||
|
# Try downloading
|
||||||
|
try:
|
||||||
|
import urllib.request
|
||||||
|
base_url = "https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0"
|
||||||
|
model_url = f"{base_url}/en_US/{voice}.onnx"
|
||||||
|
config_url = f"{base_url}/en_US/{voice}.onnx.json"
|
||||||
|
|
||||||
|
print(f"Downloading from: {model_url}")
|
||||||
|
urllib.request.urlretrieve(model_url, model_path)
|
||||||
|
urllib.request.urlretrieve(config_url, f"{model_path}.json")
|
||||||
|
print("✓ Download complete")
|
||||||
|
except Exception as e:
|
||||||
|
print(f"✗ Download failed: {e}")
|
||||||
|
return False
|
||||||
|
else:
|
||||||
|
print(f"✓ Model found: {model_path}")
|
||||||
|
|
||||||
|
# Try synthesis
|
||||||
|
try:
|
||||||
|
print("\nAttempting synthesis...")
|
||||||
|
process = subprocess.Popen(
|
||||||
|
["piper", "--model", model_path, "--output-raw"],
|
||||||
|
stdin=subprocess.PIPE,
|
||||||
|
stdout=subprocess.PIPE,
|
||||||
|
stderr=subprocess.PIPE
|
||||||
|
)
|
||||||
|
|
||||||
|
test_text = "Hello, this is a test."
|
||||||
|
stdout, stderr = process.communicate(input=test_text.encode('utf-8'), timeout=10)
|
||||||
|
|
||||||
|
if process.returncode == 0:
|
||||||
|
print(f"✓ Synthesis successful: {len(stdout)} bytes of audio")
|
||||||
|
return True
|
||||||
|
else:
|
||||||
|
print(f"✗ Synthesis failed (exit code {process.returncode})")
|
||||||
|
print(f" Error: {stderr.decode()}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
print(f"✗ Synthesis error: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
if test_piper_installation():
|
||||||
|
test_piper_synthesis()
|
||||||
|
|
||||||
|
print("\n" + "="*50)
|
||||||
Reference in New Issue
Block a user