Compare commits
10 Commits
20e6cd1f9d
...
1c495079eb
| Author | SHA1 | Date | |
|---|---|---|---|
| 1c495079eb | |||
| 5c7b339bdf | |||
| 7ef969314b | |||
| e3dcb89bb7 | |||
| 4180527821 | |||
| d18fddef57 | |||
| b48e53ce63 | |||
| f56f8086e5 | |||
| 73dc5695ca | |||
| a8802fd765 |
@@ -289,7 +289,7 @@
|
|||||||
- Include troubleshooting section
|
- Include troubleshooting section
|
||||||
- _Requirements: 13.1, 13.2, 13.3, 13.4, 13.5_
|
- _Requirements: 13.1, 13.2, 13.3, 13.4, 13.5_
|
||||||
|
|
||||||
- [ ] 15. Implement Ollama integration service
|
- [x] 15. Implement Ollama integration service
|
||||||
- Create OllamaService class in services/ollama_service.py
|
- Create OllamaService class in services/ollama_service.py
|
||||||
- Implement connection to Ollama API using httpx
|
- Implement connection to Ollama API using httpx
|
||||||
- Configure OLLAMA_BASE_URL environment variable (default: http://localhost:11434)
|
- Configure OLLAMA_BASE_URL environment variable (default: http://localhost:11434)
|
||||||
@@ -316,7 +316,7 @@
|
|||||||
- Test error handling for unavailable service
|
- Test error handling for unavailable service
|
||||||
- _Requirements: 14.1, 14.2, 14.3_
|
- _Requirements: 14.1, 14.2, 14.3_
|
||||||
|
|
||||||
- [ ] 16. Integrate Ollama into role detection
|
- [x] 16. Integrate Ollama into role detection
|
||||||
- Update RoleDetector to use OllamaService
|
- Update RoleDetector to use OllamaService
|
||||||
- Implement _detect_with_ollama() method
|
- Implement _detect_with_ollama() method
|
||||||
- Add fallback logic: try Ollama first, then pattern matching
|
- Add fallback logic: try Ollama first, then pattern matching
|
||||||
|
|||||||
157
AVAILABLE_VOICES.md
Normal file
157
AVAILABLE_VOICES.md
Normal file
@@ -0,0 +1,157 @@
|
|||||||
|
# Available Kokoro TTS Voices
|
||||||
|
|
||||||
|
Kokoro-82M has **54 voices** available across multiple languages!
|
||||||
|
|
||||||
|
## Voice Naming Convention
|
||||||
|
- **First letter**: Language code
|
||||||
|
- `a` = American English
|
||||||
|
- `b` = British English
|
||||||
|
- `e` = Spanish (español)
|
||||||
|
- `f` = French
|
||||||
|
- `h` = Hindi
|
||||||
|
- `i` = Italian
|
||||||
|
- `j` = Japanese
|
||||||
|
- `p` = Portuguese
|
||||||
|
- `z` = Chinese (Mandarin)
|
||||||
|
- **Second letter**: Gender
|
||||||
|
- `f` = Female
|
||||||
|
- `m` = Male
|
||||||
|
- **Rest**: Voice name/identifier
|
||||||
|
|
||||||
|
## American English (a) - 21 voices
|
||||||
|
|
||||||
|
### Female (af_*)
|
||||||
|
- `af_alloy` - Alloy
|
||||||
|
- `af_aoede` - Aoede
|
||||||
|
- `af_bella` - Bella ✓ (currently in app)
|
||||||
|
- `af_heart` - Heart
|
||||||
|
- `af_jessica` - Jessica
|
||||||
|
- `af_kore` - Kore
|
||||||
|
- `af_nicole` - Nicole
|
||||||
|
- `af_nova` - Nova
|
||||||
|
- `af_river` - River
|
||||||
|
- `af_sarah` - Sarah ✓ (currently in app)
|
||||||
|
- `af_sky` - Sky
|
||||||
|
|
||||||
|
### Male (am_*)
|
||||||
|
- `am_adam` - Adam ✓ (currently in app)
|
||||||
|
- `am_echo` - Echo
|
||||||
|
- `am_eric` - Eric
|
||||||
|
- `am_fenrir` - Fenrir
|
||||||
|
- `am_liam` - Liam
|
||||||
|
- `am_michael` - Michael ✓ (currently in app)
|
||||||
|
- `am_onyx` - Onyx
|
||||||
|
- `am_puck` - Puck
|
||||||
|
- `am_santa` - Santa
|
||||||
|
|
||||||
|
## British English (b) - 8 voices
|
||||||
|
|
||||||
|
### Female (bf_*)
|
||||||
|
- `bf_alice` - Alice
|
||||||
|
- `bf_emma` - Emma ✓ (currently in app)
|
||||||
|
- `bf_isabella` - Isabella ✓ (currently in app)
|
||||||
|
- `bf_lily` - Lily
|
||||||
|
|
||||||
|
### Male (bm_*)
|
||||||
|
- `bm_daniel` - Daniel
|
||||||
|
- `bm_fable` - Fable
|
||||||
|
- `bm_george` - George ✓ (currently in app)
|
||||||
|
- `bm_lewis` - Lewis ✓ (currently in app)
|
||||||
|
|
||||||
|
## Spanish (e) - 3 voices
|
||||||
|
|
||||||
|
### Female (ef_*)
|
||||||
|
- `ef_dora` - Dora
|
||||||
|
|
||||||
|
### Male (em_*)
|
||||||
|
- `em_alex` - Alex
|
||||||
|
- `em_santa` - Santa
|
||||||
|
|
||||||
|
## French (f) - 1 voice
|
||||||
|
|
||||||
|
### Female (ff_*)
|
||||||
|
- `ff_siwis` - Siwis
|
||||||
|
|
||||||
|
## Hindi (h) - 4 voices
|
||||||
|
|
||||||
|
### Female (hf_*)
|
||||||
|
- `hf_alpha` - Alpha
|
||||||
|
- `hf_beta` - Beta
|
||||||
|
|
||||||
|
### Male (hm_*)
|
||||||
|
- `hm_omega` - Omega
|
||||||
|
- `hm_psi` - Psi
|
||||||
|
|
||||||
|
## Italian (i) - 2 voices
|
||||||
|
|
||||||
|
### Female (if_*)
|
||||||
|
- `if_sara` - Sara
|
||||||
|
|
||||||
|
### Male (im_*)
|
||||||
|
- `im_nicola` - Nicola
|
||||||
|
|
||||||
|
## Japanese (j) - 5 voices
|
||||||
|
|
||||||
|
### Female (jf_*)
|
||||||
|
- `jf_alpha` - Alpha
|
||||||
|
- `jf_gongitsune` - Gongitsune
|
||||||
|
- `jf_nezumi` - Nezumi
|
||||||
|
- `jf_tebukuro` - Tebukuro
|
||||||
|
|
||||||
|
### Male (jm_*)
|
||||||
|
- `jm_kumo` - Kumo
|
||||||
|
|
||||||
|
## Portuguese (p) - 3 voices
|
||||||
|
|
||||||
|
### Female (pf_*)
|
||||||
|
- `pf_dora` - Dora
|
||||||
|
|
||||||
|
### Male (pm_*)
|
||||||
|
- `pm_alex` - Alex
|
||||||
|
- `pm_santa` - Santa
|
||||||
|
|
||||||
|
## Chinese/Mandarin (z) - 7 voices
|
||||||
|
|
||||||
|
### Female (zf_*)
|
||||||
|
- `zf_xiaobei` - Xiaobei
|
||||||
|
- `zf_xiaoni` - Xiaoni
|
||||||
|
- `zf_xiaoxiao` - Xiaoxiao
|
||||||
|
- `zf_xiaoyi` - Xiaoyi
|
||||||
|
|
||||||
|
### Male (zm_*)
|
||||||
|
- `zm_yunjian` - Yunjian
|
||||||
|
- `zm_yunxi` - Yunxi
|
||||||
|
- `zm_yunxia` - Yunxia
|
||||||
|
- `zm_yunyang` - Yunyang
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
| Language | Female Voices | Male Voices | Total |
|
||||||
|
|----------|--------------|-------------|-------|
|
||||||
|
| American English | 11 | 10 | 21 |
|
||||||
|
| British English | 4 | 4 | 8 |
|
||||||
|
| Spanish | 1 | 2 | 3 |
|
||||||
|
| French | 1 | 0 | 1 |
|
||||||
|
| Hindi | 2 | 2 | 4 |
|
||||||
|
| Italian | 1 | 1 | 2 |
|
||||||
|
| Japanese | 4 | 1 | 5 |
|
||||||
|
| Portuguese | 1 | 2 | 3 |
|
||||||
|
| Chinese | 4 | 4 | 8 |
|
||||||
|
| **TOTAL** | **29** | **26** | **55** |
|
||||||
|
|
||||||
|
## Currently Configured (8 voices)
|
||||||
|
|
||||||
|
The app currently uses only 8 voices:
|
||||||
|
- ✓ af_bella, af_sarah (American Female)
|
||||||
|
- ✓ am_adam, am_michael (American Male)
|
||||||
|
- ✓ bf_emma, bf_isabella (British Female)
|
||||||
|
- ✓ bm_george, bm_lewis (British Male)
|
||||||
|
|
||||||
|
## Recommendation
|
||||||
|
|
||||||
|
You could expand the voice options to include:
|
||||||
|
1. **More variety in American English** (11 female + 10 male available!)
|
||||||
|
2. **Additional languages** (Spanish, French, Italian, Japanese, Portuguese, Chinese, Hindi)
|
||||||
|
3. **More character variety** (different voice characteristics)
|
||||||
|
|
||||||
|
Would you like me to update the frontend to include more voices?
|
||||||
197
TESTING_GUIDE.md
Normal file
197
TESTING_GUIDE.md
Normal file
@@ -0,0 +1,197 @@
|
|||||||
|
# Testing Guide for AI-Powered Audiobook Features
|
||||||
|
|
||||||
|
## ✅ Services Status
|
||||||
|
|
||||||
|
Both services are now running:
|
||||||
|
- **Backend**: http://localhost:8002
|
||||||
|
- **Frontend**: http://localhost:5173
|
||||||
|
- **Ollama**: http://localhost:11434 (required for AI features)
|
||||||
|
|
||||||
|
## 🎯 How to Test AI Features
|
||||||
|
|
||||||
|
### Test 1: AI Speaker Detection (No Markers Needed!)
|
||||||
|
|
||||||
|
Open the frontend at http://localhost:5173 and try this text:
|
||||||
|
|
||||||
|
```
|
||||||
|
The old wizard Merlin gazed into his crystal ball with concern. Dark times are coming, he whispered to himself.
|
||||||
|
|
||||||
|
A young knight named Arthur entered the chamber. Master Merlin, I heard your summons, he said with determination.
|
||||||
|
|
||||||
|
Merlin turned to face him. Arthur, the kingdom needs you now more than ever.
|
||||||
|
|
||||||
|
Arthur drew his sword. I am ready to defend our people.
|
||||||
|
|
||||||
|
From the shadows, the sorceress Morgana watched silently. She knew the prophecy would unfold exactly as foretold.
|
||||||
|
```
|
||||||
|
|
||||||
|
**What should happen:**
|
||||||
|
1. Click "Next: Assign voices"
|
||||||
|
2. The AI should detect: **Merlin**, **Arthur**, **Morgana** (and possibly Narrator)
|
||||||
|
3. Assign different voices to each character
|
||||||
|
4. Click "Generate Audiobook"
|
||||||
|
5. The AI will intelligently split the text so:
|
||||||
|
- Narrative descriptions → Narrator voice
|
||||||
|
- Merlin's dialogue → Merlin's voice
|
||||||
|
- Arthur's dialogue → Arthur's voice
|
||||||
|
- Morgana's thoughts → Morgana's voice
|
||||||
|
|
||||||
|
### Test 2: Mixed Marked and Unmarked Text
|
||||||
|
|
||||||
|
Try this text that combines explicit markers with unmarked dialogue:
|
||||||
|
|
||||||
|
```
|
||||||
|
Alice: Hello there!
|
||||||
|
Bob: Hi Alice, how are you?
|
||||||
|
[Narrator] They continued talking in the garden.
|
||||||
|
Charlie joined them later and smiled warmly.
|
||||||
|
```
|
||||||
|
|
||||||
|
**What should happen:**
|
||||||
|
- AI detects: Alice, Bob, Narrator, Charlie
|
||||||
|
- Pattern matching catches Alice, Bob, Narrator from markers
|
||||||
|
- AI catches Charlie from the unmarked text
|
||||||
|
|
||||||
|
### Test 3: Pure Narrative (No Dialogue)
|
||||||
|
|
||||||
|
Try this descriptive text:
|
||||||
|
|
||||||
|
```
|
||||||
|
The sun set over the mountains. Birds flew across the orange sky. A gentle breeze rustled through the trees. Peace settled over the valley.
|
||||||
|
```
|
||||||
|
|
||||||
|
**What should happen:**
|
||||||
|
- AI detects: Narrator (default for pure narrative)
|
||||||
|
- All text assigned to Narrator voice
|
||||||
|
|
||||||
|
### Test 4: Complex Dialogue Scene
|
||||||
|
|
||||||
|
Try this more complex scene:
|
||||||
|
|
||||||
|
```
|
||||||
|
The detective examined the evidence carefully. This doesn't add up, she muttered.
|
||||||
|
|
||||||
|
Her partner looked over her shoulder. What did you find?
|
||||||
|
|
||||||
|
She pointed at the photograph. Look at the timestamp. This photo was taken after the victim was already dead.
|
||||||
|
|
||||||
|
The partner's eyes widened. That means someone is lying.
|
||||||
|
|
||||||
|
Exactly, the detective replied. And I think I know who.
|
||||||
|
```
|
||||||
|
|
||||||
|
**What should happen:**
|
||||||
|
- AI detects: Detective, Partner (and possibly Narrator)
|
||||||
|
- Intelligently assigns dialogue to each character
|
||||||
|
- Narrative descriptions go to Narrator
|
||||||
|
|
||||||
|
## 🔍 Verification
|
||||||
|
|
||||||
|
### Check Backend Logs
|
||||||
|
|
||||||
|
The backend logs will show which detection method is being used:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Look for these log messages:
|
||||||
|
- "AI detection found X roles"
|
||||||
|
- "Pattern detection found X roles"
|
||||||
|
- "Using AI-powered segmentation"
|
||||||
|
- "Using pattern-based segmentation as fallback"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Test via API Directly
|
||||||
|
|
||||||
|
You can also test the API directly:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Test role detection
|
||||||
|
curl -X POST http://localhost:8002/api/tts/roles \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"text": "The wizard spoke. The knight replied."}'
|
||||||
|
|
||||||
|
# Expected: Detects "Wizard" and "Knight" without any markers!
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🎭 Voice Options Available
|
||||||
|
|
||||||
|
When assigning voices, you have these options:
|
||||||
|
- **af_bella** - Female, American
|
||||||
|
- **af_sarah** - Female, American
|
||||||
|
- **am_adam** - Male, American
|
||||||
|
- **am_michael** - Male, American
|
||||||
|
- **bf_emma** - Female, British
|
||||||
|
- **bf_isabella** - Female, British
|
||||||
|
- **bm_george** - Male, British
|
||||||
|
- **bm_lewis** - Male, British
|
||||||
|
|
||||||
|
## 🚨 Troubleshooting
|
||||||
|
|
||||||
|
### If AI detection isn't working:
|
||||||
|
|
||||||
|
1. **Check Ollama is running:**
|
||||||
|
```bash
|
||||||
|
curl http://localhost:11434/api/tags
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Check llama3 model is available:**
|
||||||
|
```bash
|
||||||
|
ollama list | grep llama3
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **If model is missing:**
|
||||||
|
```bash
|
||||||
|
ollama pull llama3
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Restart Ollama:**
|
||||||
|
```bash
|
||||||
|
# Stop Ollama (Ctrl+C if running in terminal)
|
||||||
|
ollama serve
|
||||||
|
```
|
||||||
|
|
||||||
|
### If backend isn't responding:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check if backend is running
|
||||||
|
curl http://localhost:8002/api/health
|
||||||
|
|
||||||
|
# If not, restart it
|
||||||
|
cd backend
|
||||||
|
source venv/bin/activate
|
||||||
|
uvicorn main:app --reload --host 0.0.0.0 --port 8002
|
||||||
|
```
|
||||||
|
|
||||||
|
### If frontend isn't loading:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check if frontend is running
|
||||||
|
curl http://localhost:5173
|
||||||
|
|
||||||
|
# If not, restart it
|
||||||
|
cd frontend
|
||||||
|
npm run dev
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🎉 What Makes This Powerful
|
||||||
|
|
||||||
|
**Before (Pattern-only):**
|
||||||
|
- Required explicit markers like "Alice:" or "[Narrator]"
|
||||||
|
- Unmarked text all went to Narrator
|
||||||
|
- Manual markup was tedious
|
||||||
|
|
||||||
|
**After (AI-powered):**
|
||||||
|
- ✨ Detects characters automatically from context
|
||||||
|
- ✨ Intelligently assigns dialogue to speakers
|
||||||
|
- ✨ Handles narrative and dialogue seamlessly
|
||||||
|
- ✨ No manual markup needed!
|
||||||
|
- ✨ Still supports explicit markers as fallback
|
||||||
|
|
||||||
|
## 📝 Tips for Best Results
|
||||||
|
|
||||||
|
1. **Use clear character names** in the narrative (e.g., "The wizard Merlin" instead of just "He")
|
||||||
|
2. **Include dialogue tags** like "he said", "she replied" to help AI identify speakers
|
||||||
|
3. **Keep paragraphs focused** on one speaker at a time when possible
|
||||||
|
4. **Mix narrative and dialogue** for more natural audiobooks
|
||||||
|
5. **Test with short samples first** to verify voice assignments before generating long audiobooks
|
||||||
|
|
||||||
|
Enjoy your powerful AI-driven audiobook generator! 🎙️✨
|
||||||
63
Test Story.txt
Normal file
63
Test Story.txt
Normal file
@@ -0,0 +1,63 @@
|
|||||||
|
Once upon a time, in a tiny village surrounded by lush green forests, there lived three best friends: Lily the rabbit, Benny the bear, and Sammy the squirrel. They loved to explore the woods together, playing hide-and-seek behind trees and chasing after butterflies.
|
||||||
|
|
||||||
|
One sunny afternoon, they decided to have a picnic on a hill overlooking the forest. They brought their favorite snacks – carrots for Lily, honey sandwiches for Benny, and acorn cookies for Sammy.
|
||||||
|
|
||||||
|
As they sat down on a blanket, Lily asked, "Have you guys heard about the new stream that just appeared in the woods?"
|
||||||
|
|
||||||
|
Benny's eyes lit up. "Ooh, I love streams! Do we get to go see it now?"
|
||||||
|
|
||||||
|
Sammy nodded enthusiastically. "I've been wanting to explore the forest more since the snow melted. Let's go!"
|
||||||
|
|
||||||
|
Lily grinned. "But first, let's finish our snacks and enjoy the view. We can go see the stream after that."
|
||||||
|
|
||||||
|
As they munched on their treats, Benny started telling a joke: "Why did the bear go to the party?"
|
||||||
|
|
||||||
|
Sammy chuckled. "I don't know, why?"
|
||||||
|
|
||||||
|
Benny laughed. "Because he was grizzly good at dancing!"
|
||||||
|
|
||||||
|
Lily giggled. "Oh, Benny, you're such a pun master!"
|
||||||
|
|
||||||
|
After finishing their snacks, they packed up the remains and set off towards the stream. As they walked, Sammy pointed out different types of leaves and flowers growing along the path.
|
||||||
|
|
||||||
|
Benny asked, "Sammy, how do you know so much about plants?"
|
||||||
|
|
||||||
|
Sammy beamed with pride. "My mom is a great botanist. She taught me all about the different species in our forest."
|
||||||
|
|
||||||
|
Lily nodded. "I've seen her collecting samples and making notes before. She's very smart!"
|
||||||
|
|
||||||
|
As they approached the stream, Benny exclaimed, "Wow! Look at that! It's so clear and sparkly!"
|
||||||
|
|
||||||
|
Sammy gasped. "And look, there are fish swimming in it!"
|
||||||
|
|
||||||
|
Lily clapped her paws together. "This is going to be so much fun! Let's catch some fish for lunch!"
|
||||||
|
|
||||||
|
Benny and Sammy cheered as they started skipping stones across the stream. Lily laughed as she tried to catch the fish with a makeshift net made from vines.
|
||||||
|
|
||||||
|
As they played, Benny noticed something strange. "Guys, look up there."
|
||||||
|
|
||||||
|
Sammy followed his gaze and gasped. A small waterfall cascaded down a rocky cliff, creating a misty veil that surrounded them.
|
||||||
|
|
||||||
|
Lily's eyes widened. "This is amazing! I've never seen anything like it!"
|
||||||
|
|
||||||
|
Benny grinned. "I think we should name this place the 'Misty Grotto'!"
|
||||||
|
|
||||||
|
Sammy nodded enthusiastically. "And we can come back and explore more tomorrow!"
|
||||||
|
|
||||||
|
As the sun began to set, they reluctantly said their goodbyes to the stream and headed back to the village. As they walked, Lily asked, "Benny, what's your favorite thing about our forest?"
|
||||||
|
|
||||||
|
Benny thought for a moment before responding, "I think it's all the hidden treasures we find along the way. Like this stream or that waterfall."
|
||||||
|
|
||||||
|
Sammy nodded in agreement. "And there are always new things to discover! That's what I love most about our forest too."
|
||||||
|
|
||||||
|
Lily smiled. "For me, it's the friends I make and the adventures we have together. You guys are my favorite treasures!"
|
||||||
|
|
||||||
|
As they approached the village, Benny turned to his friends and said, "You know what? Let's plan a camping trip for next weekend! We can bring sleeping bags and roast marshmallows over a campfire!"
|
||||||
|
|
||||||
|
Sammy's eyes lit up. "That sounds like so much fun! Can we invite some other villagers too?"
|
||||||
|
|
||||||
|
Lily nodded. "I'm sure Mom would love to come. She's always telling me about the camping trips she went on when she was younger."
|
||||||
|
|
||||||
|
Benny grinned. "It'll be a blast! Let's get planning and make it the best camping trip ever!"
|
||||||
|
|
||||||
|
As they entered their cozy little homes, Lily, Benny, and Sammy all smiled, knowing that they had each other and the magical forest to explore, filled with hidden treasures and endless adventures waiting to be discovered.
|
||||||
@@ -1,2 +1,3 @@
|
|||||||
PORT=8000
|
PORT=8000
|
||||||
LOG_LEVEL=info
|
LOG_LEVEL=info
|
||||||
|
OLLAMA_BASE_URL=http://localhost:11434
|
||||||
|
|||||||
@@ -62,7 +62,43 @@ pip install --upgrade pip
|
|||||||
pip install -r requirements.txt
|
pip install -r requirements.txt
|
||||||
```
|
```
|
||||||
|
|
||||||
### 3. Environment Configuration
|
### 3. Ollama Setup (Optional but Recommended)
|
||||||
|
|
||||||
|
For AI-powered speaker detection and text segmentation, install Ollama:
|
||||||
|
|
||||||
|
**Installation:**
|
||||||
|
- Visit [ollama.com](https://ollama.com) and download for your platform
|
||||||
|
- Or use package managers:
|
||||||
|
```bash
|
||||||
|
# macOS
|
||||||
|
brew install ollama
|
||||||
|
|
||||||
|
# Linux
|
||||||
|
curl -fsSL https://ollama.com/install.sh | sh
|
||||||
|
```
|
||||||
|
|
||||||
|
**Pull the required model:**
|
||||||
|
```bash
|
||||||
|
ollama pull llama3
|
||||||
|
```
|
||||||
|
|
||||||
|
**Start Ollama service:**
|
||||||
|
```bash
|
||||||
|
ollama serve
|
||||||
|
```
|
||||||
|
|
||||||
|
Verify Ollama is running:
|
||||||
|
```bash
|
||||||
|
curl http://localhost:11434/api/tags
|
||||||
|
```
|
||||||
|
|
||||||
|
**Benefits of using Ollama:**
|
||||||
|
- ✨ Intelligently detects speakers in unmarked text
|
||||||
|
- ✨ Automatically segments dialogue and assigns to correct characters
|
||||||
|
- ✨ Handles narrative text without explicit role markers
|
||||||
|
- ✨ Falls back to pattern matching if unavailable
|
||||||
|
|
||||||
|
### 4. Environment Configuration
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cp .env.example .env
|
cp .env.example .env
|
||||||
@@ -72,9 +108,10 @@ Edit `.env` if needed:
|
|||||||
```bash
|
```bash
|
||||||
PORT=8000
|
PORT=8000
|
||||||
LOG_LEVEL=info
|
LOG_LEVEL=info
|
||||||
|
OLLAMA_BASE_URL=http://localhost:11434 # Optional: Ollama API URL
|
||||||
```
|
```
|
||||||
|
|
||||||
### 4. Run the Server
|
### 5. Run the Server
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
uvicorn main:app --reload --host 0.0.0.0 --port 8000
|
uvicorn main:app --reload --host 0.0.0.0 --port 8000
|
||||||
@@ -162,7 +199,10 @@ curl http://localhost:8000/api/health
|
|||||||
|
|
||||||
### POST /api/tts/roles
|
### POST /api/tts/roles
|
||||||
|
|
||||||
Detect distinct roles/speakers in text.
|
Detect distinct roles/speakers in text using AI-powered detection.
|
||||||
|
|
||||||
|
**AI-Powered Detection:**
|
||||||
|
When Ollama is available, the system uses AI (llama3) to intelligently detect speakers in unmarked text. This allows you to input plain narrative text without explicit role markers, and the AI will identify all characters and speakers.
|
||||||
|
|
||||||
**Request Body:**
|
**Request Body:**
|
||||||
```json
|
```json
|
||||||
@@ -182,11 +222,21 @@ Detect distinct roles/speakers in text.
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
**Supported Role Patterns:**
|
**Detection Methods:**
|
||||||
- Colon pattern: `RoleName: dialogue text`
|
1. **AI Detection (Primary)**: Uses Ollama with llama3 to intelligently identify speakers in unmarked text
|
||||||
- Bracket pattern: `[RoleName] dialogue text`
|
- Example: "The wizard spoke softly. The knight replied." → Detects "Wizard" and "Knight"
|
||||||
|
2. **Pattern Matching (Fallback)**: Falls back to pattern-based detection if Ollama unavailable
|
||||||
|
- Colon pattern: `RoleName: dialogue text`
|
||||||
|
- Bracket pattern: `[RoleName] dialogue text`
|
||||||
|
|
||||||
**Example:**
|
**Example with AI Detection:**
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:8000/api/tts/roles \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"text":"The wizard gazed into his crystal ball. The knight drew his sword."}'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Example with Pattern Markers:**
|
||||||
```bash
|
```bash
|
||||||
curl -X POST http://localhost:8000/api/tts/roles \
|
curl -X POST http://localhost:8000/api/tts/roles \
|
||||||
-H "Content-Type: application/json" \
|
-H "Content-Type: application/json" \
|
||||||
@@ -195,7 +245,13 @@ curl -X POST http://localhost:8000/api/tts/roles \
|
|||||||
|
|
||||||
### POST /api/tts
|
### POST /api/tts
|
||||||
|
|
||||||
Generate multi-voice audiobook.
|
Generate multi-voice audiobook with AI-powered text segmentation.
|
||||||
|
|
||||||
|
**AI-Powered Segmentation:**
|
||||||
|
When Ollama is available, the system uses AI to intelligently split text and assign each segment to the appropriate speaker. This means you can input plain narrative text, and the AI will automatically:
|
||||||
|
- Assign dialogue to the correct character
|
||||||
|
- Assign narrative descriptions to the Narrator
|
||||||
|
- Handle mixed dialogue and narration seamlessly
|
||||||
|
|
||||||
**Request Body:**
|
**Request Body:**
|
||||||
```json
|
```json
|
||||||
@@ -211,13 +267,21 @@ Generate multi-voice audiobook.
|
|||||||
```
|
```
|
||||||
|
|
||||||
**Parameters:**
|
**Parameters:**
|
||||||
- `text` (required): Text to synthesize
|
- `text` (required): Text to synthesize (can be plain narrative or marked with role patterns)
|
||||||
- `lang` (required): Language code (a=American English, b=British English, f=French, j=Japanese)
|
- `lang` (required): Language code (a=American English, b=British English, f=French, j=Japanese)
|
||||||
- `roles` (required): Array of role-to-voice mappings
|
- `roles` (required): Array of role-to-voice mappings
|
||||||
- `speed` (optional): Playback speed (0.5-1.5, default: 1.0)
|
- `speed` (optional): Playback speed (0.5-1.5, default: 1.0)
|
||||||
|
|
||||||
**Response:**
|
**Response:**
|
||||||
- Content-Type: `audio/wav`
|
- Content-Type: `audio/wav`
|
||||||
|
- Content-Disposition: `attachment; filename=audiobook.wav`
|
||||||
|
|
||||||
|
**Segmentation Methods:**
|
||||||
|
1. **AI Segmentation (Primary)**: Uses Ollama to intelligently split text and assign speakers
|
||||||
|
- Handles unmarked dialogue and narrative
|
||||||
|
- Automatically assigns dialogue to characters and descriptions to Narrator
|
||||||
|
2. **Pattern Segmentation (Fallback)**: Uses explicit role markers when AI unavailable
|
||||||
|
- Requires `RoleName:` or `[RoleName]` markers in text
|
||||||
- Content-Disposition: `attachment; filename="audiobook.wav"`
|
- Content-Disposition: `attachment; filename="audiobook.wav"`
|
||||||
- Body: Audio file bytes
|
- Body: Audio file bytes
|
||||||
|
|
||||||
|
|||||||
10
backend/check_voices.py
Normal file
10
backend/check_voices.py
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Check available Kokoro TTS voices."""
|
||||||
|
from kokoro import VOICE_NAMES
|
||||||
|
|
||||||
|
print("Available Kokoro TTS voices:")
|
||||||
|
print("=" * 50)
|
||||||
|
for voice in sorted(VOICE_NAMES):
|
||||||
|
print(f" {voice}")
|
||||||
|
print("=" * 50)
|
||||||
|
print(f"Total: {len(VOICE_NAMES)} voices")
|
||||||
@@ -62,7 +62,7 @@ app = FastAPI(
|
|||||||
# Configure CORS
|
# Configure CORS
|
||||||
app.add_middleware(
|
app.add_middleware(
|
||||||
CORSMiddleware,
|
CORSMiddleware,
|
||||||
allow_origins=["http://localhost:5173", "http://localhost:3000"],
|
allow_origins=["http://localhost:5173", "http://localhost:5174", "http://localhost:3000"],
|
||||||
allow_credentials=True,
|
allow_credentials=True,
|
||||||
allow_methods=["*"],
|
allow_methods=["*"],
|
||||||
allow_headers=["*"],
|
allow_headers=["*"],
|
||||||
@@ -111,6 +111,8 @@ async def detect_roles(request: RoleDetectionRequest):
|
|||||||
"""
|
"""
|
||||||
Analyzes text and returns detected speaker roles.
|
Analyzes text and returns detected speaker roles.
|
||||||
|
|
||||||
|
Uses AI-based detection (Ollama) first, then falls back to pattern matching.
|
||||||
|
|
||||||
Supports two pattern types:
|
Supports two pattern types:
|
||||||
- Colon pattern: "RoleName:" at line start
|
- Colon pattern: "RoleName:" at line start
|
||||||
- Bracket pattern: "[RoleName]" preceding dialogue
|
- Bracket pattern: "[RoleName]" preceding dialogue
|
||||||
@@ -120,7 +122,7 @@ async def detect_roles(request: RoleDetectionRequest):
|
|||||||
logger.info(f"Role detection requested for text of length {len(request.text)}")
|
logger.info(f"Role detection requested for text of length {len(request.text)}")
|
||||||
|
|
||||||
try:
|
try:
|
||||||
roles = role_detector.detect_roles(request.text)
|
roles = await role_detector.detect_roles(request.text)
|
||||||
logger.info(f"Detected {len(roles)} roles: {[r['label'] for r in roles]}")
|
logger.info(f"Detected {len(roles)} roles: {[r['label'] for r in roles]}")
|
||||||
return RolesResponse(roles=roles)
|
return RolesResponse(roles=roles)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
@@ -163,7 +165,7 @@ async def generate_audiobook(request: TTSRequest):
|
|||||||
)
|
)
|
||||||
|
|
||||||
# Validate language code
|
# Validate language code
|
||||||
supported_langs = ['a', 'b', 'f', 'j']
|
supported_langs = ['a', 'b', 'e', 'f', 'h', 'i', 'j', 'p', 'z']
|
||||||
if request.lang not in supported_langs:
|
if request.lang not in supported_langs:
|
||||||
logger.error(f"Invalid language code: {request.lang}")
|
logger.error(f"Invalid language code: {request.lang}")
|
||||||
return JSONResponse(
|
return JSONResponse(
|
||||||
|
|||||||
@@ -5,3 +5,4 @@ soundfile==0.12.1
|
|||||||
python-multipart==0.0.6
|
python-multipart==0.0.6
|
||||||
pydantic==2.5.3
|
pydantic==2.5.3
|
||||||
numpy==1.26.3
|
numpy==1.26.3
|
||||||
|
httpx==0.26.0
|
||||||
|
|||||||
@@ -29,3 +29,27 @@ INFO: 127.0.0.1:53090 - "POST /api/tts HTTP/1.1" 400 Bad Request
|
|||||||
2025-12-09 01:20:19,494 - services.tts_service - INFO - Generated audiobook: 44 bytes
|
2025-12-09 01:20:19,494 - services.tts_service - INFO - Generated audiobook: 44 bytes
|
||||||
WARNING: Defaulting repo_id to hexgrad/Kokoro-82M. Pass repo_id='hexgrad/Kokoro-82M' to suppress this warning.
|
WARNING: Defaulting repo_id to hexgrad/Kokoro-82M. Pass repo_id='hexgrad/Kokoro-82M' to suppress this warning.
|
||||||
INFO: 127.0.0.1:53092 - "POST /api/tts HTTP/1.1" 200 OK
|
INFO: 127.0.0.1:53092 - "POST /api/tts HTTP/1.1" 200 OK
|
||||||
|
INFO: 127.0.0.1:54291 - "OPTIONS /api/tts/roles HTTP/1.1" 200 OK
|
||||||
|
2025-12-09 10:15:45,219 - main - INFO - Role detection requested for text of length 4
|
||||||
|
2025-12-09 10:15:45,226 - main - INFO - Detected 1 roles: ['Narrator']
|
||||||
|
INFO: 127.0.0.1:54291 - "POST /api/tts/roles HTTP/1.1" 200 OK
|
||||||
|
INFO: 127.0.0.1:54296 - "OPTIONS /api/tts HTTP/1.1" 200 OK
|
||||||
|
2025-12-09 10:15:51,949 - main - INFO - Audio generation requested: lang=a, roles=1, speed=1.0
|
||||||
|
2025-12-09 10:15:51,953 - services.tts_service - INFO - Generating audiobook: lang=a, speed=1.0, roles=1
|
||||||
|
2025-12-09 10:15:51,955 - services.tts_service - INFO - Segmented text into 1 segments
|
||||||
|
Fatal Python error: init_sys_streams: can't initialize sys standard streams
|
||||||
|
Python runtime state: core initialized
|
||||||
|
OSError: [Errno 9] Bad file descriptor
|
||||||
|
|
||||||
|
Current thread 0x00000001f799e080 (most recent call first):
|
||||||
|
<no Python frame>
|
||||||
|
2025-12-09 10:15:55,293 - services.tts_service - INFO - Generated audiobook: 44 bytes
|
||||||
|
INFO: 127.0.0.1:54296 - "POST /api/tts HTTP/1.1" 200 OK
|
||||||
|
2025-12-09 10:16:52,356 - main - INFO - Role detection requested for text of length 40
|
||||||
|
2025-12-09 10:16:52,358 - main - INFO - Detected 1 roles: ['Narrator']
|
||||||
|
INFO: 127.0.0.1:54431 - "POST /api/tts/roles HTTP/1.1" 200 OK
|
||||||
|
2025-12-09 10:17:02,016 - main - INFO - Audio generation requested: lang=a, roles=1, speed=1.0
|
||||||
|
2025-12-09 10:17:02,017 - services.tts_service - INFO - Generating audiobook: lang=a, speed=1.0, roles=1
|
||||||
|
2025-12-09 10:17:02,018 - services.tts_service - INFO - Segmented text into 1 segments
|
||||||
|
2025-12-09 10:17:02,814 - services.tts_service - INFO - Generated audiobook: 44 bytes
|
||||||
|
INFO: 127.0.0.1:54433 - "POST /api/tts HTTP/1.1" 200 OK
|
||||||
|
|||||||
369
backend/services/ollama_service.py
Normal file
369
backend/services/ollama_service.py
Normal file
@@ -0,0 +1,369 @@
|
|||||||
|
"""
|
||||||
|
Ollama integration service for AI-based speaker detection and text segmentation.
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
from typing import List, Optional, Dict, Tuple
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class OllamaService:
|
||||||
|
"""
|
||||||
|
Service for integrating with Ollama API for AI-based speaker detection.
|
||||||
|
|
||||||
|
Uses llama3:8b model to intelligently detect speakers in text without
|
||||||
|
explicit role markers.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, base_url: Optional[str] = None):
|
||||||
|
"""
|
||||||
|
Initializes Ollama service with base URL.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
base_url: Ollama API base URL. Defaults to OLLAMA_BASE_URL env var
|
||||||
|
or http://localhost:11434 if not set.
|
||||||
|
"""
|
||||||
|
self.base_url = base_url or os.getenv("OLLAMA_BASE_URL", "http://localhost:11434")
|
||||||
|
self.model = "llama3:latest" # Using llama3:latest (8B model)
|
||||||
|
self.timeout = 30.0 # 30 seconds timeout
|
||||||
|
|
||||||
|
async def detect_speakers(self, text: str) -> List[str]:
|
||||||
|
"""
|
||||||
|
Sends text to Ollama for speaker detection.
|
||||||
|
Samples up to 4 chunks of 5000 characters from different parts of the text
|
||||||
|
to ensure comprehensive speaker detection in long texts.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
text: Input text to analyze for speakers
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of detected speaker names (deduplicated across all samples)
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
Exception: If Ollama is unavailable or request fails
|
||||||
|
"""
|
||||||
|
# Get text samples from different parts of the text
|
||||||
|
samples = self._sample_text(text, chunk_size=5000, num_samples=4)
|
||||||
|
|
||||||
|
all_speakers = set()
|
||||||
|
|
||||||
|
try:
|
||||||
|
async with httpx.AsyncClient(timeout=self.timeout) as client:
|
||||||
|
# Process each sample
|
||||||
|
for i, sample in enumerate(samples):
|
||||||
|
prompt = self._build_prompt(sample)
|
||||||
|
|
||||||
|
logger.debug(f"Sending sample {i+1}/{len(samples)} to Ollama ({len(sample)} chars)")
|
||||||
|
|
||||||
|
response = await client.post(
|
||||||
|
f"{self.base_url}/api/generate",
|
||||||
|
json={
|
||||||
|
"model": self.model,
|
||||||
|
"prompt": prompt,
|
||||||
|
"stream": False,
|
||||||
|
"format": "json"
|
||||||
|
}
|
||||||
|
)
|
||||||
|
response.raise_for_status()
|
||||||
|
|
||||||
|
result = response.json()
|
||||||
|
response_text = result.get("response", "")
|
||||||
|
|
||||||
|
speakers = self._parse_response(response_text)
|
||||||
|
logger.info(f"Sample {i+1} detected {len(speakers)} speakers: {speakers}")
|
||||||
|
|
||||||
|
# Add to our set (automatically deduplicates)
|
||||||
|
all_speakers.update(speakers)
|
||||||
|
|
||||||
|
speaker_list = sorted(list(all_speakers))
|
||||||
|
logger.info(f"Ollama detected {len(speaker_list)} total unique speakers: {speaker_list}")
|
||||||
|
return speaker_list
|
||||||
|
|
||||||
|
except httpx.TimeoutException:
|
||||||
|
logger.warning("Ollama request timed out after 30 seconds")
|
||||||
|
raise Exception("Ollama request timed out")
|
||||||
|
except httpx.HTTPError as e:
|
||||||
|
logger.warning(f"Ollama HTTP error: {e}")
|
||||||
|
raise Exception(f"Ollama service error: {e}")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Ollama detection failed: {e}")
|
||||||
|
raise
|
||||||
|
|
||||||
|
def _sample_text(self, text: str, chunk_size: int = 5000, num_samples: int = 4) -> List[str]:
|
||||||
|
"""
|
||||||
|
Samples text from different parts to ensure comprehensive speaker detection.
|
||||||
|
|
||||||
|
For short texts, returns the full text.
|
||||||
|
For long texts, samples from beginning, middle sections, and end.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
text: Input text to sample
|
||||||
|
chunk_size: Size of each sample in characters
|
||||||
|
num_samples: Number of samples to extract
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of text samples
|
||||||
|
"""
|
||||||
|
text_len = len(text)
|
||||||
|
|
||||||
|
# If text is shorter than one chunk, return it as-is
|
||||||
|
if text_len <= chunk_size:
|
||||||
|
return [text]
|
||||||
|
|
||||||
|
# If text is shorter than what we'd sample, return fewer samples
|
||||||
|
max_possible_samples = (text_len // chunk_size) + 1
|
||||||
|
actual_samples = min(num_samples, max_possible_samples)
|
||||||
|
|
||||||
|
samples = []
|
||||||
|
|
||||||
|
if actual_samples == 1:
|
||||||
|
# Just take the beginning
|
||||||
|
samples.append(text[:chunk_size])
|
||||||
|
else:
|
||||||
|
# Distribute samples evenly across the text
|
||||||
|
step = text_len // actual_samples
|
||||||
|
|
||||||
|
for i in range(actual_samples):
|
||||||
|
start = i * step
|
||||||
|
end = min(start + chunk_size, text_len)
|
||||||
|
samples.append(text[start:end])
|
||||||
|
|
||||||
|
return samples
|
||||||
|
|
||||||
|
def _build_prompt(self, text: str) -> str:
|
||||||
|
"""
|
||||||
|
Constructs prompt for speaker detection.
|
||||||
|
|
||||||
|
Instructs model to return JSON array of speaker names.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
text: Input text to analyze (should already be sampled/truncated)
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Formatted prompt string
|
||||||
|
"""
|
||||||
|
prompt = f"""Analyze the following text and identify all distinct speakers or characters.
|
||||||
|
Return ONLY a JSON object with a "speakers" array containing the names of speakers.
|
||||||
|
If there are no clear speakers, return {{"speakers": ["Narrator"]}}.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
- For dialogue with character names, list those characters
|
||||||
|
- For narrative text, use "Narrator"
|
||||||
|
- For mixed content, include both characters and "Narrator" if appropriate
|
||||||
|
|
||||||
|
Text to analyze:
|
||||||
|
{text}
|
||||||
|
|
||||||
|
Return format: {{"speakers": ["Speaker1", "Speaker2", ...]}}
|
||||||
|
"""
|
||||||
|
return prompt
|
||||||
|
|
||||||
|
def _parse_response(self, response: str) -> List[str]:
|
||||||
|
"""
|
||||||
|
Parses Ollama response to extract speaker names.
|
||||||
|
|
||||||
|
Handles both JSON and plain text responses.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
response: Response text from Ollama
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of speaker names
|
||||||
|
"""
|
||||||
|
if not response or not response.strip():
|
||||||
|
return []
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Try to parse as JSON
|
||||||
|
data = json.loads(response.strip())
|
||||||
|
|
||||||
|
# Handle different possible JSON structures
|
||||||
|
if isinstance(data, dict):
|
||||||
|
if "speakers" in data:
|
||||||
|
speakers = data["speakers"]
|
||||||
|
elif "characters" in data:
|
||||||
|
speakers = data["characters"]
|
||||||
|
elif "roles" in data:
|
||||||
|
speakers = data["roles"]
|
||||||
|
else:
|
||||||
|
# Try to find any array in the response
|
||||||
|
for value in data.values():
|
||||||
|
if isinstance(value, list):
|
||||||
|
speakers = value
|
||||||
|
break
|
||||||
|
else:
|
||||||
|
return []
|
||||||
|
elif isinstance(data, list):
|
||||||
|
speakers = data
|
||||||
|
else:
|
||||||
|
return []
|
||||||
|
|
||||||
|
# Ensure we have a list of strings
|
||||||
|
if isinstance(speakers, list):
|
||||||
|
return [str(s).strip() for s in speakers if s]
|
||||||
|
|
||||||
|
return []
|
||||||
|
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
logger.warning(f"Failed to parse Ollama response as JSON: {response[:100]}")
|
||||||
|
# Try to extract speaker names from plain text
|
||||||
|
# Look for common patterns like lists or quoted names
|
||||||
|
return []
|
||||||
|
|
||||||
|
async def segment_text(self, text: str, speakers: List[str]) -> List[Tuple[str, str]]:
|
||||||
|
"""
|
||||||
|
Uses AI to segment text and assign each segment to a speaker.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
text: Input text to segment
|
||||||
|
speakers: List of known speakers in the text
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of tuples: (speaker_name, text_segment)
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
Exception: If Ollama is unavailable or request fails
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
async with httpx.AsyncClient(timeout=60.0) as client: # Longer timeout for segmentation
|
||||||
|
prompt = self._build_segmentation_prompt(text, speakers)
|
||||||
|
|
||||||
|
logger.info(f"Requesting AI segmentation for {len(text)} chars with {len(speakers)} speakers")
|
||||||
|
|
||||||
|
response = await client.post(
|
||||||
|
f"{self.base_url}/api/generate",
|
||||||
|
json={
|
||||||
|
"model": self.model,
|
||||||
|
"prompt": prompt,
|
||||||
|
"stream": False,
|
||||||
|
"format": "json"
|
||||||
|
}
|
||||||
|
)
|
||||||
|
response.raise_for_status()
|
||||||
|
|
||||||
|
result = response.json()
|
||||||
|
response_text = result.get("response", "")
|
||||||
|
|
||||||
|
segments = self._parse_segmentation_response(response_text)
|
||||||
|
logger.info(f"AI segmentation produced {len(segments)} segments")
|
||||||
|
|
||||||
|
return segments
|
||||||
|
|
||||||
|
except httpx.TimeoutException:
|
||||||
|
logger.warning("Ollama segmentation request timed out")
|
||||||
|
raise Exception("Ollama segmentation timed out")
|
||||||
|
except httpx.HTTPError as e:
|
||||||
|
logger.warning(f"Ollama HTTP error during segmentation: {e}")
|
||||||
|
raise Exception(f"Ollama service error: {e}")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Ollama segmentation failed: {e}")
|
||||||
|
raise
|
||||||
|
|
||||||
|
def _build_segmentation_prompt(self, text: str, speakers: List[str]) -> str:
|
||||||
|
"""
|
||||||
|
Constructs prompt for text segmentation with speaker attribution.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
text: Input text to segment
|
||||||
|
speakers: List of known speakers
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Formatted prompt string
|
||||||
|
"""
|
||||||
|
speakers_str = ", ".join(f'"{s}"' for s in speakers)
|
||||||
|
|
||||||
|
prompt = f"""You are analyzing a story or dialogue. Your task is to split the text into segments and assign each segment to the appropriate speaker.
|
||||||
|
|
||||||
|
Known speakers: {speakers_str}
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
1. Split the text into logical segments (sentences or short paragraphs)
|
||||||
|
2. Assign each segment to the speaker who is speaking or being described
|
||||||
|
3. Use "Narrator" for narrative text, descriptions, or text that doesn't belong to a specific character
|
||||||
|
4. If a character is mentioned but not speaking, use "Narrator"
|
||||||
|
5. Only use speakers from the known speakers list
|
||||||
|
|
||||||
|
Return a JSON object with a "segments" array. Each segment should have:
|
||||||
|
- "speaker": the name of the speaker (must be from the known speakers list or "Narrator")
|
||||||
|
- "text": the text content for that segment
|
||||||
|
|
||||||
|
Example format:
|
||||||
|
{{
|
||||||
|
"segments": [
|
||||||
|
{{"speaker": "Narrator", "text": "The old wizard gazed into his crystal ball."}},
|
||||||
|
{{"speaker": "Wizard", "text": "I see danger approaching."}},
|
||||||
|
{{"speaker": "Narrator", "text": "A young knight heard the warning."}}
|
||||||
|
]
|
||||||
|
}}
|
||||||
|
|
||||||
|
Text to segment:
|
||||||
|
{text}
|
||||||
|
|
||||||
|
Return ONLY the JSON object with the segments array.
|
||||||
|
"""
|
||||||
|
return prompt
|
||||||
|
|
||||||
|
def _parse_segmentation_response(self, response: str) -> List[Tuple[str, str]]:
|
||||||
|
"""
|
||||||
|
Parses Ollama segmentation response to extract speaker-text pairs.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
response: Response text from Ollama
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of tuples: (speaker_name, text_content)
|
||||||
|
"""
|
||||||
|
if not response or not response.strip():
|
||||||
|
return []
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Try to parse as JSON
|
||||||
|
data = json.loads(response.strip())
|
||||||
|
|
||||||
|
# Extract segments array
|
||||||
|
segments_data = []
|
||||||
|
if isinstance(data, dict):
|
||||||
|
if "segments" in data:
|
||||||
|
segments_data = data["segments"]
|
||||||
|
else:
|
||||||
|
# Try to find any array in the response
|
||||||
|
for value in data.values():
|
||||||
|
if isinstance(value, list):
|
||||||
|
segments_data = value
|
||||||
|
break
|
||||||
|
elif isinstance(data, list):
|
||||||
|
segments_data = data
|
||||||
|
|
||||||
|
# Convert to tuples
|
||||||
|
segments = []
|
||||||
|
for seg in segments_data:
|
||||||
|
if isinstance(seg, dict) and "speaker" in seg and "text" in seg:
|
||||||
|
speaker = str(seg["speaker"]).strip()
|
||||||
|
text = str(seg["text"]).strip()
|
||||||
|
if speaker and text:
|
||||||
|
segments.append((speaker, text))
|
||||||
|
|
||||||
|
return segments
|
||||||
|
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
logger.warning(f"Failed to parse Ollama segmentation response as JSON: {response[:200]}")
|
||||||
|
return []
|
||||||
|
|
||||||
|
async def is_available(self) -> bool:
|
||||||
|
"""
|
||||||
|
Checks if Ollama service is available.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
True if service responds to health check, False otherwise
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
async with httpx.AsyncClient(timeout=5.0) as client:
|
||||||
|
response = await client.get(f"{self.base_url}/api/tags")
|
||||||
|
return response.status_code == 200
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug(f"Ollama availability check failed: {e}")
|
||||||
|
return False
|
||||||
@@ -2,13 +2,20 @@
|
|||||||
Role detection service for identifying distinct speakers in text.
|
Role detection service for identifying distinct speakers in text.
|
||||||
"""
|
"""
|
||||||
import re
|
import re
|
||||||
from typing import List, Set, Dict
|
import logging
|
||||||
|
from typing import List, Set, Dict, Optional
|
||||||
|
from .ollama_service import OllamaService
|
||||||
|
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
class RoleDetector:
|
class RoleDetector:
|
||||||
"""
|
"""
|
||||||
Service for detecting distinct speaker roles in text.
|
Service for detecting distinct speaker roles in text.
|
||||||
|
|
||||||
|
Uses AI-based detection (Ollama) as primary method, with fallback to pattern matching.
|
||||||
|
|
||||||
Supports two pattern types:
|
Supports two pattern types:
|
||||||
1. Colon pattern: "RoleName:" at the start of a line
|
1. Colon pattern: "RoleName:" at the start of a line
|
||||||
2. Bracket pattern: "[RoleName]" preceding dialogue
|
2. Bracket pattern: "[RoleName]" preceding dialogue
|
||||||
@@ -19,10 +26,23 @@ class RoleDetector:
|
|||||||
BRACKET_PATTERN = r'\[([A-Z][a-zA-Z\s]+?)\]'
|
BRACKET_PATTERN = r'\[([A-Z][a-zA-Z\s]+?)\]'
|
||||||
DEFAULT_ROLE = "Narrator"
|
DEFAULT_ROLE = "Narrator"
|
||||||
|
|
||||||
def detect_roles(self, text: str) -> List[Dict[str, str]]:
|
def __init__(self, ollama_service: Optional[OllamaService] = None):
|
||||||
|
"""
|
||||||
|
Initializes RoleDetector with optional OllamaService.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
ollama_service: Optional OllamaService instance for AI-based detection.
|
||||||
|
If None, creates a new instance.
|
||||||
|
"""
|
||||||
|
self.ollama_service = ollama_service or OllamaService()
|
||||||
|
|
||||||
|
async def detect_roles(self, text: str) -> List[Dict[str, str]]:
|
||||||
"""
|
"""
|
||||||
Analyzes text to identify distinct speaker roles.
|
Analyzes text to identify distinct speaker roles.
|
||||||
|
|
||||||
|
Uses AI-based detection (Ollama) first, then falls back to pattern matching.
|
||||||
|
Merges and deduplicates results from both methods.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
text: Input text to analyze
|
text: Input text to analyze
|
||||||
|
|
||||||
@@ -33,15 +53,26 @@ class RoleDetector:
|
|||||||
if not text or not text.strip():
|
if not text or not text.strip():
|
||||||
return [{"id": self.DEFAULT_ROLE, "label": self.DEFAULT_ROLE}]
|
return [{"id": self.DEFAULT_ROLE, "label": self.DEFAULT_ROLE}]
|
||||||
|
|
||||||
# Extract roles from both patterns
|
all_roles = set()
|
||||||
|
|
||||||
|
# Try AI-based detection first
|
||||||
|
ollama_roles = await self._detect_with_ollama(text)
|
||||||
|
if ollama_roles:
|
||||||
|
all_roles.update(ollama_roles)
|
||||||
|
logger.info(f"AI detection found {len(ollama_roles)} roles: {sorted(ollama_roles)}")
|
||||||
|
|
||||||
|
# Always run pattern matching as well (for marked text or as fallback)
|
||||||
colon_roles = self._extract_colon_pattern(text)
|
colon_roles = self._extract_colon_pattern(text)
|
||||||
bracket_roles = self._extract_bracket_pattern(text)
|
bracket_roles = self._extract_bracket_pattern(text)
|
||||||
|
pattern_roles = colon_roles.union(bracket_roles)
|
||||||
|
|
||||||
# Combine and deduplicate
|
if pattern_roles:
|
||||||
all_roles = colon_roles.union(bracket_roles)
|
all_roles.update(pattern_roles)
|
||||||
|
logger.info(f"Pattern detection found {len(pattern_roles)} roles: {sorted(pattern_roles)}")
|
||||||
|
|
||||||
# If no roles found, return default
|
# If no roles found by either method, return default
|
||||||
if not all_roles:
|
if not all_roles:
|
||||||
|
logger.info("No roles detected, using default Narrator")
|
||||||
return [{"id": self.DEFAULT_ROLE, "label": self.DEFAULT_ROLE}]
|
return [{"id": self.DEFAULT_ROLE, "label": self.DEFAULT_ROLE}]
|
||||||
|
|
||||||
# Convert to role dictionaries and sort for consistent ordering
|
# Convert to role dictionaries and sort for consistent ordering
|
||||||
@@ -50,8 +81,40 @@ class RoleDetector:
|
|||||||
for role in sorted(all_roles)
|
for role in sorted(all_roles)
|
||||||
]
|
]
|
||||||
|
|
||||||
|
logger.info(f"Total unique roles after merging: {len(role_list)}")
|
||||||
return role_list
|
return role_list
|
||||||
|
|
||||||
|
async def _detect_with_ollama(self, text: str) -> Set[str]:
|
||||||
|
"""
|
||||||
|
Uses Ollama with llama3:8B to intelligently detect speakers.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
text: Input text to analyze
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Set of speaker names or empty set if Ollama unavailable/fails.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# Check if Ollama is available
|
||||||
|
is_available = await self.ollama_service.is_available()
|
||||||
|
if not is_available:
|
||||||
|
logger.info("Ollama service not available, skipping AI detection")
|
||||||
|
return set()
|
||||||
|
|
||||||
|
# Detect speakers using Ollama
|
||||||
|
speakers = await self.ollama_service.detect_speakers(text)
|
||||||
|
|
||||||
|
if speakers:
|
||||||
|
logger.info(f"Ollama detected speakers: {speakers}")
|
||||||
|
return set(speakers)
|
||||||
|
else:
|
||||||
|
logger.info("Ollama returned no speakers")
|
||||||
|
return set()
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Ollama detection failed: {e}, falling back to pattern matching")
|
||||||
|
return set()
|
||||||
|
|
||||||
def _extract_colon_pattern(self, text: str) -> Set[str]:
|
def _extract_colon_pattern(self, text: str) -> Set[str]:
|
||||||
"""
|
"""
|
||||||
Extracts roles from "RoleName:" pattern at line starts.
|
Extracts roles from "RoleName:" pattern at line starts.
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ from typing import Dict, List, Tuple, Optional
|
|||||||
import numpy as np
|
import numpy as np
|
||||||
import soundfile as sf
|
import soundfile as sf
|
||||||
from kokoro import KPipeline
|
from kokoro import KPipeline
|
||||||
|
from .ollama_service import OllamaService
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -30,9 +31,10 @@ class TTSService:
|
|||||||
COLON_PATTERN = r'^([A-Z][a-zA-Z\s]+?):\s*(.+)$'
|
COLON_PATTERN = r'^([A-Z][a-zA-Z\s]+?):\s*(.+)$'
|
||||||
BRACKET_PATTERN = r'^\[([A-Z][a-zA-Z\s]+?)\]\s*(.+)$'
|
BRACKET_PATTERN = r'^\[([A-Z][a-zA-Z\s]+?)\]\s*(.+)$'
|
||||||
|
|
||||||
def __init__(self):
|
def __init__(self, ollama_service: Optional[OllamaService] = None):
|
||||||
"""Initializes TTS service with empty pipeline cache."""
|
"""Initializes TTS service with empty pipeline cache and optional Ollama service."""
|
||||||
self.pipelines: Dict[str, KPipeline] = {}
|
self.pipelines: Dict[str, KPipeline] = {}
|
||||||
|
self.ollama_service = ollama_service or OllamaService()
|
||||||
logger.info("TTSService initialized")
|
logger.info("TTSService initialized")
|
||||||
|
|
||||||
def get_pipeline(self, lang_code: str) -> KPipeline:
|
def get_pipeline(self, lang_code: str) -> KPipeline:
|
||||||
@@ -41,7 +43,7 @@ class TTSService:
|
|||||||
Caches pipelines to avoid reloading.
|
Caches pipelines to avoid reloading.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
lang_code: Language code (a, b, f, j)
|
lang_code: Language code (a, b, e, f, h, i, j, p, z)
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
KPipeline instance for the language
|
KPipeline instance for the language
|
||||||
@@ -49,7 +51,7 @@ class TTSService:
|
|||||||
Raises:
|
Raises:
|
||||||
ValueError: If language code is not supported
|
ValueError: If language code is not supported
|
||||||
"""
|
"""
|
||||||
supported_langs = ['a', 'b', 'f', 'j']
|
supported_langs = ['a', 'b', 'e', 'f', 'h', 'i', 'j', 'p', 'z']
|
||||||
if lang_code not in supported_langs:
|
if lang_code not in supported_langs:
|
||||||
raise ValueError(
|
raise ValueError(
|
||||||
f"Unsupported language code: {lang_code}. "
|
f"Unsupported language code: {lang_code}. "
|
||||||
@@ -72,15 +74,62 @@ class TTSService:
|
|||||||
logger.error(f"Failed to initialize pipeline for language {lang_code}: {e}")
|
logger.error(f"Failed to initialize pipeline for language {lang_code}: {e}")
|
||||||
raise
|
raise
|
||||||
|
|
||||||
|
async def segment_text_with_ai(
|
||||||
|
self,
|
||||||
|
text: str,
|
||||||
|
role_voice_map: Dict[str, str]
|
||||||
|
) -> Optional[List[Tuple[str, str, str]]]:
|
||||||
|
"""
|
||||||
|
Uses AI to segment text and assign speakers intelligently.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
text: Input text to segment
|
||||||
|
role_voice_map: Mapping of role IDs to voice IDs
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
List of tuples: (role_id, voice_id, text_content) or None if AI unavailable
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
# Check if Ollama is available
|
||||||
|
is_available = await self.ollama_service.is_available()
|
||||||
|
if not is_available:
|
||||||
|
logger.info("Ollama not available for AI segmentation")
|
||||||
|
return None
|
||||||
|
|
||||||
|
# Get list of speakers from role_voice_map
|
||||||
|
speakers = list(role_voice_map.keys())
|
||||||
|
|
||||||
|
# Request AI segmentation
|
||||||
|
ai_segments = await self.ollama_service.segment_text(text, speakers)
|
||||||
|
|
||||||
|
if not ai_segments:
|
||||||
|
logger.warning("AI segmentation returned no segments")
|
||||||
|
return None
|
||||||
|
|
||||||
|
# Convert to format with voice assignments
|
||||||
|
segments = []
|
||||||
|
for speaker, content in ai_segments:
|
||||||
|
voice = role_voice_map.get(speaker, self.DEFAULT_VOICE)
|
||||||
|
segments.append((speaker, voice, content))
|
||||||
|
|
||||||
|
logger.info(f"AI segmented text into {len(segments)} segments")
|
||||||
|
return segments
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"AI segmentation failed: {e}")
|
||||||
|
return None
|
||||||
|
|
||||||
def segment_text(
|
def segment_text(
|
||||||
self,
|
self,
|
||||||
text: str,
|
text: str,
|
||||||
role_voice_map: Dict[str, str]
|
role_voice_map: Dict[str, str]
|
||||||
) -> List[Tuple[str, str, str]]:
|
) -> List[Tuple[str, str, str]]:
|
||||||
"""
|
"""
|
||||||
Segments text into (role_id, voice, text_content) tuples.
|
Segments text into (role_id, voice, text_content) tuples using pattern matching.
|
||||||
Identifies role markers and assigns segments accordingly.
|
Identifies role markers and assigns segments accordingly.
|
||||||
|
|
||||||
|
This is the fallback method when AI segmentation is not available.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
text: Input text to segment
|
text: Input text to segment
|
||||||
role_voice_map: Mapping of role IDs to voice IDs
|
role_voice_map: Mapping of role IDs to voice IDs
|
||||||
@@ -118,7 +167,7 @@ class TTSService:
|
|||||||
voice = role_voice_map.get("Narrator", self.DEFAULT_VOICE)
|
voice = role_voice_map.get("Narrator", self.DEFAULT_VOICE)
|
||||||
segments.append(("Narrator", voice, line))
|
segments.append(("Narrator", voice, line))
|
||||||
|
|
||||||
logger.info(f"Segmented text into {len(segments)} segments")
|
logger.info(f"Pattern-based segmentation produced {len(segments)} segments")
|
||||||
return segments
|
return segments
|
||||||
|
|
||||||
def _chunk_text(self, text: str, max_length: int = 500) -> List[str]:
|
def _chunk_text(self, text: str, max_length: int = 500) -> List[str]:
|
||||||
@@ -186,13 +235,16 @@ class TTSService:
|
|||||||
try:
|
try:
|
||||||
# Generate audio for chunk
|
# Generate audio for chunk
|
||||||
result = pipeline(chunk, voice=voice, speed=speed)
|
result = pipeline(chunk, voice=voice, speed=speed)
|
||||||
# Kokoro returns a generator that yields audio arrays
|
# Kokoro returns a generator that yields Result objects
|
||||||
# Collect all arrays and concatenate them
|
# Each Result object has an .audio attribute (torch.Tensor)
|
||||||
audio_parts = []
|
audio_parts = []
|
||||||
for part in result:
|
for part in result:
|
||||||
if isinstance(part, np.ndarray):
|
# Extract audio tensor and convert to numpy
|
||||||
# Flatten to 1D if needed
|
if hasattr(part, 'audio'):
|
||||||
audio_parts.append(part.flatten())
|
audio_tensor = part.audio
|
||||||
|
# Convert torch tensor to numpy and flatten
|
||||||
|
audio_np = audio_tensor.cpu().numpy().flatten()
|
||||||
|
audio_parts.append(audio_np)
|
||||||
|
|
||||||
# Concatenate all parts
|
# Concatenate all parts
|
||||||
if audio_parts:
|
if audio_parts:
|
||||||
@@ -247,6 +299,8 @@ class TTSService:
|
|||||||
Main orchestration method for audiobook generation.
|
Main orchestration method for audiobook generation.
|
||||||
Segments text, synthesizes each segment, concatenates, and returns audio bytes.
|
Segments text, synthesizes each segment, concatenates, and returns audio bytes.
|
||||||
|
|
||||||
|
Uses AI segmentation when available, falls back to pattern matching.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
text: Input text
|
text: Input text
|
||||||
lang: Language code
|
lang: Language code
|
||||||
@@ -264,8 +318,14 @@ class TTSService:
|
|||||||
# Get or create pipeline for language
|
# Get or create pipeline for language
|
||||||
pipeline = self.get_pipeline(lang)
|
pipeline = self.get_pipeline(lang)
|
||||||
|
|
||||||
# Segment text by roles
|
# Try AI segmentation first, fall back to pattern matching
|
||||||
segments = self.segment_text(text, role_voice_map)
|
segments = await self.segment_text_with_ai(text, role_voice_map)
|
||||||
|
|
||||||
|
if segments is None:
|
||||||
|
logger.info("Using pattern-based segmentation as fallback")
|
||||||
|
segments = self.segment_text(text, role_voice_map)
|
||||||
|
else:
|
||||||
|
logger.info("Using AI-powered segmentation")
|
||||||
|
|
||||||
if not segments:
|
if not segments:
|
||||||
raise ValueError("No segments to synthesize")
|
raise ValueError("No segments to synthesize")
|
||||||
|
|||||||
4
backend/start_backend.sh
Executable file
4
backend/start_backend.sh
Executable file
@@ -0,0 +1,4 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
cd "$(dirname "$0")"
|
||||||
|
source venv/bin/activate
|
||||||
|
uvicorn main:app --reload --host 0.0.0.0 --port 8002
|
||||||
149
backend/test_ai_segmentation.py
Normal file
149
backend/test_ai_segmentation.py
Normal file
@@ -0,0 +1,149 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Test script for AI-powered text segmentation.
|
||||||
|
Tests the integration between OllamaService and TTSService for intelligent text splitting.
|
||||||
|
"""
|
||||||
|
import asyncio
|
||||||
|
import sys
|
||||||
|
from services.ollama_service import OllamaService
|
||||||
|
from services.tts_service import TTSService
|
||||||
|
|
||||||
|
|
||||||
|
async def test_ai_segmentation():
|
||||||
|
"""Test AI segmentation with a sample narrative text."""
|
||||||
|
print("Testing AI-Powered Text Segmentation\n" + "="*50)
|
||||||
|
|
||||||
|
# Sample text without explicit role markers
|
||||||
|
text = """The old wizard gazed into his crystal ball. "I see danger approaching," he whispered.
|
||||||
|
A young knight heard the warning and drew his sword. "I will protect the kingdom," he declared boldly.
|
||||||
|
The princess watched from her tower window. She knew the prophecy would come true."""
|
||||||
|
|
||||||
|
print(f"\nInput text:\n{text}\n")
|
||||||
|
|
||||||
|
# Initialize services
|
||||||
|
ollama_service = OllamaService()
|
||||||
|
tts_service = TTSService(ollama_service=ollama_service)
|
||||||
|
|
||||||
|
# Check if Ollama is available
|
||||||
|
is_available = await ollama_service.is_available()
|
||||||
|
if not is_available:
|
||||||
|
print("❌ Ollama is not available. Please start Ollama service.")
|
||||||
|
return False
|
||||||
|
|
||||||
|
print("✓ Ollama service is available\n")
|
||||||
|
|
||||||
|
# Define role-voice mapping (simulating what user would assign)
|
||||||
|
role_voice_map = {
|
||||||
|
"Narrator": "af_bella",
|
||||||
|
"The old wizard": "am_adam",
|
||||||
|
"A young knight": "am_michael",
|
||||||
|
"The princess": "af_sarah"
|
||||||
|
}
|
||||||
|
|
||||||
|
print(f"Role-voice mapping: {role_voice_map}\n")
|
||||||
|
|
||||||
|
# Test AI segmentation
|
||||||
|
try:
|
||||||
|
print("Requesting AI segmentation...")
|
||||||
|
segments = await tts_service.segment_text_with_ai(text, role_voice_map)
|
||||||
|
|
||||||
|
if segments is None:
|
||||||
|
print("❌ AI segmentation returned None")
|
||||||
|
return False
|
||||||
|
|
||||||
|
print(f"\n✓ AI segmentation successful! Generated {len(segments)} segments:\n")
|
||||||
|
|
||||||
|
for i, (speaker, voice, content) in enumerate(segments, 1):
|
||||||
|
print(f"Segment {i}:")
|
||||||
|
print(f" Speaker: {speaker}")
|
||||||
|
print(f" Voice: {voice}")
|
||||||
|
print(f" Text: {content[:80]}{'...' if len(content) > 80 else ''}")
|
||||||
|
print()
|
||||||
|
|
||||||
|
# Verify segments are properly assigned
|
||||||
|
speakers_used = set(seg[0] for seg in segments)
|
||||||
|
print(f"Speakers used in segmentation: {speakers_used}")
|
||||||
|
|
||||||
|
# Check if AI intelligently assigned different speakers
|
||||||
|
if len(speakers_used) > 1:
|
||||||
|
print("\n✓ SUCCESS: AI intelligently assigned multiple speakers!")
|
||||||
|
return True
|
||||||
|
else:
|
||||||
|
print("\n⚠ WARNING: Only one speaker used. AI may not have detected dialogue.")
|
||||||
|
return True # Still a success, just not ideal
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
print(f"\n❌ ERROR: {e}")
|
||||||
|
import traceback
|
||||||
|
traceback.print_exc()
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
async def test_fallback_to_pattern():
|
||||||
|
"""Test that pattern-based segmentation works as fallback."""
|
||||||
|
print("\n" + "="*50)
|
||||||
|
print("Testing Fallback to Pattern-Based Segmentation\n")
|
||||||
|
|
||||||
|
# Text with explicit markers
|
||||||
|
text = """Alice: Hello there!
|
||||||
|
Bob: Hi Alice, how are you?
|
||||||
|
[Narrator] They continued their conversation in the garden."""
|
||||||
|
|
||||||
|
print(f"Input text:\n{text}\n")
|
||||||
|
|
||||||
|
tts_service = TTSService()
|
||||||
|
|
||||||
|
role_voice_map = {
|
||||||
|
"Alice": "af_bella",
|
||||||
|
"Bob": "am_adam",
|
||||||
|
"Narrator": "af_sarah"
|
||||||
|
}
|
||||||
|
|
||||||
|
print(f"Role-voice mapping: {role_voice_map}\n")
|
||||||
|
|
||||||
|
# Use pattern-based segmentation
|
||||||
|
segments = tts_service.segment_text(text, role_voice_map)
|
||||||
|
|
||||||
|
print(f"✓ Pattern segmentation generated {len(segments)} segments:\n")
|
||||||
|
|
||||||
|
for i, (speaker, voice, content) in enumerate(segments, 1):
|
||||||
|
print(f"Segment {i}: [{speaker}] {content}")
|
||||||
|
|
||||||
|
# Verify all speakers were detected
|
||||||
|
speakers_used = set(seg[0] for seg in segments)
|
||||||
|
expected_speakers = {"Alice", "Bob", "Narrator"}
|
||||||
|
|
||||||
|
if speakers_used == expected_speakers:
|
||||||
|
print(f"\n✓ SUCCESS: All expected speakers detected: {speakers_used}")
|
||||||
|
return True
|
||||||
|
else:
|
||||||
|
print(f"\n❌ FAIL: Expected {expected_speakers}, got {speakers_used}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
"""Run all tests."""
|
||||||
|
print("\n" + "="*70)
|
||||||
|
print("AI-POWERED TEXT SEGMENTATION TEST SUITE")
|
||||||
|
print("="*70 + "\n")
|
||||||
|
|
||||||
|
# Test 1: AI segmentation
|
||||||
|
test1_passed = await test_ai_segmentation()
|
||||||
|
|
||||||
|
# Test 2: Pattern fallback
|
||||||
|
test2_passed = await test_fallback_to_pattern()
|
||||||
|
|
||||||
|
# Summary
|
||||||
|
print("\n" + "="*70)
|
||||||
|
print("TEST SUMMARY")
|
||||||
|
print("="*70)
|
||||||
|
print(f"AI Segmentation: {'✓ PASSED' if test1_passed else '❌ FAILED'}")
|
||||||
|
print(f"Pattern Fallback: {'✓ PASSED' if test2_passed else '❌ FAILED'}")
|
||||||
|
print("="*70 + "\n")
|
||||||
|
|
||||||
|
return test1_passed and test2_passed
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
success = asyncio.run(main())
|
||||||
|
sys.exit(0 if success else 1)
|
||||||
168
backend/test_end_to_end_ai.py
Normal file
168
backend/test_end_to_end_ai.py
Normal file
@@ -0,0 +1,168 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
End-to-end test demonstrating AI-powered audiobook generation.
|
||||||
|
Shows the complete workflow from unmarked text to multi-voice audio.
|
||||||
|
"""
|
||||||
|
import asyncio
|
||||||
|
import sys
|
||||||
|
from services.role_detector import RoleDetector
|
||||||
|
from services.tts_service import TTSService
|
||||||
|
from services.ollama_service import OllamaService
|
||||||
|
|
||||||
|
|
||||||
|
async def test_complete_workflow():
|
||||||
|
"""Test the complete AI-powered audiobook generation workflow."""
|
||||||
|
print("\n" + "="*70)
|
||||||
|
print("END-TO-END AI-POWERED AUDIOBOOK GENERATION TEST")
|
||||||
|
print("="*70 + "\n")
|
||||||
|
|
||||||
|
# Sample story without any role markers
|
||||||
|
story = """The old wizard Merlin gazed into his crystal ball with concern.
|
||||||
|
"Dark times are coming," he whispered to himself.
|
||||||
|
|
||||||
|
A young knight named Arthur entered the chamber. "Master Merlin, I heard your summons," he said with determination.
|
||||||
|
|
||||||
|
Merlin turned to face him. "Arthur, the kingdom needs you now more than ever."
|
||||||
|
|
||||||
|
Arthur drew his sword. "I am ready to defend our people."
|
||||||
|
|
||||||
|
From the shadows, the sorceress Morgana watched silently. She knew the prophecy would unfold exactly as foretold."""
|
||||||
|
|
||||||
|
print("STORY TEXT (unmarked narrative):")
|
||||||
|
print("-" * 70)
|
||||||
|
print(story)
|
||||||
|
print("-" * 70 + "\n")
|
||||||
|
|
||||||
|
# Initialize services
|
||||||
|
ollama_service = OllamaService()
|
||||||
|
role_detector = RoleDetector(ollama_service=ollama_service)
|
||||||
|
tts_service = TTSService(ollama_service=ollama_service)
|
||||||
|
|
||||||
|
# Check Ollama availability
|
||||||
|
is_available = await ollama_service.is_available()
|
||||||
|
if not is_available:
|
||||||
|
print("❌ Ollama is not available. This test requires Ollama.")
|
||||||
|
print(" Please start Ollama with: ollama serve")
|
||||||
|
return False
|
||||||
|
|
||||||
|
print("✓ Ollama service is available\n")
|
||||||
|
|
||||||
|
# Step 1: Detect roles using AI
|
||||||
|
print("STEP 1: AI Role Detection")
|
||||||
|
print("-" * 70)
|
||||||
|
try:
|
||||||
|
roles = await role_detector.detect_roles(story)
|
||||||
|
print(f"✓ Detected {len(roles)} roles:\n")
|
||||||
|
for role in roles:
|
||||||
|
print(f" - {role['label']}")
|
||||||
|
print()
|
||||||
|
except Exception as e:
|
||||||
|
print(f"❌ Role detection failed: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
# Step 2: User assigns voices (simulated)
|
||||||
|
print("STEP 2: Voice Assignment (simulated user input)")
|
||||||
|
print("-" * 70)
|
||||||
|
role_voice_map = {}
|
||||||
|
voice_options = ["af_bella", "am_adam", "am_michael", "af_sarah", "bf_emma"]
|
||||||
|
|
||||||
|
for i, role in enumerate(roles):
|
||||||
|
voice = voice_options[i % len(voice_options)]
|
||||||
|
role_voice_map[role['id']] = voice
|
||||||
|
print(f" {role['label']} → {voice}")
|
||||||
|
print()
|
||||||
|
|
||||||
|
# Step 3: AI text segmentation
|
||||||
|
print("STEP 3: AI Text Segmentation")
|
||||||
|
print("-" * 70)
|
||||||
|
try:
|
||||||
|
segments = await tts_service.segment_text_with_ai(story, role_voice_map)
|
||||||
|
|
||||||
|
if segments is None:
|
||||||
|
print("❌ AI segmentation failed")
|
||||||
|
return False
|
||||||
|
|
||||||
|
print(f"✓ AI segmented text into {len(segments)} segments:\n")
|
||||||
|
|
||||||
|
for i, (speaker, voice, text) in enumerate(segments, 1):
|
||||||
|
# Truncate long text for display
|
||||||
|
display_text = text[:60] + "..." if len(text) > 60 else text
|
||||||
|
print(f" {i}. [{speaker}] ({voice})")
|
||||||
|
print(f" \"{display_text}\"")
|
||||||
|
print()
|
||||||
|
|
||||||
|
# Verify intelligent segmentation
|
||||||
|
speakers_used = set(seg[0] for seg in segments)
|
||||||
|
print(f"Speakers used: {speakers_used}")
|
||||||
|
|
||||||
|
# Check if dialogue was properly assigned
|
||||||
|
has_multiple_speakers = len(speakers_used) > 1
|
||||||
|
has_narrator = "Narrator" in speakers_used or any("Merlin" in s or "Arthur" in s or "Morgana" in s for s in speakers_used)
|
||||||
|
|
||||||
|
if has_multiple_speakers and has_narrator:
|
||||||
|
print("✓ AI successfully assigned dialogue to characters and narrative to appropriate speakers")
|
||||||
|
else:
|
||||||
|
print("⚠ Segmentation may not be optimal")
|
||||||
|
print()
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
print(f"❌ Segmentation failed: {e}")
|
||||||
|
import traceback
|
||||||
|
traceback.print_exc()
|
||||||
|
return False
|
||||||
|
|
||||||
|
# Step 4: Verify segmentation quality
|
||||||
|
print("STEP 4: Segmentation Quality Analysis")
|
||||||
|
print("-" * 70)
|
||||||
|
|
||||||
|
# Count segments per speaker
|
||||||
|
speaker_counts = {}
|
||||||
|
for speaker, _, _ in segments:
|
||||||
|
speaker_counts[speaker] = speaker_counts.get(speaker, 0) + 1
|
||||||
|
|
||||||
|
print("Segments per speaker:")
|
||||||
|
for speaker, count in sorted(speaker_counts.items(), key=lambda x: x[1], reverse=True):
|
||||||
|
print(f" {speaker}: {count} segments")
|
||||||
|
print()
|
||||||
|
|
||||||
|
# Check for dialogue detection
|
||||||
|
dialogue_keywords = ['"', "'", "said", "whispered", "replied"]
|
||||||
|
dialogue_segments = 0
|
||||||
|
for _, _, text in segments:
|
||||||
|
if any(kw in text.lower() for kw in dialogue_keywords):
|
||||||
|
dialogue_segments += 1
|
||||||
|
|
||||||
|
print(f"Dialogue segments detected: {dialogue_segments}/{len(segments)}")
|
||||||
|
print()
|
||||||
|
|
||||||
|
# Summary
|
||||||
|
print("="*70)
|
||||||
|
print("TEST SUMMARY")
|
||||||
|
print("="*70)
|
||||||
|
print(f"✓ Role Detection: {len(roles)} roles detected")
|
||||||
|
print(f"✓ Voice Assignment: {len(role_voice_map)} voices assigned")
|
||||||
|
print(f"✓ Text Segmentation: {len(segments)} segments created")
|
||||||
|
print(f"✓ Speaker Distribution: {len(speakers_used)} unique speakers used")
|
||||||
|
print(f"✓ Dialogue Detection: {dialogue_segments} dialogue segments")
|
||||||
|
print()
|
||||||
|
print("🎉 AI-POWERED AUDIOBOOK GENERATION: SUCCESS!")
|
||||||
|
print("="*70 + "\n")
|
||||||
|
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
"""Run the end-to-end test."""
|
||||||
|
try:
|
||||||
|
success = await test_complete_workflow()
|
||||||
|
return success
|
||||||
|
except Exception as e:
|
||||||
|
print(f"\n❌ Test failed with exception: {e}")
|
||||||
|
import traceback
|
||||||
|
traceback.print_exc()
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
success = asyncio.run(main())
|
||||||
|
sys.exit(0 if success else 1)
|
||||||
175
backend/test_ollama_integration.py
Normal file
175
backend/test_ollama_integration.py
Normal file
@@ -0,0 +1,175 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Unit tests for Ollama integration in RoleDetector.
|
||||||
|
Tests the integration between RoleDetector and OllamaService.
|
||||||
|
"""
|
||||||
|
import pytest
|
||||||
|
import asyncio
|
||||||
|
from unittest.mock import AsyncMock, MagicMock, patch
|
||||||
|
from services.role_detector import RoleDetector
|
||||||
|
from services.ollama_service import OllamaService
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_detect_roles_with_ollama_available():
|
||||||
|
"""Test role detection when Ollama is available and returns speakers."""
|
||||||
|
# Create mock OllamaService
|
||||||
|
mock_ollama = AsyncMock(spec=OllamaService)
|
||||||
|
mock_ollama.is_available.return_value = True
|
||||||
|
mock_ollama.detect_speakers.return_value = ["Alice", "Bob", "Charlie"]
|
||||||
|
|
||||||
|
# Create RoleDetector with mock
|
||||||
|
detector = RoleDetector(ollama_service=mock_ollama)
|
||||||
|
|
||||||
|
# Test with text that has no pattern markers
|
||||||
|
text = "This is a conversation between Alice and Bob. Charlie joins later."
|
||||||
|
roles = await detector.detect_roles(text)
|
||||||
|
|
||||||
|
# Verify Ollama was called
|
||||||
|
mock_ollama.is_available.assert_called_once()
|
||||||
|
mock_ollama.detect_speakers.assert_called_once_with(text)
|
||||||
|
|
||||||
|
# Verify roles were detected
|
||||||
|
role_labels = [r['label'] for r in roles]
|
||||||
|
assert "Alice" in role_labels
|
||||||
|
assert "Bob" in role_labels
|
||||||
|
assert "Charlie" in role_labels
|
||||||
|
print("✓ PASS: Ollama detection works when available")
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_detect_roles_with_ollama_unavailable():
|
||||||
|
"""Test role detection falls back to pattern matching when Ollama unavailable."""
|
||||||
|
# Create mock OllamaService that's unavailable
|
||||||
|
mock_ollama = AsyncMock(spec=OllamaService)
|
||||||
|
mock_ollama.is_available.return_value = False
|
||||||
|
|
||||||
|
# Create RoleDetector with mock
|
||||||
|
detector = RoleDetector(ollama_service=mock_ollama)
|
||||||
|
|
||||||
|
# Test with text that has pattern markers
|
||||||
|
text = "Alice: Hello there!\nBob: Hi Alice!"
|
||||||
|
roles = await detector.detect_roles(text)
|
||||||
|
|
||||||
|
# Verify Ollama availability was checked
|
||||||
|
mock_ollama.is_available.assert_called_once()
|
||||||
|
# Verify detect_speakers was NOT called since unavailable
|
||||||
|
mock_ollama.detect_speakers.assert_not_called()
|
||||||
|
|
||||||
|
# Verify pattern matching still worked
|
||||||
|
role_labels = [r['label'] for r in roles]
|
||||||
|
assert "Alice" in role_labels
|
||||||
|
assert "Bob" in role_labels
|
||||||
|
print("✓ PASS: Falls back to pattern matching when Ollama unavailable")
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_detect_roles_with_ollama_error():
|
||||||
|
"""Test role detection handles Ollama errors gracefully."""
|
||||||
|
# Create mock OllamaService that throws error
|
||||||
|
mock_ollama = AsyncMock(spec=OllamaService)
|
||||||
|
mock_ollama.is_available.return_value = True
|
||||||
|
mock_ollama.detect_speakers.side_effect = Exception("Ollama connection failed")
|
||||||
|
|
||||||
|
# Create RoleDetector with mock
|
||||||
|
detector = RoleDetector(ollama_service=mock_ollama)
|
||||||
|
|
||||||
|
# Test with text that has pattern markers
|
||||||
|
text = "Alice: Hello there!\nBob: Hi Alice!"
|
||||||
|
roles = await detector.detect_roles(text)
|
||||||
|
|
||||||
|
# Verify Ollama was attempted
|
||||||
|
mock_ollama.is_available.assert_called_once()
|
||||||
|
mock_ollama.detect_speakers.assert_called_once()
|
||||||
|
|
||||||
|
# Verify pattern matching still worked as fallback
|
||||||
|
role_labels = [r['label'] for r in roles]
|
||||||
|
assert "Alice" in role_labels
|
||||||
|
assert "Bob" in role_labels
|
||||||
|
print("✓ PASS: Handles Ollama errors gracefully and falls back to patterns")
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_detect_roles_merges_ollama_and_pattern_results():
|
||||||
|
"""Test that roles from both Ollama and pattern matching are merged."""
|
||||||
|
# Create mock OllamaService
|
||||||
|
mock_ollama = AsyncMock(spec=OllamaService)
|
||||||
|
mock_ollama.is_available.return_value = True
|
||||||
|
mock_ollama.detect_speakers.return_value = ["Alice", "Charlie"]
|
||||||
|
|
||||||
|
# Create RoleDetector with mock
|
||||||
|
detector = RoleDetector(ollama_service=mock_ollama)
|
||||||
|
|
||||||
|
# Test with text that has pattern markers for different roles
|
||||||
|
text = "Alice: Hello!\nBob: Hi there!\nCharlie appears later."
|
||||||
|
roles = await detector.detect_roles(text)
|
||||||
|
|
||||||
|
# Verify both methods were used
|
||||||
|
mock_ollama.detect_speakers.assert_called_once()
|
||||||
|
|
||||||
|
# Verify roles from both sources are present
|
||||||
|
role_labels = [r['label'] for r in roles]
|
||||||
|
assert "Alice" in role_labels # From both Ollama and pattern
|
||||||
|
assert "Bob" in role_labels # From pattern only
|
||||||
|
assert "Charlie" in role_labels # From Ollama only
|
||||||
|
print("✓ PASS: Merges results from both Ollama and pattern matching")
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_detect_roles_deduplicates_across_methods():
|
||||||
|
"""Test that duplicate roles from Ollama and patterns are deduplicated."""
|
||||||
|
# Create mock OllamaService
|
||||||
|
mock_ollama = AsyncMock(spec=OllamaService)
|
||||||
|
mock_ollama.is_available.return_value = True
|
||||||
|
mock_ollama.detect_speakers.return_value = ["Alice", "Bob"]
|
||||||
|
|
||||||
|
# Create RoleDetector with mock
|
||||||
|
detector = RoleDetector(ollama_service=mock_ollama)
|
||||||
|
|
||||||
|
# Test with text where patterns also detect Alice and Bob
|
||||||
|
text = "Alice: Hello!\nBob: Hi there!"
|
||||||
|
roles = await detector.detect_roles(text)
|
||||||
|
|
||||||
|
# Verify roles are deduplicated
|
||||||
|
role_labels = [r['label'] for r in roles]
|
||||||
|
assert role_labels.count("Alice") == 1
|
||||||
|
assert role_labels.count("Bob") == 1
|
||||||
|
assert len(role_labels) == 2
|
||||||
|
print("✓ PASS: Deduplicates roles across both detection methods")
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_detect_roles_returns_narrator_when_no_roles_found():
|
||||||
|
"""Test that default Narrator is returned when no roles detected by either method."""
|
||||||
|
# Create mock OllamaService that returns empty
|
||||||
|
mock_ollama = AsyncMock(spec=OllamaService)
|
||||||
|
mock_ollama.is_available.return_value = True
|
||||||
|
mock_ollama.detect_speakers.return_value = []
|
||||||
|
|
||||||
|
# Create RoleDetector with mock
|
||||||
|
detector = RoleDetector(ollama_service=mock_ollama)
|
||||||
|
|
||||||
|
# Test with text that has no role markers
|
||||||
|
text = "This is just plain text without any speakers."
|
||||||
|
roles = await detector.detect_roles(text)
|
||||||
|
|
||||||
|
# Verify default Narrator is returned
|
||||||
|
assert len(roles) == 1
|
||||||
|
assert roles[0]['label'] == "Narrator"
|
||||||
|
print("✓ PASS: Returns default Narrator when no roles found")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
# Run tests
|
||||||
|
print("Testing Ollama Integration in RoleDetector...\n")
|
||||||
|
|
||||||
|
asyncio.run(test_detect_roles_with_ollama_available())
|
||||||
|
asyncio.run(test_detect_roles_with_ollama_unavailable())
|
||||||
|
asyncio.run(test_detect_roles_with_ollama_error())
|
||||||
|
asyncio.run(test_detect_roles_merges_ollama_and_pattern_results())
|
||||||
|
asyncio.run(test_detect_roles_deduplicates_across_methods())
|
||||||
|
asyncio.run(test_detect_roles_returns_narrator_when_no_roles_found())
|
||||||
|
|
||||||
|
print("\n" + "="*50)
|
||||||
|
print("All integration tests passed!")
|
||||||
|
print("="*50)
|
||||||
237
backend/test_ollama_service.py
Normal file
237
backend/test_ollama_service.py
Normal file
@@ -0,0 +1,237 @@
|
|||||||
|
"""
|
||||||
|
Tests for Ollama integration service.
|
||||||
|
"""
|
||||||
|
import pytest
|
||||||
|
import json
|
||||||
|
from unittest.mock import AsyncMock, patch, MagicMock
|
||||||
|
from services.ollama_service import OllamaService
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_ollama_service_initialization():
|
||||||
|
"""Test OllamaService initializes with correct defaults."""
|
||||||
|
service = OllamaService()
|
||||||
|
assert service.base_url == "http://localhost:11434"
|
||||||
|
assert service.model == "llama3:8b"
|
||||||
|
assert service.timeout == 30.0
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_ollama_service_custom_base_url():
|
||||||
|
"""Test OllamaService accepts custom base URL."""
|
||||||
|
custom_url = "http://custom-ollama:11434"
|
||||||
|
service = OllamaService(base_url=custom_url)
|
||||||
|
assert service.base_url == custom_url
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_sample_text_short():
|
||||||
|
"""Test sampling short text returns full text."""
|
||||||
|
service = OllamaService()
|
||||||
|
text = "Short text"
|
||||||
|
samples = service._sample_text(text, chunk_size=5000, num_samples=4)
|
||||||
|
|
||||||
|
assert len(samples) == 1
|
||||||
|
assert samples[0] == text
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_sample_text_long():
|
||||||
|
"""Test sampling long text returns multiple samples."""
|
||||||
|
service = OllamaService()
|
||||||
|
# Create text with position markers so we can verify different samples
|
||||||
|
text = "".join([f"Position{i:05d} " for i in range(2500)]) # ~25k characters
|
||||||
|
samples = service._sample_text(text, chunk_size=5000, num_samples=4)
|
||||||
|
|
||||||
|
assert len(samples) == 4
|
||||||
|
assert all(len(s) <= 5000 for s in samples)
|
||||||
|
# Samples should be from different parts - check they contain different position markers
|
||||||
|
assert "Position00000" in samples[0] # First sample should have early positions
|
||||||
|
assert "Position00000" not in samples[-1] # Last sample should have later positions
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_sample_text_medium():
|
||||||
|
"""Test sampling medium text returns appropriate number of samples."""
|
||||||
|
service = OllamaService()
|
||||||
|
text = "B" * 8000 # 8k characters
|
||||||
|
samples = service._sample_text(text, chunk_size=5000, num_samples=4)
|
||||||
|
|
||||||
|
# Should get 2 samples (8000 / 5000 = 1.6, so 2 samples)
|
||||||
|
assert len(samples) >= 1
|
||||||
|
assert all(len(s) <= 5000 for s in samples)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_build_prompt():
|
||||||
|
"""Test prompt builder creates correct format."""
|
||||||
|
service = OllamaService()
|
||||||
|
text = "Alice: Hello there!\nBob: Hi Alice!"
|
||||||
|
prompt = service._build_prompt(text)
|
||||||
|
|
||||||
|
assert "speakers" in prompt
|
||||||
|
assert "JSON" in prompt
|
||||||
|
assert text in prompt
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_parse_response_with_speakers_key():
|
||||||
|
"""Test parsing JSON response with 'speakers' key."""
|
||||||
|
service = OllamaService()
|
||||||
|
response = '{"speakers": ["Alice", "Bob", "Narrator"]}'
|
||||||
|
speakers = service._parse_response(response)
|
||||||
|
|
||||||
|
assert speakers == ["Alice", "Bob", "Narrator"]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_parse_response_with_characters_key():
|
||||||
|
"""Test parsing JSON response with 'characters' key."""
|
||||||
|
service = OllamaService()
|
||||||
|
response = '{"characters": ["Alice", "Bob"]}'
|
||||||
|
speakers = service._parse_response(response)
|
||||||
|
|
||||||
|
assert speakers == ["Alice", "Bob"]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_parse_response_with_roles_key():
|
||||||
|
"""Test parsing JSON response with 'roles' key."""
|
||||||
|
service = OllamaService()
|
||||||
|
response = '{"roles": ["Narrator", "Alice"]}'
|
||||||
|
speakers = service._parse_response(response)
|
||||||
|
|
||||||
|
assert speakers == ["Narrator", "Alice"]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_parse_response_with_array():
|
||||||
|
"""Test parsing JSON response that is a direct array."""
|
||||||
|
service = OllamaService()
|
||||||
|
response = '["Alice", "Bob"]'
|
||||||
|
speakers = service._parse_response(response)
|
||||||
|
|
||||||
|
assert speakers == ["Alice", "Bob"]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_parse_response_empty():
|
||||||
|
"""Test parsing empty response."""
|
||||||
|
service = OllamaService()
|
||||||
|
speakers = service._parse_response("")
|
||||||
|
assert speakers == []
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_parse_response_invalid_json():
|
||||||
|
"""Test parsing invalid JSON returns empty list."""
|
||||||
|
service = OllamaService()
|
||||||
|
response = "This is not JSON"
|
||||||
|
speakers = service._parse_response(response)
|
||||||
|
assert speakers == []
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_detect_speakers_success():
|
||||||
|
"""Test successful speaker detection with short text."""
|
||||||
|
service = OllamaService()
|
||||||
|
text = "Alice: Hello!\nBob: Hi there!"
|
||||||
|
|
||||||
|
mock_response = MagicMock()
|
||||||
|
mock_response.json.return_value = {
|
||||||
|
"response": '{"speakers": ["Alice", "Bob"]}'
|
||||||
|
}
|
||||||
|
mock_response.raise_for_status = MagicMock()
|
||||||
|
|
||||||
|
with patch('httpx.AsyncClient') as mock_client:
|
||||||
|
mock_client.return_value.__aenter__.return_value.post = AsyncMock(return_value=mock_response)
|
||||||
|
|
||||||
|
speakers = await service.detect_speakers(text)
|
||||||
|
assert set(speakers) == {"Alice", "Bob"}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_detect_speakers_multiple_samples():
|
||||||
|
"""Test speaker detection with long text requiring multiple samples."""
|
||||||
|
service = OllamaService()
|
||||||
|
text = "A" * 25000 # Long text requiring multiple samples
|
||||||
|
|
||||||
|
# Mock responses for different samples
|
||||||
|
responses = [
|
||||||
|
MagicMock(json=lambda: {"response": '{"speakers": ["Alice", "Bob"]}'}),
|
||||||
|
MagicMock(json=lambda: {"response": '{"speakers": ["Charlie", "Bob"]}'}),
|
||||||
|
MagicMock(json=lambda: {"response": '{"speakers": ["Alice", "David"]}'}),
|
||||||
|
MagicMock(json=lambda: {"response": '{"speakers": ["Eve"]}'}),
|
||||||
|
]
|
||||||
|
|
||||||
|
for r in responses:
|
||||||
|
r.raise_for_status = MagicMock()
|
||||||
|
|
||||||
|
with patch('httpx.AsyncClient') as mock_client:
|
||||||
|
mock_client.return_value.__aenter__.return_value.post = AsyncMock(side_effect=responses)
|
||||||
|
|
||||||
|
speakers = await service.detect_speakers(text)
|
||||||
|
# Should deduplicate across all samples
|
||||||
|
assert set(speakers) == {"Alice", "Bob", "Charlie", "David", "Eve"}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_detect_speakers_timeout():
|
||||||
|
"""Test speaker detection handles timeout."""
|
||||||
|
service = OllamaService()
|
||||||
|
text = "Some text"
|
||||||
|
|
||||||
|
with patch('httpx.AsyncClient') as mock_client:
|
||||||
|
import httpx
|
||||||
|
mock_client.return_value.__aenter__.return_value.post = AsyncMock(
|
||||||
|
side_effect=httpx.TimeoutException("Timeout")
|
||||||
|
)
|
||||||
|
|
||||||
|
with pytest.raises(Exception, match="timed out"):
|
||||||
|
await service.detect_speakers(text)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_detect_speakers_http_error():
|
||||||
|
"""Test speaker detection handles HTTP errors."""
|
||||||
|
service = OllamaService()
|
||||||
|
text = "Some text"
|
||||||
|
|
||||||
|
with patch('httpx.AsyncClient') as mock_client:
|
||||||
|
import httpx
|
||||||
|
mock_client.return_value.__aenter__.return_value.post = AsyncMock(
|
||||||
|
side_effect=httpx.HTTPError("Connection failed")
|
||||||
|
)
|
||||||
|
|
||||||
|
with pytest.raises(Exception, match="Ollama service error"):
|
||||||
|
await service.detect_speakers(text)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_is_available_success():
|
||||||
|
"""Test is_available returns True when service responds."""
|
||||||
|
service = OllamaService()
|
||||||
|
|
||||||
|
mock_response = MagicMock()
|
||||||
|
mock_response.status_code = 200
|
||||||
|
|
||||||
|
with patch('httpx.AsyncClient') as mock_client:
|
||||||
|
mock_client.return_value.__aenter__.return_value.get = AsyncMock(return_value=mock_response)
|
||||||
|
|
||||||
|
available = await service.is_available()
|
||||||
|
assert available is True
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_is_available_failure():
|
||||||
|
"""Test is_available returns False when service is down."""
|
||||||
|
service = OllamaService()
|
||||||
|
|
||||||
|
with patch('httpx.AsyncClient') as mock_client:
|
||||||
|
import httpx
|
||||||
|
mock_client.return_value.__aenter__.return_value.get = AsyncMock(
|
||||||
|
side_effect=httpx.ConnectError("Connection refused")
|
||||||
|
)
|
||||||
|
|
||||||
|
available = await service.is_available()
|
||||||
|
assert available is False
|
||||||
@@ -1 +1 @@
|
|||||||
VITE_API_BASE_URL=http://localhost:8000
|
VITE_API_BASE_URL=http://localhost:8002
|
||||||
|
|||||||
@@ -25,24 +25,108 @@ interface LanguageOption {
|
|||||||
label: string;
|
label: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Kokoro voice options as defined in the design document
|
// All available Kokoro voice options (54 voices across 9 languages)
|
||||||
const KOKORO_VOICES: VoiceOption[] = [
|
const KOKORO_VOICES: VoiceOption[] = [
|
||||||
|
// American English - Female (11 voices)
|
||||||
|
{ id: 'af_alloy', label: 'Alloy (Female, American)', gender: 'female', accent: 'American' },
|
||||||
|
{ id: 'af_aoede', label: 'Aoede (Female, American)', gender: 'female', accent: 'American' },
|
||||||
{ id: 'af_bella', label: 'Bella (Female, American)', gender: 'female', accent: 'American' },
|
{ id: 'af_bella', label: 'Bella (Female, American)', gender: 'female', accent: 'American' },
|
||||||
|
{ id: 'af_heart', label: 'Heart (Female, American)', gender: 'female', accent: 'American' },
|
||||||
|
{ id: 'af_jessica', label: 'Jessica (Female, American)', gender: 'female', accent: 'American' },
|
||||||
|
{ id: 'af_kore', label: 'Kore (Female, American)', gender: 'female', accent: 'American' },
|
||||||
|
{ id: 'af_nicole', label: 'Nicole (Female, American)', gender: 'female', accent: 'American' },
|
||||||
|
{ id: 'af_nova', label: 'Nova (Female, American)', gender: 'female', accent: 'American' },
|
||||||
|
{ id: 'af_river', label: 'River (Female, American)', gender: 'female', accent: 'American' },
|
||||||
{ id: 'af_sarah', label: 'Sarah (Female, American)', gender: 'female', accent: 'American' },
|
{ id: 'af_sarah', label: 'Sarah (Female, American)', gender: 'female', accent: 'American' },
|
||||||
|
{ id: 'af_sky', label: 'Sky (Female, American)', gender: 'female', accent: 'American' },
|
||||||
|
|
||||||
|
// American English - Male (10 voices)
|
||||||
{ id: 'am_adam', label: 'Adam (Male, American)', gender: 'male', accent: 'American' },
|
{ id: 'am_adam', label: 'Adam (Male, American)', gender: 'male', accent: 'American' },
|
||||||
|
{ id: 'am_echo', label: 'Echo (Male, American)', gender: 'male', accent: 'American' },
|
||||||
|
{ id: 'am_eric', label: 'Eric (Male, American)', gender: 'male', accent: 'American' },
|
||||||
|
{ id: 'am_fenrir', label: 'Fenrir (Male, American)', gender: 'male', accent: 'American' },
|
||||||
|
{ id: 'am_liam', label: 'Liam (Male, American)', gender: 'male', accent: 'American' },
|
||||||
{ id: 'am_michael', label: 'Michael (Male, American)', gender: 'male', accent: 'American' },
|
{ id: 'am_michael', label: 'Michael (Male, American)', gender: 'male', accent: 'American' },
|
||||||
|
{ id: 'am_onyx', label: 'Onyx (Male, American)', gender: 'male', accent: 'American' },
|
||||||
|
{ id: 'am_puck', label: 'Puck (Male, American)', gender: 'male', accent: 'American' },
|
||||||
|
{ id: 'am_santa', label: 'Santa (Male, American)', gender: 'male', accent: 'American' },
|
||||||
|
|
||||||
|
// British English - Female (4 voices)
|
||||||
|
{ id: 'bf_alice', label: 'Alice (Female, British)', gender: 'female', accent: 'British' },
|
||||||
{ id: 'bf_emma', label: 'Emma (Female, British)', gender: 'female', accent: 'British' },
|
{ id: 'bf_emma', label: 'Emma (Female, British)', gender: 'female', accent: 'British' },
|
||||||
{ id: 'bf_isabella', label: 'Isabella (Female, British)', gender: 'female', accent: 'British' },
|
{ id: 'bf_isabella', label: 'Isabella (Female, British)', gender: 'female', accent: 'British' },
|
||||||
|
{ id: 'bf_lily', label: 'Lily (Female, British)', gender: 'female', accent: 'British' },
|
||||||
|
|
||||||
|
// British English - Male (4 voices)
|
||||||
|
{ id: 'bm_daniel', label: 'Daniel (Male, British)', gender: 'male', accent: 'British' },
|
||||||
|
{ id: 'bm_fable', label: 'Fable (Male, British)', gender: 'male', accent: 'British' },
|
||||||
{ id: 'bm_george', label: 'George (Male, British)', gender: 'male', accent: 'British' },
|
{ id: 'bm_george', label: 'George (Male, British)', gender: 'male', accent: 'British' },
|
||||||
{ id: 'bm_lewis', label: 'Lewis (Male, British)', gender: 'male', accent: 'British' },
|
{ id: 'bm_lewis', label: 'Lewis (Male, British)', gender: 'male', accent: 'British' },
|
||||||
|
|
||||||
|
// Spanish - Female (1 voice)
|
||||||
|
{ id: 'ef_dora', label: 'Dora (Female, Spanish)', gender: 'female', accent: 'Spanish' },
|
||||||
|
|
||||||
|
// Spanish - Male (2 voices)
|
||||||
|
{ id: 'em_alex', label: 'Alex (Male, Spanish)', gender: 'male', accent: 'Spanish' },
|
||||||
|
{ id: 'em_santa', label: 'Santa (Male, Spanish)', gender: 'male', accent: 'Spanish' },
|
||||||
|
|
||||||
|
// French - Female (1 voice)
|
||||||
|
{ id: 'ff_siwis', label: 'Siwis (Female, French)', gender: 'female', accent: 'French' },
|
||||||
|
|
||||||
|
// Hindi - Female (2 voices)
|
||||||
|
{ id: 'hf_alpha', label: 'Alpha (Female, Hindi)', gender: 'female', accent: 'Hindi' },
|
||||||
|
{ id: 'hf_beta', label: 'Beta (Female, Hindi)', gender: 'female', accent: 'Hindi' },
|
||||||
|
|
||||||
|
// Hindi - Male (2 voices)
|
||||||
|
{ id: 'hm_omega', label: 'Omega (Male, Hindi)', gender: 'male', accent: 'Hindi' },
|
||||||
|
{ id: 'hm_psi', label: 'Psi (Male, Hindi)', gender: 'male', accent: 'Hindi' },
|
||||||
|
|
||||||
|
// Italian - Female (1 voice)
|
||||||
|
{ id: 'if_sara', label: 'Sara (Female, Italian)', gender: 'female', accent: 'Italian' },
|
||||||
|
|
||||||
|
// Italian - Male (1 voice)
|
||||||
|
{ id: 'im_nicola', label: 'Nicola (Male, Italian)', gender: 'male', accent: 'Italian' },
|
||||||
|
|
||||||
|
// Japanese - Female (4 voices)
|
||||||
|
{ id: 'jf_alpha', label: 'Alpha (Female, Japanese)', gender: 'female', accent: 'Japanese' },
|
||||||
|
{ id: 'jf_gongitsune', label: 'Gongitsune (Female, Japanese)', gender: 'female', accent: 'Japanese' },
|
||||||
|
{ id: 'jf_nezumi', label: 'Nezumi (Female, Japanese)', gender: 'female', accent: 'Japanese' },
|
||||||
|
{ id: 'jf_tebukuro', label: 'Tebukuro (Female, Japanese)', gender: 'female', accent: 'Japanese' },
|
||||||
|
|
||||||
|
// Japanese - Male (1 voice)
|
||||||
|
{ id: 'jm_kumo', label: 'Kumo (Male, Japanese)', gender: 'male', accent: 'Japanese' },
|
||||||
|
|
||||||
|
// Portuguese - Female (1 voice)
|
||||||
|
{ id: 'pf_dora', label: 'Dora (Female, Portuguese)', gender: 'female', accent: 'Portuguese' },
|
||||||
|
|
||||||
|
// Portuguese - Male (2 voices)
|
||||||
|
{ id: 'pm_alex', label: 'Alex (Male, Portuguese)', gender: 'male', accent: 'Portuguese' },
|
||||||
|
{ id: 'pm_santa', label: 'Santa (Male, Portuguese)', gender: 'male', accent: 'Portuguese' },
|
||||||
|
|
||||||
|
// Chinese/Mandarin - Female (4 voices)
|
||||||
|
{ id: 'zf_xiaobei', label: 'Xiaobei (Female, Chinese)', gender: 'female', accent: 'Chinese' },
|
||||||
|
{ id: 'zf_xiaoni', label: 'Xiaoni (Female, Chinese)', gender: 'female', accent: 'Chinese' },
|
||||||
|
{ id: 'zf_xiaoxiao', label: 'Xiaoxiao (Female, Chinese)', gender: 'female', accent: 'Chinese' },
|
||||||
|
{ id: 'zf_xiaoyi', label: 'Xiaoyi (Female, Chinese)', gender: 'female', accent: 'Chinese' },
|
||||||
|
|
||||||
|
// Chinese/Mandarin - Male (4 voices)
|
||||||
|
{ id: 'zm_yunjian', label: 'Yunjian (Male, Chinese)', gender: 'male', accent: 'Chinese' },
|
||||||
|
{ id: 'zm_yunxi', label: 'Yunxi (Male, Chinese)', gender: 'male', accent: 'Chinese' },
|
||||||
|
{ id: 'zm_yunxia', label: 'Yunxia (Male, Chinese)', gender: 'male', accent: 'Chinese' },
|
||||||
|
{ id: 'zm_yunyang', label: 'Yunyang (Male, Chinese)', gender: 'male', accent: 'Chinese' },
|
||||||
];
|
];
|
||||||
|
|
||||||
// Language options as defined in the design document
|
// Language options - all supported by Kokoro TTS
|
||||||
const LANGUAGES: LanguageOption[] = [
|
const LANGUAGES: LanguageOption[] = [
|
||||||
{ code: 'a', label: 'American English' },
|
{ code: 'a', label: 'American English' },
|
||||||
{ code: 'b', label: 'British English' },
|
{ code: 'b', label: 'British English' },
|
||||||
{ code: 'f', label: 'French' },
|
{ code: 'e', label: 'Spanish (Español)' },
|
||||||
{ code: 'j', label: 'Japanese' },
|
{ code: 'f', label: 'French (Français)' },
|
||||||
|
{ code: 'h', label: 'Hindi (हिन्दी)' },
|
||||||
|
{ code: 'i', label: 'Italian (Italiano)' },
|
||||||
|
{ code: 'j', label: 'Japanese (日本語)' },
|
||||||
|
{ code: 'p', label: 'Portuguese (Português)' },
|
||||||
|
{ code: 'z', label: 'Chinese (中文)' },
|
||||||
];
|
];
|
||||||
|
|
||||||
export default function VoiceAssignmentSection({
|
export default function VoiceAssignmentSection({
|
||||||
|
|||||||
27
test_my_story.py
Normal file
27
test_my_story.py
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
import sys
|
||||||
|
sys.path.insert(0, 'backend')
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
from services.role_detector import RoleDetector
|
||||||
|
from services.ollama_service import OllamaService
|
||||||
|
|
||||||
|
async def test():
|
||||||
|
# Read the actual story
|
||||||
|
with open('Test Story.txt', 'r') as f:
|
||||||
|
story = f.read()
|
||||||
|
|
||||||
|
print(f'Story length: {len(story)} characters\n')
|
||||||
|
|
||||||
|
# Test role detection
|
||||||
|
ollama_service = OllamaService()
|
||||||
|
role_detector = RoleDetector(ollama_service=ollama_service)
|
||||||
|
|
||||||
|
print('Testing role detection...')
|
||||||
|
roles = await role_detector.detect_roles(story)
|
||||||
|
|
||||||
|
print(f'\nDetected {len(roles)} roles:')
|
||||||
|
for role in roles:
|
||||||
|
print(f' - {role["label"]}')
|
||||||
|
|
||||||
|
asyncio.run(test())
|
||||||
24
test_ollama_quick.py
Normal file
24
test_ollama_quick.py
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
import sys
|
||||||
|
sys.path.insert(0, 'backend')
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
from services.ollama_service import OllamaService
|
||||||
|
|
||||||
|
async def test():
|
||||||
|
service = OllamaService()
|
||||||
|
available = await service.is_available()
|
||||||
|
print(f'Ollama available: {available}')
|
||||||
|
|
||||||
|
if available:
|
||||||
|
text = '''Lily asked, "Have you guys heard about the new stream?"
|
||||||
|
Benny's eyes lit up. "Ooh, I love streams!"
|
||||||
|
Sammy nodded enthusiastically. "I've been wanting to explore!"'''
|
||||||
|
|
||||||
|
print(f'\nTesting with sample text ({len(text)} chars)...')
|
||||||
|
speakers = await service.detect_speakers(text)
|
||||||
|
print(f'Detected speakers: {speakers}')
|
||||||
|
else:
|
||||||
|
print('Ollama is not available!')
|
||||||
|
|
||||||
|
asyncio.run(test())
|
||||||
Reference in New Issue
Block a user