sush0401 commited on
Commit
ba4bab7
·
verified ·
1 Parent(s): 83b0243

Update model card with demo audio + eval results

Browse files
Files changed (1) hide show
  1. README.md +33 -17
README.md CHANGED
@@ -14,27 +14,17 @@ base_model: openbmb/VoxCPM2
14
 
15
  LoRA adapter fine-tuned on VoxCPM2 for bedtime story narration in English.
16
 
17
- ## Training details
18
- - **Base model**: `openbmb/VoxCPM2`
19
  - **Dataset**: LJSpeech (6,550 clips, ~12h — half-data experiment)
20
  - **Method**: LoRA (r=32, alpha=64, targeting DiT attention layers)
21
- - **Best experiment**: exp6 (r=32, lr=5e-5, half data)
22
- - **Validation loss**: 0.872 (best across 6 experiments)
23
  - **GPU**: A100-80GB via Modal
24
 
25
- ## How to use
26
 
27
- ```python
28
- from voxcpm import VoxCPM
29
-
30
- model = VoxCPM.from_pretrained("openbmb/VoxCPM2", device="cuda", load_denoiser=True, optimize=True)
31
- # Apply LoRA weights manually or load via adapter
32
- ```
33
-
34
- ## Evaluation results
35
-
36
- | Experiment | Config | Train Loss | Val Loss |
37
- |-----------|--------|-----------|----------|
38
  | exp1 | r32, lr=1e-4 | 0.823 | 0.918 |
39
  | exp2 | r16, lr=5e-5 | 0.765 | 0.899 |
40
  | exp3 | r64, lr=1e-4 | 0.838 | 0.908 |
@@ -42,5 +32,31 @@ model = VoxCPM.from_pretrained("openbmb/VoxCPM2", device="cuda", load_denoiser=T
42
  | exp5 | r32, lr=2e-4 | 0.832 | 0.896 |
43
  | **exp6** | **r32, lr=5e-5, half data** | **0.935** | **0.872** |
44
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
  ## Part of DreamVoice
46
- This model is part of [DreamVoice](https://huggingface.co/spaces/build-small-hackathon/dreamvoice) — bedtime stories in a parent's cloned voice.
 
14
 
15
  LoRA adapter fine-tuned on VoxCPM2 for bedtime story narration in English.
16
 
17
+ ## Training
18
+ - **Base**: `openbmb/VoxCPM2`
19
  - **Dataset**: LJSpeech (6,550 clips, ~12h — half-data experiment)
20
  - **Method**: LoRA (r=32, alpha=64, targeting DiT attention layers)
21
+ - **Best**: exp6 (r=32, lr=5e-5, half data) — val loss 0.872
 
22
  - **GPU**: A100-80GB via Modal
23
 
24
+ ## Evaluation (6 experiments)
25
 
26
+ | Exp | Config | Train Loss | Val Loss |
27
+ |-----|--------|-----------|----------|
 
 
 
 
 
 
 
 
 
28
  | exp1 | r32, lr=1e-4 | 0.823 | 0.918 |
29
  | exp2 | r16, lr=5e-5 | 0.765 | 0.899 |
30
  | exp3 | r64, lr=1e-4 | 0.838 | 0.908 |
 
32
  | exp5 | r32, lr=2e-4 | 0.832 | 0.896 |
33
  | **exp6** | **r32, lr=5e-5, half data** | **0.935** | **0.872** |
34
 
35
+ ## Demo Audio
36
+
37
+ ### English bedtime stories (VoxCPM2)
38
+
39
+ | Mood | Audio |
40
+ |------|-------|
41
+ | Magical | [english_magical_mid.wav](samples/english_magical_mid.wav) |
42
+ | Funny | [english_funny_high.wav](samples/english_funny_high.wav) |
43
+ | Calming | [english_calming_low.wav](samples/english_calming_low.wav) |
44
+ | Dreamy | [english_dreamy_low.wav](samples/english_dreamy_low.wav) |
45
+
46
+ ### LoRA vs Stock comparison
47
+
48
+ | Type | Audio |
49
+ |------|-------|
50
+ | Stock VoxCPM2 | [comparison_stock_0.wav](samples/comparison_stock_0.wav) |
51
+ | LoRA exp6 | [comparison_lora_exp6_0.wav](samples/comparison_lora_exp6_0.wav) |
52
+
53
+ ### Reference voice
54
+ - [reference_voice.wav](samples/reference_voice.wav) — voice used for cloning
55
+
56
+ ## Files
57
+ - `lora_weights.safetensors` — LoRA adapter weights (12.8 MB)
58
+ - `lora_config.json` — LoRA configuration
59
+ - `training_state.json` — Training state
60
+
61
  ## Part of DreamVoice
62
+ [DreamVoice](https://huggingface.co/spaces/build-small-hackathon/dreamvoice) — bedtime stories in a parent's cloned voice.