Indonesian - Training with Mozilla Datasets

[tansilandre]

Introduction

Hello, I am Tansil from Indonesia.
I am a newbie here, just want to test if this TTS working or not and its working!!!

First thing first, I want to say thank you for @thorstenMueller for all the guide he had uploaded on Youtube, very helpful! Good work!!!
After hours of hours getting back to back reading documentation here and watching Thorsten video, I have tried 2 different system to do a training:

1. Google Collab, it is pretty easy and straight forward also free, but the limitation on GPU getting me nowhere near the model that I expected
2. I had a training with my PC (with 3060 GPU) on windows but it's getting so many crashes and not stable
3. So the solution is I am using WSL2 with windows, with CUDA activated (trust me it's hard to setup but I can give a tip)

Dataset

Since I have a bad microphone, and I want to test the 'water' first, I am using Mozilla datasets. I also aim to use Indonesian language because that's what I needed. So what I done are:

1. Download the Indonesian speaker datasets on Mozilla
2. Create a python script to group and filter a good datasets
3. I found 3 different speaker with 1500+ recorded audio and I use audio with the up_vote > 0 and down_vote < 1 (up_vote and down_vote is on mozilla datasets)
4. Unfortunately from the 3 different speaker, I only could get 1 good model (minimum 50k step)

Difficulties

1. When I first create a Model, I wasn't aware there are 2 Model needed to generate a good voice (TTS & Vocoder)
2. With only TTS Model I could generate a reasonable voice but sound robotic
3. I want to try to do a training on Vocoder but no luck to find a documentation about it
4. Gaining more knowledge, I decided to do training again with VITS (it's combined TTS model + Vocoder model)

Results

The good news is its working as I expected! Here are some output that I want to share:

1. only TTS model with no Vocoder

TTSModelNonVocoder.mp4

2. with VITS training (TTS&Vocoder)

VITS.mp4

3. long version of VITS

test_podcast.mp4

Questions

1. How can we change the voice in the model? if so, how many minutes/ sentences datasets required to change it?
2. I have trained TTS model but I cannot do further Vocoder training, could anyone share how to do a Vocoder Training? Because the TTS 7.0 is not relatable with the current documentation

[erogol]

Why 0.7.0 is not relatable with the current docs?

[tansilandre]

on Vocoder Training, it's very different between training_vocoder.py and documentation on Trainin a Vocoder section

[tansilandre]

I also cannot find on docs folder a relatable documentation about vocoder

[kin0303]

Hi, Mr. Tansil, I'm Dhea from Indonesia too, may I know what model did you train with? I'm trying to use tacotron2 but I always meet problems.

Hai mas tansil, aku dhea dari indonesia juga boleh tau nggak kamu training nya pake model apa? Aku lagi nyoba pake tacotron2 tapi selalu bermasalah.

[erogol]

@tansilandre check the recipes for vocoder training.

[tansilandre]

@BlackMamba1122 pake glowTTS lumayan bagus tapi kaya robot, kalo mau pake VITS. DM aja Dhea kalo emg mau detailnya ke [email protected]

[ardha27]

itu ditrain berapa lama kak?

[kin0303]

itu ditrain berapa lama kak?

Halo, aku juga sempet nge training vits untuk multilanguage. Untuk trainnya aku lumayan lama, sebulanan lebih hasilnya seperti ini:
https://drive.google.com/drive/folders/1qOa8jfzKRRDToN7bItJ20fL6ZNhu8LC4?usp=sharing

[BayuSasongko]

Halo @kin0303 bagaimana langkah2 cara melakukan training pada suara kita sendiri ?
saya ada project voice clone jika boleh bsa join ke [email protected]. Thx u

[jerichosiahaya]

@kin0303 how many steps did it take from you? I've trained using 4k steps and the result is not really good.

[irfanamal]

@kin0303 Boleh tau training sebulanan lebih itu pake device (GPU) apa?

[Ryuodan]

can you show us the graphs from tensorboard and how many steps did it take from you? I'm training a new model and facing some problems.