Multi-character dubbing includes two methods: Subtitle-Based Multi-Character Dubbing and Video-Based Multi-Character Dubbing.
1: Subtitle-Based Multi-Character Dubbing
Starting from v3.74, the "Subtitle Multi-Character Dubbing" feature has been added. Click the Subtitle Multi-Character Dubbing button on the left toolbar, import the SRT subtitle file in the pop-up window, and assign a character to each subtitle line to achieve multi-character voice synthesis.
2: Video-Based Voice Cloning and Multi-Character Dubbing
In the Custom Video Translation feature, you can select dubbing channels such as F5-TTS/index-tts/clone-voice/CosyVoice/GPT-SOVITS/Chatterbox. When you choose the clone character, the dubbing will use the original voice as a reference to produce a voice with the original timbre.
Using the clone-voice Dubbing Channel
You need to install the project from https://github.com/jianchang512/clone-voice. After opening the project homepage, read the instructions carefully. You can deploy the clone-voice project using the source code. For Windows systems, you can also find Releases (https://github.com/jianchang512/clone-voice/releases) in the middle-right section and download the integrated package directly. After downloading and extracting, double-click app.exe to start it.
Once it shows a successful startup, enter the default API address http://127.0.0.1:9988 into the video translation software under Menu → TTS Settings → Original Voice Cloning clone-voice in the HTTP address field. Test it to ensure no issues, and then you can start using it.
Using the CosyVoice Dubbing Channel
Similarly, you need to install the CosyVoice project. For installation instructions, see https://pyvideotrans.com/cosyvoice.html.
Alternatively, you can use third-party integrated packages, but they do not support voice cloning and only allow fixed audio assignments.
After installation, go to https://github.com/jianchang512/cosyvoice-api/blob/main/api.py to download the api.py file and place it in the CosyVoice project directory, in the same folder as the webui.py file.
Then start api.py and enter the API address into the video translation software under Menu → TTS Settings → CosyVoice in the API address field. The default address is http://127.0.0.1:9233.
Using the F5-TTS Dubbing Channel
You need to install the F5-TTS project. For detailed installation instructions, see https://pyvideotrans.com/f5tts.html.
Enter the API address into the video translation software under Menu → TTS Settings → F5-TTS API Address.
Select the clone character in the main interface to perform voice cloning dubbing
Note: While clone-voice supports over ten languages, F5-TTS and CosyVoice only support Chinese and English voice cloning.
Setting Multiple Characters
Also, in the Custom Video Translation feature, select a single video to process. After the translation phase ends, it will pause. At this point, you can assign a specific character to each subtitle line in the right-hand interface to achieve multi-character dubbing.
In the main interface, you need to select a default dubbing character. If no individual assignments are made, all subtitles will use this default character.
