Multi-role dubbing is divided into two types: Subtitle-Based Multi-Role Dubbing and Video-Based Multi-Role Dubbing.
I: Subtitle-Based Multi-Role Dubbing
Starting from v3.74, a "Subtitle Multi-Role Dubbing" feature has been added. Click the Subtitle Multi-Role Dubbing button on the left toolbar. In the pop-up window, import the SRT subtitle file you want to dub. Then, set a role for each subtitle to achieve multi-role voicing.
II: Original Voice Cloning and Multi-Role Dubbing Based on Video
In the Custom Video Translation function, you can select F5-TTS/index-tts/clone-voice/CosyVoice/GPT-SOVITS/Chatterbox etc. in the dubbing channel. Select the clone role, and the original sound will be used as a reference audio for dubbing to obtain a dubbing with the original tone.
Using the clone-voice Dubbing Channel
You need to install the https://github.com/jianchang512/clone-voice project. After opening the project homepage, read the instructions carefully. You can deploy the clone-voice project using the source code. If you are using a Windows system, you can also find Releases (https://github.com/jianchang512/clone-voice/releases) in the middle of the right side, download the integrated package directly, unzip it and double-click app.exe to start it.
When it shows that the startup is successful, fill in the default API address http://127.0.0.1:9988 into the HTTP address of Menu--TTS Settings--Original Tone Cloning clone-voice in the video translation software. After testing that there is no problem, you can start using it.
Using the CosyVoice Dubbing Channel
You also need to install the CosyVoice project. For the installation tutorial, see https://pyvideotrans.com/cosyvoice.html
Of course, you can also use a third-party integrated package, but the third-party integrated package does not support cloning tones, and can only specify fixed audio.
After installing according to the tutorial, go to this address https://github.com/jianchang512/cosyvoice-api/blob/main/api.py to download the api.py file, put it under the CosyVoice project, in the same directory as the webui.py file.
Then start api.py, and fill in the API address into the API address of Menu--TTS Settings-CosyVoice in the video translation software, the default address http://127.0.0.1:9233
Using the F5-TTS Dubbing Channel
You need to install the F5-TTS project, see https://pyvideotrans.com/f5tts.html for a detailed installation tutorial
Fill in the F5-TTS API address in the video translation software menu-TTS settings.
Select clone in the role selection on the main interface to perform cloned voice dubbing.
Note that, in addition to clone-voice supporting more than ten languages, F5-TTS and CosyVoice only support Chinese and English language cloning.
Setting Multiple Roles
Similarly, in the Custom Video Translation function, select a single video to be processed. After the translation stage is over, it will pause. At this time, you can set a separate role for each subtitle on the right interface to achieve multi-role dubbing.
In the dubbing role on the main interface, you need to select a default dubbing role. If you do not set it separately, all of them will use the default role.
