Dia-1.6B TTS : Greatest Textual content-to-Dialogue Technology Mannequin

In search of the precise text-to-speech mannequin? The 1.6 billion parameter mannequin Dia is likely to be the one for you. You’d even be stunned to listen to that this mannequin was created by two undergraduates and with zero funding! On this article, you’ll study concerning the mannequin, the right way to entry and use the mannequin and likewise see the outcomes to actually know what this mannequin is able to. Earlier than utilizing the mannequin, it might be acceptable to get acquainted with it.

What’s Dia-1.6B?

The fashions skilled with the purpose of getting textual content as enter and pure speech as output, are known as text-to-speech fashions. The Dia-1.6B parameter mannequin developed by Nari Labs belongs to the text-to-speech fashions household. That is an attention-grabbing mannequin that’s able to producing lifelike dialogue from a transcript. It’s additionally price noting that the mannequin can produce nonverbal communications like snicker, sneeze, whistle and so forth. Thrilling isn’t it? 

Tips on how to Entry the Dia-1.6B?

Two methods through which we will entry the Dia-1.6B mannequin:

1. Utilizing Hugging Face API with Google Collab
2. Utilizing Hugging Face Areas

The primary one would require getting the API key after which integrating it in Google Collab with code. The latter is a no-code and permits us to interactively use Dia-1.6B. 

1. Utilizing Hugging Face and Collab

The mannequin is offered on Hugging Face and could be run with the assistance of 10 GB of VRAM, supplied by the T4 GPU in Google Collab pocket book. We’ll show the identical with a mini dialog.

Earlier than we start, let’s get our Hugging Face entry token which will probably be required to run the code. Go to https://huggingface.co/settings/tokens and generate a key, in the event you don’t have one already. 

Be sure that to allow the next permissions:

Open a brand new pocket book in Google Collab and add this key within the secrets and techniques (Title must be HF_Token):

Notice: Swap to T4 GPU to run this pocket book. Then solely you’d have the ability to use the 10GB of VRAM, required for working this mannequin. 

Let’s now get our fingers on the the mannequin:

1. First clone the Dia’s Git repository:

!git clone https://github.com/nari-labs/dia.git

2. Set up the native bundle:

!pip set up ./dia

3. Set up the soundfile audio library:

!pip set up soundfile

After working the earlier instructions, restart the session earlier than continuing.

4. After the installations, let’s do the mandatory imports and initialize the mannequin:

import soundfile as sf

from dia.mannequin import Dia

import IPython.show as ipd

mannequin = Dia.from_pretrained("nari-labs/Dia-1.6B")

5. Initialize the textual content for the textual content to speech conversion:

textual content = "[S1] That is how Dia sounds. (snicker) [S2] Do not snicker an excessive amount of. [S1] (clears throat) Do share your ideas on the mannequin."

6. Run inference on the mannequin:

output = mannequin.generate(textual content)

sampling_rate = 44100 # Dia makes use of 44.1Khz sampling fee.

output_file="dia_sample.mp3"

sf.write(output_file, output, sampling_rate) # Saving the audio

ipd.Audio(output_file) # Displaying the audio

Output:

The speech could be very human-like and the mannequin is doing nice with non-verbal communication. It’s price noting that the outcomes aren’t reproducible as there are not any templates for the voices. 

Notice: You’ll be able to strive fixing the seed of the mannequin to breed the outcomes.

2. Utilizing Hugging Face Areas

Let’s attempt to clone a voice utilizing the mannequin through Hugging Face areas. Right here we have now an choice to make use of the mannequin instantly on the utilizing the net interface: https://huggingface.co/areas/nari-labs/Dia-1.6B

Right here you may move the enter textual content and moreover you can even use the ‘Audio Immediate’ to copy the voice. I handed the audio we generated within the earlier part. 

The next textual content was handed as an enter:

[S1] Dia is an open weights textual content to dialogue mannequin. 
[S2] You get full management over scripts and voices. 
[S1] Wow. Wonderful. (laughs) 
[S2] Strive it now on Git hub or Hugging Face.

I’ll allow you to be the decide, do you’re feeling that the mannequin has efficiently captured and replicated the sooner voices?

Notice: I obtained a number of errors whereas producing the speech utilizing Hugging Face areas, strive altering the enter textual content or audio immediate to get the mannequin to work.

Issues to recollect whereas utilizing Dia-1.6B

Right here are some things that it is best to consider, whereas utilizing Dia-1.6B:

• The mannequin shouldn’t be fine-tuned on a particular voice. So, it’ll get a unique voice on each run. You’ll be able to strive fixing the seed of the mannequin to breed the outcomes.
• Dia makes use of 44.1 KHz sampling fee.
• After putting in the libraries, ensure to restart the Collab pocket book. 
• I obtained a number of errors whereas producing the speech utilizing Hugging Face areas, strive altering the Enter Textual content or Audio Immediate to get the mannequin to work.

Conclusion

The mannequin outcomes are very promising, particularly after we see what it will probably do in comparison with the competitors. The mannequin’s greatest power is its assist for a variety of non-verbal communication. The mannequin has a definite tone and speech feels pure, however alternatively because it’s not fine-tuned on particular voices, it won’t be straightforward to breed a specific voice. Like some other generative AI software, this mannequin must be used responsibly.

Steadily Requested Questions

Obsessed with know-how and innovation, a graduate of Vellore Institute of Expertise. At the moment working as a Knowledge Science Trainee, specializing in Knowledge Science. Deeply serious about Deep Studying and Generative AI, desperate to discover cutting-edge methods to resolve complicated issues and create impactful options.