YouTube has announced its plans to launch a new AI-powered dubbing tool that will assist creators in translating their videos into different languages, aiming to help them reach a global audience. Revealed at VidCon 2023, this initiative seeks to provide creators with a convenient and cost-effective solution for translating content they do not speak themselves. The tool will utilize Aloud, developed by Google, and will incorporate the expertise of the AI team from Area 120, an experimental tech division within YouTube’s parent company.
The translation system, as described on the official Aloud website, begins with the AI transcribing the video into a script. Creators can then edit the transcription to correct errors, add clarifications, or highlight critical text where timing is crucial. The edited script is then returned to Aloud, which automatically translates the video into the desired language. Once completed, creators can upload the new audio tracks onto their original video to publish the newly dubbed content.
According to a Google representative, creators do not need to understand the languages they are dubbing into. Aloud handles complex tasks such as translation, timing, and speech synthesis, while creators can focus on reviewing and verifying the transcription.
The specific launch date for the Aloud update has not been announced yet. However, YouTube is already working on expanding the capabilities of the AI. Currently, Aloud can only translate English content into Spanish or Portuguese, but there are plans to support other languages, including Hindi, Indonesian, and different dialects.
In the future, YouTube plans to introduce additional features to enhance enunciation, including voice preservation, improved emotion transfer, and even lip reanimation. The platform will also implement safeguards to ensure only creators can dub their own content.
The Aloud AI is currently being tested with hundreds of YouTube creators, with plans to expand the program over time. As of June 2023, more than 10,000 videos have been dubbed in over 70 languages. Creators can join the early access program by completing the official Google Docs form.
To experience an example of an Aloud dub, viewers can watch the channel trailer for the Amoeba Sisters channel on YouTube. By selecting the Spanish audio track in the gear icon menu, they can hear the robotic voice that the AI creates.
YouTube’s introduction of the AI-powered dubbing tool demonstrates its commitment to supporting creators in reaching a global audience by breaking down language barriers and expanding accessibility to diverse viewers worldwide.
