The Programmer News Hubb
Advertisement Banner
  • Home
  • Technical Insights
  • Tricks & Tutorial
  • Contact
No Result
View All Result
  • Home
  • Technical Insights
  • Tricks & Tutorial
  • Contact
No Result
View All Result
Gourmet News Hubb
No Result
View All Result
Home Technical Insights

How Google improves video voice dubbing through deep learning

admin by admin
December 26, 2022
in Technical Insights


With less than 20% of the world’s population speaking English as their first or second language, Google is ramping up the efficiency of video voice dubbing with technologies for cross-lingual voice transfer and lip reanimation using deep learning and TensorFlow. 

The first technology keeps the voice similar to that of the original speaker and the second adjusts the speaker’s lip movements in the video to better match the audio generated in the target language. 

Google performs cross-lingual voice transfer by creating synthetic voices in the target language that best fits the speaker’s original voice. This technology was made possible by first pre-training a multilingual text-to-speech (TTS) model based on the cross-language voice transfer approach. Then Google, fine-tuned the model parameters by retraining with a fixed mixing ratio of the adaptation data and original multilingual data 

For lip reanimation, Google trained a multistage model that maps audio to lip shapes and the appearance of the speaker. Then they used the original videos of the speaker for training, isolated the frequency and represented the faces in a space that decouples 3D geometry, head pose, texture and lighting.Then a GAN-based approach is used to blend these synthesized textures with the original video before later refinement using a  super-resolution network.

Google’s new system for video dubbing through deep learning uses a combination of natural language processing, speech recognition, and audio-video analysis to create more natural-sounding and accurate voice dubs. 

“We strongly believe that dubbing is a creative process. With these techniques, we strive to make a broader range of content available and enjoyable in a variety of other languages,” Google wrote in a blog post.



Source link

Previous Post

SD Times Q&A: Five things to look for in 2023

Next Post

Google announces innovations in privacy-enhancing technologies

Next Post

Google announces innovations in privacy-enhancing technologies

Recommended

Creatio’s latest updates speed up building of no-code apps

6 months ago

Why the Grumpy Designer Isn’t Sold on the AI Hype Machine

3 months ago

ngrok adds new package for adding ingress to Go applications

3 months ago

An Introduction to SQLite with Python

7 months ago

Atlassian Intelligence provides developers a virtual teammate

1 month ago

The Bad Freelance Habits You Should Avoid

5 months ago

© The Programmer News Hubb All rights reserved.

Use of these names, logos, and brands does not imply endorsement unless specified. By using this site, you agree to the Privacy Policy and Terms & Conditions.

Navigate Site

  • Home
  • Technical Insights
  • Tricks & Tutorial
  • Contact

Newsletter Sign Up.

No Result
View All Result
  • Home
  • Technical Insights
  • Tricks & Tutorial
  • Contact

© 2022 The Programmer News Hubb All rights reserved.