Buzz Hays wants to make sure his colleagues in Hollywood understand the pros and cons of generative AI, in particular, fine-tuning models.
And a good starting point for explaining the controversial topic is the work Hays, global lead for Google Cloud‘s entertainment industry solutions, and his team did to pull off Warner Bros. Discovery’s “The Wizard of Oz” at the Sphere event in Las Vegas back in August.
“The thing that got us through this project is a process that sometimes gets confused, and I’ve heard people use the terms incorrectly, but there’s a process called fine-tuning of the models,” Hays told Variety. “I could say, give me an output of the Tin Man doing something without the model understanding anything about the actual Tin Man, so it comes up with some vague approximation. But that’s not good enough when you’re doing something like this. So we actually fine-tuned our models, which is a process that basically allows any IP owner to take a base model that’s been trained and has been using all the sources that people have opted into, to contribute to. So it’s a legitimate model, but it doesn’t understand the Tin Man. So what we can now do is fine-tune it on the Tin Man to say, ‘OK, you know how people move. Now we have a person that looks like this. So whenever we refer to Tin Man, please use this version of Tin Man for that.’”
Hays continued: “In that case, we had fine-tuned models for each of the main characters in the movie, and that allowed us to have a faithful reproduction of those actors, no matter where they looked and how close they were to camera, and if they turned around. Things like, what does the back of the Tin Man’s head look like? Because there’s not a lot of that in the movie, but there’s a couple of shots in there that we could tune on so that in those occasions where he had to turn his head, we knew exactly what he looked like.”
Hays, who was a Hollywood producer for more than three decades before transitioning to his current position at the tech giant, says “all of the data that’s used for that is owned and maintained by the customer” and “it never gets assumed into the model.” The Google Cloud entertainment industry boss wants to make sure this concept is crystal clear moving forward, because he sees the tech only becoming more integral to the production process across the film and TV industries.
“That’s where the power of AI comes. I think especially with franchise titles, we have to have a certain likeness that is persistent throughout the series, then it starts to make sense that these fine-tune models exist for that particular reason,” Hays said. “So I think it kind of runs the gamut when it comes to how these things get leveraged as to what makes for a successful output and what doesn’t.”
Hays emphasized that while this tech is here to stay, Google Cloud has put in place “significant guardrails” to protect IP and likeness as it continues to expand throughout entertainment.
“We don’t allow our models to generate a known celebrity or likeness of anybody. There’s very careful protections put on those,” Hays said. “We have guardrails on things like violence. We have guardrails on generating children, a bunch of these things. So all these guardrails contribute to this indemnification policy idea that we will protect you but don’t do bad things, basically what it is, but we don’t even allow you to do bad things, because those those guardrails are protecting that.”
Hays points to how those principles were used for “The Wizard of Oz” at the Sphere project and says he “values” the restrictions put on what the Google Cloud team can do with generative AI.
“It was really about, how can we honor this original film?” Hays said. “We have a lot of restrictions that came from the studio that says, you can’t have an actress replace Dorothy’s performance, for example. You can’t replace voices. You can’t use motion capture to do a new performance that didn’t exist in the original, those kinds of things. I’m a big fan of restrictions in filmmaking. I think the best art comes from restrictions, not from unlimited possibilities. So it was a really interesting way to work on this.”
‘ The preceding article may include information circulated by third parties ’
‘ Some details of this article were extracted from the following source variety.com ’
