Author | James The ups and downs of the US election have come to an end. At the same time, some people couldn't stand the ups and downs of emotions and decided to temporarily go to the theater to block some information. Among the recent North American box office hits, the movie "

author | james

The ups and downs of the US election have come to an end. At the same time, some people couldn't stand the ups and downs of emotions and decided to temporarily go to the theater to block some information. In the recent North American box office, the movie "Here" (Here), created by the original cast of "Forrest Gump" and starring Tom Hanks, premiered on November 1, with a box office of approximately 5 million US dollars, ranking 5th. Although the performance is not outstanding, there are some highlights worth mentioning.

First of all, the movie tells the story of the life of a traditional white man like Forrest Gump that takes place in a living room in New Jersey. The audience liked the movie more than the blue camp critics.

Secondly, the movie is a movie that heavily uses AI face-changing technology. But in the U.S. market, no one seems to take AI too seriously, whether it’s professional commentators or ordinary viewers.

You should know that in " under the stranger " and " legend ", our famous characters have been changed or repainted with AI. However, in the Chinese market, just using AI alone is not enough. It was enough to attract criticism from many viewers.

Many viewers believe that the facial expression of Jackie Chan after the AI ​​face-swapping in "Legend" is stiff, which makes people laugh, and even causes laughter in provocative scenes. Some viewers also believe that as long as AI is used, regardless of whether the effect is good or not, it is a kind of emotional consumption and lacks sincerity.

Third, Hanks is even willing that the AI ​​company will continue to use his facial image after his death so that his image can be "immortalized" in future movies. This behavior even triggered protests from actors, protesting that he affected the acting space of young actors.

In the United States, which has just experienced an unprecedented general strike in Hollywood, what is the attitude of film and television practitioners and consumers towards AI? Perhaps there will be a redistribution of interests, and there will be visible dissatisfaction and resistance, but more often than not, AI's entry into the film and television industry is "moistening things quietly", and before you know it, it has already spread everywhere. yes.

Real-time face-changing, creating a "magic mirror of age-reversing" for actors

"This Place of Peace" is one of the first full-length films in Hollywood to be built around AI visual effects. The film uses real-time AI facial transformation technology developed by visual effects company Metaphysic to allow actors Tom Hanks and Robin Wright to span a 60-year age span.

During the filming process, the crew watched two monitors at the same time. One showed the actor's true appearance, and the other showed the processed "reverse age" appearance.

metaphysic trained a custom machine learning model on footage from Hanks and Wright’s previous films, feeding it extensive data on facial movement, skin texture, and appearance under different lighting conditions and camera angles.

As a result, any facial transformations generated by this model are instantaneous, eliminating the need for traditional months-long manual post-production processes. "It's incredible to see Metaphysic's AI-generated content integrated perfectly into the live footage! The actors can even use the technology as a ' 'Mirror of Youth' - testing your younger self's performance choices in real time. This feedback loop and youthful representation are absolutely essential to achieve authentic, enjoyable results."

metaphysic even used the same technique to dominate a show. The entire concert scene was shown to the public with face-swapping models. At the 2022 " America's Got Talent " (agt), metaphysic demonstrated the AI-generated rock star Elvis. The entire process was performed by motion capture actors and the trained Elvis model in real time, directly on the TV. on the screen.

Looking back on the spring of 2023, when Entertainment Capital visited visual effects giants in the industry such as Tiangong Yicai, the commonly used face-changing technology required a lot of heavy manpower investment to avoid "cross-talk". At the time, the cases cited by their staff focused on the perfection of its effect and its ability to be seamlessly integrated into the production line.All face-changing effects can be enabled and disabled in the original pipeline with one click, which provides great convenience for post-production.

However, I am afraid that no matter how convenient is for post-production, it is not as simple as producing the finished product directly from the filming studio.

metaphysic said that streams AI-generated photorealistic content at up to 30 frames per second and maps it to real-world scenes. is a huge advancement in face replacement technology and will be useful for creating immersive content in the near future. ar/vr, gaming and entertainment experiences are critical.

In fact, another thing worth noting is that the technology used by this time is not a "new" technology. To be precise, it is not the same wave of as the "aigc" represented by chatgpt, midjourney, etc. In principle, it is closer to the algorithm used by deepfake in the early years, that is, generative adversarial network (gan) . This means that this technology is of the same origin as the domestic "AI face-changing" technology that has been used in places such as " Wandering Earth 2" and "Legend", but has only undergone secondary development and optimization by Metaphysic.

Since 2014, gan has made significant progress in the fields of image generation and style transfer, becoming one of the core algorithms of deepfake technology. The diffusion model concept proposed in 2015 has developed rapidly after 2020 and is applied to high-quality "Venture Picture" scenes. The transformer proposed in 2017 has become the basic architecture of large language models (such as the gpt series).

gan is equivalent to letting two AIs act as "painters" and "identifiers" respectively. One draws and the other distinguishes good from bad. Until the "identifier" can no longer distinguish which are human works, the "painter" draws a picture that meets the requirements. Requested image. Therefore, this technology will be very effective in "lossless image enlargement", "watermark removal" and face replacement.

Xu Jian, the visual effects director of "The Wandering Earth 2", once shared his technical route selection process on Zhihu. He pointed out that the mainstream age-reverse face-changing methods in the world at that time were divided into three categories. The completely computer-generated route represented by

wētā fx requires actors to wear motion capture equipment to create a digital human, and then change the face of the digital human. The route represented by Industrial Light and Magic requires a surround camera to collect necessary data by shooting infrared lenses, and then only uses a computer to generate the face . These methods have high budgets, lack of relevant domestic talents, and the collection equipment at that time was too heavy and inconvenient to wear.

In "Captain America: Civil War", the visual effects team used Robert Downey Jr. 's early film and television works and visual materials to train an AI model based on the deepfake algorithm and complete the face change. Deepfake has a long history, and the earliest works that caused a stir were trained by netizens on their own home computers, which has proven that it is easy to use.

According to Xu Jian, as early as a year and a half before the filming of "The Wandering Earth 2" started, the visual effects team began technical experiments. After testing 5 or 6 face-changing methods, the visual effects team finally decided to use deepfake, and the final iteration was 500 After tens of thousands of times, replace the real footage.

is different from previous deepfakes that relied on frame-by-frame operations. The metaphysic method generates face-changing effects in real time by analyzing facial feature points and mapping them to trained age groups.

This is largely due to the company's progress in engineering capabilities and the substantial improvement in chip computing capabilities. In the early days of deepfake technology in 2017, producing a deepfake video of just a few minutes could take hours or even days, depending on the length of the video and the performance of the hardware used.

With the advancement of hardware and algorithms, the efficiency of deepfake generation has been significantly improved. Specialized hardware such as modern GPUs and TPUs significantly speeds up the computing process. In terms of algorithms, improved architectures such as stylegan reduce generation time to just minutes or even seconds per frame. As engineering optimization continues and hardware continues to increase, it is only a matter of time before gan-based face swapping transitions to real-time rendering.

Although it is "straight out", taking into account the quality requirements of the big screen, the facial effects generated by AI will be further refined in the post-processing stage, including removing small flaws, adjusting facial light and shadow, and the earlobes, neck and other parts are modified by makeup artists A slight "upgrade" was made to ensure that the picture is more consistent with the real face.

At the same time, "This Place of Peace of Mind" achieves "leveraging strengths and avoiding weaknesses" through fewer martial arts scenes, and the possible technical disadvantages are not clearly felt in the actual effect.

"Legend" also adopted the method of deepfake in the later stage. According to director Tang Jili, face-changing Jackie Chan's appearance rate in the movie exceeded 70%. However, "Legend" challenged the most difficult task of action scenes + daytime light sources + long-term face-changing. From the perspective of the finished product, there is still a relatively obvious gap with the foreign face-changing effects. It can be said that courage is commendable .

metaphysic's technology not only works well, but also significantly reduces the cost of . The New York Times report reminds that convincing audiences they are seeing Hanks and Wright in their 20s would require hundreds of vfx artists, tens of millions of dollars and months of post-production work by major studios. You wouldn't normally spend this much money on a movie like "This Is Here".

Although the production cost of "This Place" is about $50 million, which seems a bit high, it is less than 1/4 of the budget of some Marvel movies.

You have no choice but to embrace, AI is already on the way

"Heart of Peace" director Zemeckis mentioned that he could have chosen multiple actors for each role, which was a common practice in filming film and television dramas in the past.

But if you do this, it is okay if it is a group portrait of multiple characters. In a place like a stage play, the audience will quickly find the problem that the characters do not look alike. Traditional makeup methods can make Hanks look ten years younger, but they cannot make him 50 years younger. The problem with computer special effects is time and money.

However ai software changes the cost calculation to . Metaphysic's technology generates a real-time digital human with just a few clicks of a button, without six months of production lag or huge funding requirements.

ed Ulbrich has spent 30 years in the traditional visual effects industry. His works include: "Top Gun: Maverick", "Black Panther", "Avengers: Infinity War", "Suicide Squad", "Benjamin Button" "Strange Case" and "Titanic". He later joined metaphysic to oversee production and quality control. "Quality, speed and cost" were the keys to his decision to transform.

“My career flashed before my eyes, I could take an entire team from my old company and work with the best cgi tools money could buy and yet not be able to deliver the results we are showing now. And this It only takes milliseconds. It's time to say goodbye to cgi. How can I continue to make images with horses, carts, rocks and sticks in today's world with a clear conscience? "

Of course, no matter how good the technology is, if you want to use , you must do it. Fundamentally, it is also inseparable from the support of the actors themselves.

Tom Hanks is over sixty years old, but he needs to play his younger self in the movie. He has expressed a strong interest in extending his acting career.

"If I could play someone younger than me - I always read about characters that I would love to play and think, oh my gosh, I'm 68 and I can't play Iago because he's in his 20s Years old. But if I have the chance, I will not hesitate to act."

Perhaps it is not a coincidence that Jackie Chan, who has also achieved great success in his acting career, also highly recognizes and supports the face-changing technology, which is why "Legend" 》Collaboration inside.

Jackie Chan expressed at the press conference that although he could not truly return to his youth, he could realize this dream through film technology. He even wrote a poem: "In an instant, my hair and black hair turn gray, who will I tell when I grow old? If the spring breeze has pity on the flowers, can you allow me to be young again!"

Face-changing technology not only helps older actors, but also reduces the need for makeup and special effects. The burden greatly expanded the actor's acting career, and they "continued to perform" even after their death.

When Hanks saw the "magic mirror" effect generated in real time, he was completely conquered, describing it as a "viewing experience that spans time and space." He is even willing that the AI ​​company will continue to use his facial image after his death so that his image can be "immortalized" in future movies.

metaphysic said it has partnered with Hollywood's major talent agency CAA to help stars control the use of their own data sets. However, all promises made by technology manufacturers cannot be accepted without hesitation.

actor dan dewhirst signed a contract with tech company synthesia in 2021, becoming one of the first actors to have his likeness turned into an avatar for . The other party provides an acknowledgment that the avatar cannot be used for illegal or objectionable purposes.

However, two years later, he received a link to a false CNN report that showed him posing as a television news anchor to spread fake news promoted by Venezuela's Nicolás Maduro government. His profile picture told viewers that the country's "economic situation was good," along with a series of other inaccurate and controversial narratives.

Last month, Dewhirst released a statement about his experience through the UK's Screen Actors Guild Equity to warn others against signing AI avatar contracts.

Hanks said that he actually had some fears.

"They can make a movie about me for the next 122 years if they want to, should they be legally allowed to do that? What about my legacy?"

But he is undeterred, on the contrary, he seems Documents are ready to be signed. "Come on, let's work out a deal now."

Of course, others don't necessarily think so. Shizhifu mentioned that 60-year-old Nicolas Cage felt scared when he was interviewed by The New Yorker. "They'd have to put me into a computer and match my eye color and changes - I don't know. they'd just steal my body via digital ai and do whatever they want with it ."

This is just over the past year Come on, one of the many film and television people who protest against AI.

In the UK, ITV released a recruitment notice in early October, looking for a "generative AI innovation leader", saying that the purpose of AI is to improve production efficiency and has no intention of replacing human creators.

In response to this, many screenwriters including Lisa McGee, the screenwriter of the British drama "Derry Girls" and Jack Brooke, the screenwriter of "", collectively protested. The British Drama Writers Guild has also expressed concern about this, believing that funds should be used to hire human writers and creative talents rather than purchase AI tools. Nowadays, many young actors are protesting against the AI-based behavior of Tom Hanks, believing that this will undoubtedly hinder the acting opportunities of young actors.

"Legend" made a high-profile mention of AI as a selling point during its domestic promotion. This incident also aroused the rebellious psychology of some viewers. The audience was very active in criticizing the "hardness" of the face-swap results and other issues. In fact, in Hollywood, if you publicly say that you use AI technology, there is also a certain risk of triggering the backlash of practitioners.

Even Hanks himself admitted in an interview that if he could always play young roles, it would make young actors in their 20s miss out on opportunities.

The history of Hollywood can be described as a series of technological leaps, starting with the invention of the camera. Each new technology resulted in the loss and creation of jobs and the reorganization of the entire industry. However, past technological leaps have tended to have smaller impacts: home video transformed movie distribution, digital cameras transformed filmmaking, CGI transformed visual effects.

"The difference is that this time AI has the potential to disrupt multiple aspects of our production line, and it feels like a disruptor for the entire industry." Lori McCreary, CEO of Revelations Entertainment and a Producers Guild of America board member, said she worked with Morgan Freeman co-founded the production company.

ai is evolving so quickly, and many people in Hollywood know so little about it, that it's difficult to predict where it will ultimately be most advantageous and where it will hit first.Susan Sprung, CEO of the Producers Guild of America, said:

"Everyone is nervous, but no one knows exactly what to worry about."

But so far, the application of AI in Hollywood is more practical than large-scale disruption and The nightmare scenario of unemployment is much milder and more gradual. The changes that were billed as an industry earthquake didn't happen, but rather slowly seeped into the soil. Cristóbal Valenzuela, CEO of

runway, said,

"Too many people in Hollywood today think that just type 'movie' and hit enter and you'll get a movie. Once you start using it, you realize: 'Oh, actually It's not that useful yet, it's full of flaws, and it doesn't really live up to my expectations at all.'"

The New York Times found that although there are many AI optimists in the film industry, they are often reluctant to express this view publicly for fear. It is considered to be favoring the machine, or appears to encourage colleagues to lose their jobs. Sometimes enthusiastic early adopters schedule interviews only to cancel at the last minute because their bosses are wary.

The current AI applications in the film and television industry are just like the AI ​​in "This Place" - invisible but everywhere, filling the entire screen. As long as you don't say it, I don't mention it, and the audience doesn't feel it's a drama, then we are still good friends.