How Pixar’s Use of AI Ignited the Fire in ‘Elemental’

Pixar Harnesses AI to Bring Fire to Life in Elemental

The Challenge of Animating Fire

Pixar, the renowned animation studio, recently faced a unique challenge while working on their new movie, Elemental. The film, directed by Peter Sohn, features characters made of different elements, including fire. The animators at Pixar found it difficult to capture the ethereal nature of fire onscreen and make it visually appealing and believable. While they had previously developed tools to create flame effects, shaping fire into a character proved to be a conundrum.

According to Sanjay Bakshi, the VFX supervisor at Pixar, their fire simulations aimed to mimic reality and appeared very naturalistic. However, when they attempted to mold fire into a character like Ember, one of the movie’s leads, the results were terrifying rather than captivating. The fire appeared too realistic and resembled a human figure made of real pyro, similar to Studio Ghibli’s Calcifer or Nicolas Cage’s Ghost Rider.

Exploring Machine Learning as a Solution

To solve this problem, Pixar turned to machine learning. Paul Kanyuk, a crowds technical supervisor at Pixar, had been working on crowd animation for years and was familiar with the challenges of making clothes on groups of people look realistic. In his search for a solution, Kanyuk discovered research on neural style transfer (NST), a type of artificial intelligence commonly used to transform images to resemble the style of famous artists like Van Gogh or Picasso.

Around 2019, Kanyuk came across a paper from Siggraph Asia that explored using NST to manipulate voxels (3D pixels with volume) in animation. He believed that NST could be applied to Pixar‘s fire problem. However, he cautioned that machine learning outcomes are uncertain, with only about a 50% chance of success. Nevertheless, Kanyuk and Sohn, who also signed on as the film’s director, decided to pursue all ideas.

Kanyuk enlisted the help of Disney Research Studios, a lab specializing in AI and machine learning. After several meetings, they finally cracked the problem. The team created a set of swirly, pointy, and cartoonish flames called the “fleur-de-lis.” By combining these flames with the original fire simulation using NST, they achieved the desired movement, intensity, and Pixar‘s unique style. This breakthrough allowed them to harmonize the look of Ember in a truly unique way.

The Computing Power Challenge

While NST proved to be a successful solution, implementing it on a large scale presented another challenge. Rendering all 1,600 shots from Elemental using NST required significant computing power, especially since it required numerous graphic processors. At first, Pixar believed they could only use NST for Ember’s close-up shots due to resource limitations.

However, they realized that to maintain consistency, they needed to apply NST to other fire characters as well. This increased the requirements and prompted the animators to find a solution. They tapped into the GPUs available on every Pixar animator’s computer, virtualized them, and utilized half of the GPU’s processing power overnight. This approach significantly accelerated the rendering time from five minutes per frame to just one second.

Eventually, they succeeded in rendering all the necessary shots, thanks to the combination of NST and efficient use of computing resources. Kanyuk emphasizes that Pixar has only scratched the surface of what NST can accomplish and is excited about the imagery they can create using this technology.

Editorial: The Intersection of Art, Technology, and Emotion

Pixar‘s use of AI and machine learning to animate fire in Elemental highlights the studio’s commitment to pushing the boundaries of animation techniques. It exemplifies the meeting of art and technology, with the latter serving as a tool to express emotions and create connections between the audience and the film.

The success of this endeavor demonstrates the immense potential of AI in the creative field. By combining human artistry and the capabilities of machine learning algorithms, artists can achieve results that were previously unattainable. Pixar‘s willingness to embrace new technologies showcases their dedication to innovation and their ongoing pursuit of visual excellence.

However, it is crucial to remember that technology should always be viewed as a tool rather than a replacement for human creativity. While machine learning can assist in creating captivating visuals, it is the artists’ vision, storytelling, and emotional depth that truly resonate with audiences. The perfect balance between art and technology allows for compelling storytelling with a touch of technical brilliance.

Advice for Future Animation Projects

Pixar‘s experience with Elemental offers valuable insights for future animation projects seeking to leverage AI and machine learning. Here are a few key takeaways:

1. Embrace Collaboration:

Collaboration between different fields, such as animation and AI research, can lead to groundbreaking solutions. Pixar‘s partnership with Disney Research Studios demonstrates the power of collaboration in finding innovative ways to overcome challenges.

2. Continual Learning:

It is essential to stay abreast of the latest research and advancements in AI and machine learning. Pixar‘s Paul Kanyuk’s dedication and curiosity led him to discover the potential of NST as a solution for their fire animation problem. Continuous learning and exploration of emerging technologies can open doors to unexpected creative possibilities.

3. Balancing Efficiency and Creativity:

While deploying AI and machine learning techniques can significantly enhance efficiency, it is crucial to preserve the artistic vision and emotional depth of a project. The success of Pixar‘s animation relied on finding the right balance between automating certain aspects and retaining the artist’s touch.

4. Resource Optimization:

When working with computationally intensive tasks, optimizing computing resources can alleviate constraints. Pixar‘s approach of virtualizing GPUs and utilizing idle processing power demonstrates a practical solution to handle resource-intensive rendering processes.

In conclusion, Pixar‘s use of AI and machine learning to animate fire in Elemental showcases the studio’s commitment to artistic innovation and technical excellence. By embracing collaboration, continual learning, and a careful balance between technology and creativity, future animation projects can explore new frontiers and captivate audiences in unprecedented ways.