AI overview
Artificial intelligence (AI) is revolutionising both visual effects (VFX) and practical special effects (SFX), significantly altering the role of designers in both fields. AI is now streamlining SFX and VFX processes across the board.
For VFX, AI-driven tools can automate tasks such as rendering complex simulations of natural phenomena like fire, water or smoke. These tools can analyse vast datasets to predict realistic movements and interactions, enabling quicker, more precise effects generation.
Similarly, in practical SFX, AI is aiding in the design and prototyping of physical models and animatronics, enhancing accuracy and efficiency in creating life-like creatures and environments.
The creative potential of AI extends to both VFX and practical effects. In VFX, machine learning algorithms can generate entirely new textures, environments and characters, reducing manual labour while offering fresh creative possibilities.
AI can also help blend practical SFX with digital VFX by tracking movement and adjusting lighting automatically, making it easier to integrate physical and digital elements seamlessly.
However, as AI takes over more technical tasks, SFX and VFX designers must adapt by acquiring new skills, working alongside AI to ensure that the generated effects - whether digital or physical - maintain the artistic vision and integrity of the project. Their expertise remains essential in guiding AI tools to achieve the desired results.
Download a PDF version of this page.
How can I prepare for the future?
To prepare for the future, SFX and VFX designers could consider embracing cutting-edge technologies like AI, real-time rendering, virtual production, on set tracking and autonomous robotics.
By keeping up with advancements in both physical effects (SFX) and digital effects (VFX), they can streamline workflows and elevate the visual quality of their work.
Staying proficient with AI-driven tools, understanding how to manipulate them for your own purposes and mastering the integration of physical and digital effects will be key to maintaining your role as a creative and technical leader in production. Literally asking AI tools how to learn about some of these subjects can be a great way to get started.
It is also now possible to run AIs directly on your own computer and even train the AI with your own historical data set, giving you the opportunity to create your own bespoke AI that understands your style and workflows.
Keep a close eye out for how these progress and consider labelling, archiving and storing your own data/production notes for future use. You might be able to create your perfect assistant in the future.
What AI tools can I use right now?
Asset tracking systems can now automatically carry out resource allocation from a known stock list by combining shooting schedules with predicted equipment lists. This will still need checking but with some bespoke training on your way of working and preferences, has the potential to do this whole process automatically.
When this is combined with Internet of Things (IoT) solutions, real time data about the performance and condition of the equipment can also be made available, allowing an AI to predict equipment failure and maintenance schedules.
Full tech-vis simulation environments allow a SFX and VFX team to collaboratively work through rigging challenges before they are encountered on set. From a human created film simulation, AI is able to create pick lists, calculate lifting loads, predict power consumption, to name a few capabilities.
Utilising new AI-based image correlation techniques, which are replacing traditional photogrammetry and videogrammetry pipelines, such as Gaussian Splats for face and performance capture.
AI computer vision is unlocking various markerless tracking techniques which is allowing tracking to take place on set with much less VFX work needed in post production.
Virtual production techniques now include screen panels that are water resistant and able to handle traditional SFX systems, allowing AI-based real-time rendering workflows to coexist with traditional SFX approaches.
Bespoke trained generative AI solutions for automatic mapping and creation of base textures for models allowing artists to focus on creative detail and overall narrative intent. New local compute solutions allow absolute privacy and confidentiality for critical projects.
Real-time Generative AI models allowing for fast iteration of non-destructive image and video manipulation processes, facilitate greater and granular artistic input through immediate response. These can also be trained on custom datasets and back catalogue materials, improving consistency and continuity between projects.
AI-driven effects simulation: AI tools that automate the creation of complex simulations, such as explosions, water and smoke, make effects more realistic and efficient to produce, taking into account real and simulated environments.
Real-time rendering: Allows for the instant rendering of VFX and SFX elements, enabling supervisors to see immediate results and make adjustments on the fly.
Virtual production: Combining real and digital elements on set, virtual production tools allow VFX and SFX supervisors to collaborate in creating seamless scenes.
3D scanning and printing for SFX: 3D technology that enables the scanning of physical objects to create digital models, or printing physical models for practical effects on set.
Augmented reality (AR) and virtual reality (VR): AR and VR are increasingly used in pre-visualisation, allowing supervisors to plan and test effects in immersive environments.
Autonomous robotics: Stunt robotics, autonomous robots and physical manipulation robots can allow on set SFX to take place where previously it was too dangerous, or simply not possible previously
3D volumetric capture technologies: Capturing live performances directly into a 3D pixel format and 3D mesh-based formats, allowing repositioning of cameras after the recording has been made.
ScreenSkills offers a variety of training opportunities for people at all stages of their career. Explore all training, events and opportunities.
ScreenSkills resources:
- Visual effects (VFX) supervisor job profile
- Special effects supervisor case study with Chris Corbould
Other resources:
- Platforms like Houdini and Maya provide tutorials on AI-assisted simulations
- Unreal Engine offers free online courses in real-time rendering for VFX, integrating augmented reality (AR) and virtual reality (VR) and virtual production
- Unity also has tutorials on AR and VR technologies
- Coursera has courses on 3D printing
- Bow Robotics, Udemy and Coursera have resources on autonomous robotics