.png)
Volumetric Video Explained: How Volumetric Capture Works for Film, XR and Digital Humans
Volumetric video records a real performance as moving 3D geometry and texture, so the viewer can walk around it instead of watching it from a single camera angle.
A normal video is a flat window. A volumetric clip is closer to a filmed sculpture: every frame contains a full 3D shape of the performer, coloured by dozens of synchronised cameras. Drop it into a headset, a game engine or a holographic display and the person is simply there, from every side.
That makes volumetric video powerful, but it is often misunderstood. It is not the same as motion capture, and it is not a digital double. Each technique records something different, and each produces a different kind of asset with different strengths downstream.
At Mimic Productions, volumetric capture is treated as one option inside a wider digital human pipeline that also includes 3D scanning, motion capture, character creation and real-time integration. The right choice depends on what has to happen to the performance after it is recorded.
Table of Contents
What Is Volumetric Video?
Volumetric video is a sequence of 3D captures of a real subject over time.
Each frame stores the shape of the performer and the colour of their surface. Played back in order, those frames become a moving 3D performance that can be viewed from any angle, at any distance, with six degrees of freedom.
The key difference from traditional video is where the camera decision happens:
In normal video, the camera angle is chosen on set
In volumetric video, the camera angle is chosen later, by the director or the viewer
In a headset, the viewer's own head becomes the camera
That shift is why volumetric content has become important for XR, immersive storytelling and holographic displays. The performance is captured once and can be experienced from positions that did not exist on the day of the shoot.
How Volumetric Capture Works
A volumetric capture studio surrounds the performer with a ring or dome of synchronised cameras. Depending on the system, this can mean a few dozen cameras or more than a hundred, often combined with infrared or depth sensors.
The process usually follows five stages:
Capture: every camera records the performance at the same instant
Reconstruction: software calculates the 3D shape of the subject for each frame
Meshing and texturing: the shape becomes a mesh and is coloured from the camera views
Cleanup: artefacts, holes and noisy areas are repaired
Compression: the sequence is reduced to a size that can stream or run in an engine
Reconstruction is closely related to photogrammetry and full body scanning. The difference is time. A 3D body scanning session captures one perfect pose in high detail. Volumetric capture repeats a lighter version of that process many times per second.
Newer pipelines also use neural methods. Techniques such as Gaussian splatting can represent volumetric performances with fewer artefacts on hair and fine detail, a shift we compare in 3D Gaussian Splatting vs Photogrammetry vs NeRF.
What a Volumetric Video File Contains
A volumetric clip is not a single video file in the usual sense. It is a stack of data that has to stay in sync.
Common components include:
A mesh or point cloud for every frame
A texture or colour stream that wraps the geometry
Audio recorded alongside the performance
Optional depth, normal or alpha data for compositing
Raw captures are very large, so delivery formats focus on compression. Studios typically export to engine-friendly sequences for Unreal Engine or Unity, to streaming formats for web and mobile AR, or to standardised codecs designed for volumetric media.
The practical point for producers is simple: plan storage, processing time and playback hardware from the start. A beautiful capture that cannot stream on the target device is not a finished asset.
Volumetric Video vs Motion Capture
This is the comparison most teams need, because the two are easy to confuse.
Volumetric video records what the performer looked like: their real shape, clothing, hair and colour, frame by frame. What you capture is what you get.
Motion capture records how the performer moved: joint rotations, facial expressions and timing. That data then drives any character you choose, from a photoreal digital human to a creature or cartoon.
Question | Volumetric video | Motion capture |
What is recorded | Appearance and shape | Movement data |
Can the look change later | Very limited | Yes, any character |
Can the performance be edited | Limited | Fully, keyframe by keyframe |
Relighting | Difficult, lighting is baked in | Full, character is lit in engine |
If you need the real person exactly as they are, volumetric capture is direct. If you need to change the character, relight the scene or edit the performance, motion capture is usually the better foundation. Our guide to performance capture covers how body, face and voice are recorded together for that route.
Volumetric Video vs Digital Doubles
A digital double is a fully built 3D replica of a real person, scanned, modelled, textured and rigged so it can be animated like any character.
Volumetric video and digital doubles both start from a real person, but they behave very differently:
A volumetric clip is a recording. It plays back the original performance
A digital double is a puppet. It can perform anything, including actions the actor never did
A volumetric clip has the actor's real lighting baked in
A digital double can be lit to match any shot or environment
For film and long-running brand characters, the flexibility of a rigged double usually wins. For short, authentic performances that must look exactly like the person on the day, volumetric capture can be faster. Many productions now combine the two, using scanning and 3D character services for the hero asset and volumetric capture for specific moments.
Strengths and Limitations
Volumetric capture is excellent at one thing: preserving a real human performance with very little interpretation in between.
Strengths:
Authentic appearance, including clothing, hair and real expressions
No rigging or animation stage for the captured take
Viewable from any angle in XR and on 3D displays
Fast path from performance to a viewable 3D result
Limitations:
Lighting is baked into the texture and is hard to change
Performances are difficult to edit or re-time
Data sizes are large, even after compression
Fine hair, transparent and reflective materials can break reconstruction
The capture volume limits how far the performer can move
None of these are reasons to avoid volumetric video. They are reasons to decide early what the asset must do after the shoot.
Comparison Table
Method | What it captures | Best for | Main limitation |
Volumetric video | Moving 3D shape and colour | XR experiences, holograms, authentic performances | Baked lighting, limited editing |
Motion capture | Body and facial movement data | Games, film, animated characters | Needs a separate character asset |
3D body scanning | One high detail static pose | Digital doubles, fashion, character bases | No motion on its own |
Rigged digital double | Scan plus rig plus animation | Film, long-running brand characters | Longer build time |
Gaussian splatting | Neural 3D representation | Fast realistic scenes, emerging 4D capture | Tooling and engine support still maturing |
Where Volumetric Video Is Used
Volumetric capture is most valuable wherever audiences can move around the subject.
XR and VR: performers who share space with the viewer in a headset
Augmented reality: life-size people placed into real rooms through a phone
Holographic displays: 3D presenters for events, retail and museums
Music and entertainment: artists captured once and shown in immersive formats
Sports and fitness: athlete movement viewed from any angle for fans and coaching
Education and heritage: recorded testimonies and demonstrations preserved in 3D
For immersive projects, the capture usually feeds into wider XR production. For stage and retail installations, the same data can drive hologram experiences where a 3D presence matters more than a flat screen.
Planning a Volumetric Shoot
Volumetric shoots reward preparation. Most problems can be avoided before the performer steps on stage.
Step 1: Define the delivery platform first
A headset, a phone AR app and a holographic display each have different limits on file size, resolution and frame rate. The platform decides the capture settings.
Step 2: Plan wardrobe and props
Avoid very fine patterns, shiny or transparent materials and loose fabrics that hide body shape. Simple, matte clothing reconstructs most cleanly.
Step 3: Block the performance inside the volume
The performer has to stay inside the calibrated capture area. Rehearse movement, turns and exits so nothing is cut off.
Step 4: Keep takes purposeful
Every second of volumetric data costs processing and storage. Short, well rehearsed takes are more efficient than long exploratory ones.
Step 5: Decide what happens after capture
If the asset will need relighting, re-animation or a different look later, plan a hybrid route with scanning and motion capture from the start rather than trying to force it out of a volumetric clip.
The Future of Volumetric Video
The biggest changes in volumetric video are coming from neural reconstruction and better compression.
Expect to see:
Neural and Gaussian splat based capture with cleaner hair and edges
Relightable volumetric assets that respond to new environments
Smaller files that stream reliably to phones and headsets
Hybrid pipelines where volumetric data drives rigged characters
Lighter capture setups that reduce the need for very large camera arrays
As these improve, the line between recording a performance and building a character will keep getting thinner. The most useful skill for producers will be knowing which parts of a project need authentic capture and which need full creative control.
Frequently Asked Questions
What is volumetric video?
Volumetric video is a 3D recording of a real person or object over time. Each frame stores shape and colour, so the result can be viewed from any angle in XR, game engines or 3D displays.
How is volumetric video different from 360 video?
360 video records a flat image of everything around a fixed camera. Volumetric video records the subject itself in 3D, so the viewer can move around it instead of only looking around from one point.
How many cameras are needed for volumetric capture?
It depends on the system and quality target. Professional stages typically use dozens of synchronised cameras, and some use more than a hundred, often combined with depth or infrared sensors.
Is volumetric video the same as motion capture?
No. Volumetric video records appearance and shape, while motion capture records movement data that drives a separate 3D character. They are often used together in hybrid pipelines.
Can volumetric video be used in Unreal Engine or Unity?
Yes. Volumetric captures are commonly exported as mesh sequences or plugin-supported formats for Unreal Engine and Unity, where they can be placed in real-time scenes.
Can you change the lighting on volumetric video?
Only to a limited degree, because lighting from the capture stage is baked into the texture. Relightable volumetric capture is an active area of development, but rigged digital characters remain more flexible for lighting changes.
What are the main limitations of volumetric video?
The main limits are large file sizes, baked lighting, limited editing after capture and difficulty with fine hair, transparent and reflective materials. Planning delivery and wardrobe early reduces most of these issues.
When should a production choose a digital double instead?
Choose a digital double when the character must perform new actions, appear in many lighting setups or be reused over a long period. Choose volumetric capture when a specific, authentic performance needs to be preserved exactly as it happened.
Conclusion
Volumetric video captures something no other technique does as directly: a real human performance, in three dimensions, exactly as it happened.
Its value depends on what the project needs next. If the performance must stay authentic and be viewed from every angle, volumetric capture is a strong route. If the character must be relit, edited, re-animated or reused for years, scanning, motion capture and rigged digital doubles provide the control.
The strongest productions treat these as complementary tools, chosen shot by shot, inside one coordinated digital human pipeline.
For inquiries, please contact: Press Department, Mimic Productions info@mimicproductions.com
Comments