AURA: AUdio-dRiven streaming Avatar
BMVC 2026 Also at the ECCV 2026 workshops Gen4AVC (poster) and AVGenL (oral + poster)
Current audio-driven video diffusion models can synthesize realistic talking-head videos but generally require dozens of diffusion denoising steps over multi-second video chunks, making them unsuitable for streaming applications such as voice assistants and virtual avatars, where low latency and online causal generation are required properties. AURA generates high-resolution video chunk-by-chunk at 9.7 fps with a rolling KV cache, requiring only the audio of the current chunk rather than the multi-second audio look-ahead buffered by bidirectional models, making it well-suited to interactive avatar applications. Our two-stage distillation pipeline first aligns the causal student with the teacher, by training it to match the teacher's trajectories with fewer diffusion steps and a chunk-wise causal attention mask, then applies Distribution Matching Distillation (DMD) with backward simulation to close the remaining quality gap. On standard talking-head benchmarks, AURA matches the teacher's quality while running 17× faster, achieving 720p avatar streaming on a single GPU. To support reproducibility and further research, we release our model's weights along with a dataset of 30K synthetic videos and their precomputed teacher trajectories.
@inproceedings{cohen2026aura,
title = {{AURA}: {AU}dio-d{R}iven streaming {A}vatar},
author = {Cohen, Nathaniel and Dufour, Nicolas and Royer, Am{\'e}lie
and Newson, Alasdair and P{\'e}rez, Patrick},
booktitle = {British Machine Vision Conference (BMVC)},
year = {2026},
}