AniPortrait

AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation

https://huggingface.co/spaces/ZJYang/AniPortrait_officialhuggingface.co

AniPortait: Vid2Vid (Video Driven Synthesis of Portrait Animation

https://replicate.com/camenduru/aniportrait-vid2vidreplicate.com

https://replicate.com/camenduru/aniportrait-vid2vid

AniPortrait is a novel framework for generating high-quality animation driven by audio and a reference portrait image.

The methodology is divided into two stages. Initially, they extract 3D intermediate representations from audio and project them into a sequence of 2D facial landmarks.

Subsequently, they employ a robust diffusion model, coupled with a motion module, to convert the landmark sequence into photorealistic and temporally consistent portrait animation.

Experimental results demonstrate the superiority of AniPortrait in terms of facial naturalness, pose diversity, and visual quality, thereby offering an enhanced perceptual experience.

Moreover, our methodology exhibits considerable potential in terms of flexibility and controllability, which can be effectively applied in areas such as facial motion editing or face reenactment.

THE Release code and model weights are at this https URL

GitHub - Zejun-Yang/AniPortrait: AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait AnimationGitHub

ZJYang/AniPortrait at mainhuggingface

https://huggingface.co/ZJYang/AniPortrait/tree/main