> For the complete documentation index, see [llms.txt](https://metaverse-imagen.gitbook.io/ai-tools-research/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://metaverse-imagen.gitbook.io/ai-tools-research/ai-tools-main-categories/audio-speech-and-music/lip-sync-and-animated-avatars/aniportrait.md).

# AniPortrait

## AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation

{% embed url="<https://huggingface.co/spaces/ZJYang/AniPortrait_official>" %}
&#x20;
{% endembed %}

## AniPortait: Vid2Vid (Video Driven Synthesis of Portrait Animation

{% embed url="<https://replicate.com/camenduru/aniportrait-vid2vid>" %}
<https://replicate.com/camenduru/aniportrait-vid2vid>   &#x20;
{% endembed %}

AniPortrait is a novel framework for generating high-quality animation driven by audio and a reference portrait image.&#x20;

The methodology is divided into two stages. Initially, they extract 3D intermediate representations from audio and project them into a sequence of 2D facial landmarks.&#x20;

Subsequently, they employ a robust diffusion model, coupled with a motion module, to convert the landmark sequence into photorealistic and temporally consistent portrait animation.&#x20;

Experimental results demonstrate the superiority of AniPortrait in terms of facial naturalness, pose diversity, and visual quality, thereby offering an enhanced perceptual experience.&#x20;

Moreover, our methodology exhibits considerable potential in terms of flexibility and controllability, which can be effectively applied in areas such as facial motion editing or face reenactment.&#x20;

THE  Release code and model weights are at [this https URL](https://github.com/scutzzj/AniPortrait)

{% embed url="<https://github.com/scutzzj/AniPortrait>" %}

{% embed url="<https://huggingface.co/ZJYang/AniPortrait/tree/main>" %}
<https://huggingface.co/ZJYang/AniPortrait/tree/main>   &#x20;
{% endembed %}