Creating Realistic AI-Models of Humans for Animation
Creating Realistic AI-Models of Humans for Animation
Tools Used:
kohya-LoRA,
Character Creator (CC) - AI-powered Character Creator to create a 'Puppet'.
iClone real-time 3D animation software
Headshot plugin for AI-powered Character Creator (CC).
Motion Live plugin for iClone.
BIRME (Bulk Image Resizing Made Easy) ver. 2.0 Software
DaVinci Resolve (Video Editing software)
You can create AI Actors using 3D Puppets and LoRA technology. LoRA stands for Low-Rank Adaptation, a mathematical technique to reduce the number of parameters that are trained.
LoRA was developed by researchers at Microsoft, and Simo Ryu has applied it to Stable Diffusion, which is a Deep Learning, Text-to-Image model released in 2022. I'll take you through a step-by -step process in the ‘Work Flow’.
PROCESS: There are 3 Steps involved;
Step 1. Prepare images - First, gather and upload images of a person (approx. 30 images or so).
Step 2. Train these images with 'kohya-LoRa' and Render them using Automatic1111.
(In this tutorial, we will train a realistic dataset of Images for the 'Puppet'. We will use a real person - so you can judge the likeness at the end for yourself.
Step 3. Create a Puppet using Character Creator (CC) Software and Plugins. You do not need to know any 3D. There is no learning curve to this software. You just basically use sliders and dragging and dropping Assets onto your 3D character. Character Creator (CC) is a full character creation software for designers to easily generate, import and customize stylized or realistic character Assets for use in Animation Movies, Games, and even as Avatars in a Metaverse.
a). There are other options for doing this as well. Obviously you can use ‘Blender’ and ‘MetaHuman’, but the learning curve is steep. (MetaHuman (https://www.unrealengine.com is from Unreal Engine). It is a High-fidelity Digital Humans creating software. MetaHuman is a complete framework that gives any creator the power to use highly realistic human characters in any way, but it has a steep learning curve as I mentioned).
b). The ‘Workflow’ for this tutorial is through Character Creator (‘CC’). This means we can then switch out to any character using the Dataset. Once we have the rendered files of out ‘Puppet’, we can then switch out those Datasets from one character to another and it's really an easy and efficient way of working.
Basically, we will show you how to create a fully Animated AI Character. The advantage of this technique is that you don't have to act out the scenes before. In fact you don't need to look like your Actor or Actress. This method could potentially be used in the movie industry and the concept can be used for an entire movie itself.
Last updated