Turn Videos Into Animations With Kinetix AI For Unity!

In today's post, I would like to introduce you to Kinetix, which provides powerful AI tools for cloud infrastructure to create animations from videos, perform automated complex rig retargeting, and host emotes, or best referred to as 3D humanoid characters, for games and apps across all major platforms such as mobile, WebGL, VR/AR, and standalone.

But most importantly, all of these features are available through their new Unity SDK, which was released a few days ago.

First, let's discuss how we can generate animations and how AI can help us with this.

One way is to use Kinetix's verified animations, which you can easily integrate into Unity. They have a large catalog of animations that can be retrieved through their Unity SDK by using an Emote ID or specifying certain criteria.

Another way is to create our own animations through their Unity SDK by uploading a video to the Kinetix cloud platform. The steps are fairly simple:

1. Capture one or more recordings. In my case, I am using an iPhone 13 Pro, but honestly, any mobile device with a decent camera would do the job.

2. Use the Kinetix Unity SDK's "Create" tab, which is provided with their UI Emote Wheel Samples.

3. You will be prompted with a QR Code or link that will redirect you to upload your recorded videos.

4. Wait for 5-10 minutes for Kinetix to process the video. Once it's completed, Kinetix will email you to let you know it's ready.

5. A GUID will be generated with each video, which you can use to pull the animation into Unity.

Images below reflect some of the features and prototype I created to demonstrate the power of Kinetix AI.

View fullsize

AI Animation (Boxing)

View fullsize

AI Animation (Rig)

View fullsize

Kinetix Animations

View fullsize

AI Retargeting

View fullsize

(AI) Video To Animation

View fullsize

Recording Videos For (AI)

View fullsize

Retargeting Rigs

Now, let's take a look at how to set up a simple Unity project to demonstrate what I just explained. Also, consider watching my detailed YouTube video if you are more of a visual person.

How to set it up Kinetix AI in Unity?

• Install Unity 2020.3 LTS or greater

• Be sure to have Git as the next steps will require having it installed.

• In Unity > Package Manager > Click on + then "“add package from git url” and add:

  • com.unity.nuget.newtonsoft-json

  • com.unity.inputsystem

  • https://github.com/git-kinetix/kinetix-unity-sdk-packagemanager.git

• Verify that the “New Input System” or “Both” is set under player settings (Kinetix Unity SDK uses the new input system)

• Click on “Kinetix” under the Unity Toolbar and Install Core Bundle (Web2 and Web3 versions are available)

• You will need a Virtual World Key to setup the examples. Kinetix will email you that information after registration. Be sure to keep it around for the next step.

• Import Kinetix UI Emote Wheel Samples by going to the Package Manager > Kinetix UI Emote Wheel > Samples > Import

  • If you’re using URP be sure to upgrade all the materials under the samples common folder.

• Then simply double click on any of the scene located under Assets > Samples > Kinetix > Version

  • Each scene will hold a GameObject in the hierarchy called “SampleScript”, click on it to add your key

• Hit Play and have FUN!

If you would like to test the full version of my project as shown on YouTube, feel free to clone it or download it from GitHub here.

Compatibility

• You can use Kinetix with any desktop or laptop. In fact, I used it with my gaming PC while working on this video, as well as my macOS whenever I had to work from a different location other than my office. It worked great.

• It is recommended to use Chrome for uploading videos or using their portal features.

• Kinetix supports exporting generated Emotes/3D Characters. The supported file types are glb and fbx.

Where to go / resources?

• For animation results from video recordings, be sure to check out: Kinetix recommendations

• For pricing details, visit: https://docs.kinetix.tech/support/faq/pricing

• To register, check out: https://kinetix.tech.

As a wrap-up, I am extremely happy to have had the opportunity to work with their tool. This video was sponsored, but I honestly can't stress enough how important a tool like this is going to be for me in the future. Personally, I have very little time to create rigs, animate, and do all the hard work required to make something look awesome and animated. Well, now there's no need for me to do all of that because I can rely on them to provide me with the necessary tools and support to enhance my productivity, which is what I value the most.

Thanks everyone and have an amazing day!

Dilmer