Character AI, the California-based synthetic intelligence (AI) platform, unveiled its first video technology mannequin on Monday. Dubbed AvatarFX, it’s an image-to-video mannequin that may generate 2D and 3D animated characters. The AI agency claims the generated movies will maintain temporal consistency with face, hand and physique motion. Moreover, the movies will function speech, which is powered by the corporate’s native text-to-speech (TTS) fashions. Character AI mentioned AvatarFX will probably be launched within the coming months, and the paid subscribers will get entry to it first.
Character AI’s AvatarFX Will not Generate Practical Human Characters
In a weblog submit, the corporate detailed the brand new video technology mannequin. To this point, Character AI has centered on textual content and image-based fashions. With AvatarFX, the AI agency is taking a stab at AI-generated movies. To this point, the corporate has said that Character AI+ subscribers will get entry to the mannequin, nevertheless, it isn’t recognized whether or not it’ll later be expanded to the free tier.
AvatarFX can generate movies of 2D animated characters, 3D cartoon characters, and non-human faces. The corporate claims that one of many main highlights of the mannequin is temporal consistency with face, hand, and physique actions. This implies the topic will stay constant between frames, whereas glitches like further arms and convoluted facial expressions will probably be much less frequent. It is essential to keep in mind that these are claims, and the device’s capabilities can’t be verified till it’s launched.
In contrast to most video technology fashions, AvatarFX doesn’t help textual content inputs for video technology. As an alternative, the mannequin solely accepts photographs as enter. Character AI claims that this permits customers to realize higher management over the generated output. The movies may also function speech, which will probably be generated utilizing native TTS fashions.
Coming to the structure, the AI agency mentioned that AvatarFX is constructed on flow-based diffusion fashions. The bottom mannequin was constructed from the bottom up utilizing the Doc Picture Transformer (DiT) structure, which is a transformer encoder mannequin. Character AI claims that it additionally used a brand new inference technique, which preserves visible high quality, movement consistency, and expressive variety even in longer length movies.
AI-powered video technology fashions at all times elevate issues about deepfakes and doubtlessly dangerous content material. Character AI said that it has taken a number of measures to minimise such dangers. The AvatarFX has in-built security filters that test the dialogues customers write for the movies.
The AI mannequin additionally doesn’t approve video technology utilizing photographs of minors, high-profile politicians, and notable figures. Different human images are made unrecognisable within the technology course of, so the particular person can’t be recognized from the video, the corporate claimed. Moreover, every video is watermarked to let viewers know it’s AI-generated. Nonetheless, the corporate didn’t specify if the watermark is added simply to the front-end of the video or in its metadata.
Character AI mentioned it has additionally added a set of latest phrases for the function, which prohibit AvatarFX’s utilization for impersonation, bullying, deepfakes, and using protected IP with out permission. Violations would lead to a “strict” one-strike ban.