OpenAIs Sora first look: YouTuber Marques Brownlee breaks down the problems with the AI video model

Some of the highly-anticipated AI-related merchandise has simply arrived: OpenAI’s AI video generator Sora launched on Monday as a part of the corporate’s 12 Days of OpenAI occasion.

OpenAI has offered sneak peeks at Sora’s output up to now. However, how completely different is it at launch? OpenAI has actually been arduous at work to replace and enhance its AI video generator in preparation for its public launch.

YouTuber Marques Brownlee had a first take a look at Sora, releasing his video overview of the most recent OpenAI product hours earlier than OpenAI even formally introduced the launch. What did Brownlee assume?

What Sora is sweet at

In response to Brownlee, his Sora testing discovered that the AI video generator excels at creating landscapes. AI generated overhead, drone-like photographs of nature or well-known landscapes look similar to real-life inventory footage. In fact, as Brownlee factors out, in case you are particularly well-versed in how the environment of a landmark look, one may be capable to spot the variations. Nevertheless, there’s not an excessive amount of that appears distinctly AI-generated in some of these Sora-created clips.

Maybe the kind of video Sora is finest capable of create, in accordance with Brownlee, are summary movies. Background or screensaver sort summary artwork might be made fairly effectively by Sora even with particular directions.

Brownlee additionally discovered that Sora-generated sure sorts of animated content material, like stop-motion or claymation sort animation, look satisfactory at occasions because the typically jerky actions that also plague AI video appear like stylistic decisions.

Most surprisingly, Brownlee discovered that Sora was capable of deal with very particular animated textual content visuals. Phrases typically present up as garbled textual content in different AI picture and video era fashions. With Sora, Brownlee discovered that so long as the textual content was particular, say a number of phrases on title card, Sora was capable of generate the visible with appropriate spelling.

The place Sora goes flawed

Sora, nonetheless, nonetheless presents lots of the similar issues that every one AI video mills that got here earlier than it have struggled with.

The very first thing Brownlee mentions is object permanence. Sora has points with displaying, say, a selected object in a person’s hand all through the runtime of the video. Generally the item will transfer or simply immediately disappear. Identical to with AI textual content, Sora’s AI video suffers from hallucinations. 

Which brings Brownlee to Sora’s greatest drawback: Physics usually. Photorealistic video appears to be fairly difficult for Sora as a result of it may well’t simply appear to get motion down proper. An individual merely strolling will begin slowing down or dashing up in unnatural methods. Physique components or objects will immediately warp into one thing fully completely different at occasions as effectively.

And, whereas Brownlee did point out these enhancements with textual content, except you’re getting very particular, Sora nonetheless garbles the spelling of any type of background textual content such as you may see on buildings or road indicators.

Sora may be very a lot an ongoing work, as OpenAI shared through the launch. Whereas it could provide a step up from different AI video mills, it is clear that there are just a few areas the place all AI video fashions are going to seek out difficult.