Adobe teases generative AI video tools

On Monday, Adobe announced plans to add AI video synthesis tools to Premiere Pro and After Effects, Reuters reported. This follows last week’s announcement that AI-powered video editing would be coming to Premiere Pro in May.

Adobe Generator Video Effects will be provided by Adobe Firefly. Firefly, the image synthesis mode we featured in March, will be expanded to include video and audio creation tools. Premiere Pro is Adobe’s flagship video editing application and the industry’s favorite tool for editing TV shows and Hollywood movies.

Nowadays, performing certain special effects or editing tasks can often be time consuming. Adobe aims to simplify this process with generative artificial intelligence. In a demo video posted on their blog, Adobe showed “Firefly for Video”generating a background soundtrack, generating sound effects, changing the visual season in a video, creating a visual effect for the word “melted”, locating and brightening a person’s face, and also creating audio recordings using text commands.

In its promotional blog, Adobe mentions the features we could expect from its new AI tools, including some of the ones listed above (such as creating sound effects and music), as well as text color improvements, text and logo animations, automatic creation storyboards and creativity. co-pilots”that can supposedly generate practical tips to help users learn editing skills.

If such text-based video editing tools sound familiar, it’s because Runway announced similar capabilities in September, although Runway has yet to deliver on those promises with the accuracy shown in the teaser. Similarly, it’s probably wise to take an Adobe marketing video with a grain of salt until the product is released.

(And only time will tell if Adobe Firefly can ever create a video of Will Smith eating spaghetti that can rival the masterpiece developed by ModelScope, an open source video synthesis model.)

In a separate but related announcement last week, Adobe announced the integration of video text editing into Adobe Premiere Pro, which will allow editors to trim and reorder videos based on auto-detected speech transcripts extracted from video clips. Here’s how Adobe describes the feature:

Premiere Pro is the only professional editing software that includes text-based editing, revolutionizing the way creators approach their craft, making video editing as easy as copying and pasting text. Text editing powered by Adobe Sensei automatically parses and transcribes clips so editors, producers and assistants can simply copy and paste sentences in any order they want and instantly see them appear on the timeline. Transcripts can be searched in the transcript window to identify exact words and phrases.

Adobe originally announced a similar video text editing capability as part of “Project Blink”in October, though Adobe now says these features come from its “Adobe Sensei”framework. It is not clear if the difference is technical or just a trademark. It’s worth noting that Descript offers a similar feature for editing videos and podcasts.

Adobe says its Firefly features for video will begin rolling out “later this year”as well as allowing people to sign up to try out Firefly, which is currently in closed beta. In the meantime, Adobe is promising that text-based video editing will be coming to Adobe Premiere Pro in May.