Jess Weatherbed is a journalist who focuses on the creative industries, computing and internet culture. Jess began her career as a news reporter at TechRadar. She also covered hardware reviews.
Adobe has launched new generative AI filmmaking tools that allow users to create fun sound effects and control the generated video outputs. Along with the familiar text prompts, which allow you to describe the actions that Adobe’s Firefly models should take, users can use onomatopoeia to create custom sounds and reference footage to guide movement in Firefly generated videos.
This tool, which is launching as a beta in the Firefly app ( ), can be used to create audio from recorded or generated footage. It offers greater control than Google’s Veo 3 tool. The interface is similar to a video timeline, and users can match the effects created in time with uploaded footage. Users can, for example, play a video showing a horse walking down a road, and simultaneously record “clip-clop” sounds in time with the hoof steps. They can also add a text description such as “hooves pounding concrete” to the video. The tool will then provide four sound effects options.
This builds upon the Project Super Sonic experience that Adobe demonstrated at its Max event in Oct. It does not work for speech but it can create impact sounds such as twigs snapping and footsteps as well as atmospheric sounds like nature sounds and city atmosphere. Firefly Text to Video generator will also be getting new advanced controls
. Composition Reference allows users upload a video along with their text prompt in order to mirror the composition in the generated video. This should make it easier for users to achieve specific results compared to repeatedly entering text descriptions. Keyframe cropping allows users to crop and upload images that Firefly will use to create video between. New style presets include anime, vector art and claymation.
The “claymation’ option looked like 3D animation from the early 2000s. Adobe continues to add support for rival AI tools within its own software. Alexandru Costin, Adobe’s Generative AI Lead, told The Verge similar controls and presets could be used with third-party AI in the future. Adobe may be trying to maintain its position at the top of creative software as AI tools become more popular, even if they lag behind OpenAI and Google when it comes to generative models.
