VFX Vlog 12 - Converting Audio to Keyframes in After Effects

Extracting Keyframes from Audio

Import an audio clip into the composition. You can use any other audio clip as well, but for this tutorial it would make more sense to have a simple voice audio track of someone talking. When you expand the layer, Adobe After Effects will display the waveform for the audio track.

To convert the volume of this audio layer to keyframes, you can use the Keyframe Assistant. Simply select the audio layer and then go to the main menu and select Animation -> Keyframe Assistant -> Convert Audio to Keyframes.

A new Null Object called 'Audio Amplitude' will be created in your composition. If you expand the layer you will see 3 sider expression control effects applied to it. Check out my After Effects Expression Controls tutorial if you want to learn what they are for and how to use them.

The 3 slider effects have been keyframed to represent the volume of the left, the right and both channels of the audio layer.

As you know, by themselves, these sliders have no effect on anything in your composition, but we can hook up effect parameters and layer properties to them using expressions. If you're unfamiliar with expressions, check out my separate Adobe After Effects Expressions tutorial.

Let's hook the visibility of the open mouth layer up to the Both Channels slider so it only becomes visible whenever the volume of the audio goes above a certain level.

Utilising the Generated Audio Keyframes

We want the open mouth to only be visible whenever the audio level is above a certain level. This will make it appear as if the mouth opens whenever the puppet talk. While this won't be perfectly natural, it'll be enough to convey the feeling of the puppet talking.

To determine the volume threshold to control the visibility of the open mouth, scrub through your footage and have a look at the Both Channels slider value during the lows and highs. Select a good average value that indicates that the person is talking. For me, that is around 4.

Since we want to link the value of the Both Channels slider effect to another property in our composition, make sure the Audio Amplitude layer is expanded so you can see the effect and its keyframes. Then reveal the opacity property of the open mouth layer and alt-click on the stopwatch icon to add an expression to it. Then take the pick whip icon on the Opacity property and drag it onto the Both Channels Slider property.

Let go to insert the expression into your text editor for the Opacity of the Mouth Open layer. Your expression should now be

thisComp.layer("Audio Amplitude").effect("Both Channels")("Slider")

This expression will set the opacity of the mouth open layer to the value of the keyframes for the Both Channels slider effect. That is not really what we want. We want the opacity to be fully opaque (100) when the slider value is above our designated volume threshold (4) or 0 otherwise. For this, change your expression to

thisComp.layer("Audio Amplitude").effect("Both Channels")("Slider") > 4 ? 100 : 0

This is a simple terniary expression that says 'If the value of the Both Channels slider is greater than 4, set the Opacity of the Mouth Open layer to 100, otherwise set it to 0'.

If you now play back your footage, the mouth of the puppet will open every time the audio level goes above 4, giving the expression of the puppet talking. And the greatest thing is that we achieved this without having to manually animate the open mouth to sync it up with the audio.

Other Things to do with Audio To Keyframes Conversion

Remember that you can hook the generated keyframes for your audio layer up to any effect parameter or layer property that you want. This allows you to create great audio visualisationsand can save you large amounts of manual keyframing work.

As another quick example, let's apply a bulge effect to my head inflates and deflates in sync with the audio. Apply a Buldge effect to the puppet layer and place the Bulge Center right on the face of the puppet.

You can control the reach of the effect by altering the Horizontal Radius and Vertical Radius properties and you can change the strength of bulge by increasing or decreasing the Bulge Height property.

We want to link the Bulge Height property to the Both Channels slider value and you can do that exactly like we did before by using expressions and the pick whip icon. You may have to multiply or divide the expression to scale the bulge height to suit your audio keyframes. Once you have linked the audio keyframes up to the Bulge Height property, your puppet's head will inflate in sync with the audio!

There's a million other uses for extracting keyframes from audio in Adobe After Effects and I encourage you to play around with this technique and have some fun!


Please leave a comment

Click here to enter some text...


Get in touch via social media


Tobias Avatar
I am passionate about providing high quality VFX & Film Making Edutainment online. Check out my training courses on Udemy
© 2020 Surfaced Studio