The Riot Games panel, held on Saturday morning as part of the Year of Games Symposium, offered a rare behind-the-scenes look at how large creative teams create large-scale productions and media.
Among the speakers, Annie De Brock stood out to me the most for her reflections on the animation process she employed during her work within DC’s Creature Commandos, and how an underappreciated aspect of animation plays a huge role in its creation, namely music.
Before the panel, I thought that the process behind animation worked in a straightforward linear way: the animators sketch the scenes on a storyboard, then the animators animate what’s on the storyboard board, and all the extra details like music and special effects are done in post-production. Annie completely changed all of that the moment she mentioned that, most of the time, you build animation and sequence it all around the music of a scene. This was evident in the scene she presented in the panel, which was a scene in Episode 3 of Creature Commandos, where G.I. Robot goes on a massive killing spree. All the aspects of the scene, the music, the gunshots, the absurdity of a G.I. Joe Robot killing Nazis, everything coalesces into a fun and wonderful piece of animated media. And it isn’t just one of those aspects taking the forefront, dominating the scene, and making the scene what it is; it’s all of them combining seamlessly into a singular, exciting, and grandiose whole.
Now, why is that important? Well, as I mentioned earlier, I think a lot of people, including myself, are used to thinking of animation as visuals first, then everything else later. Which makes sense as animations are exactly that, animated pieces of images. What Anne described goes against that norm, as in reality, music isn’t just a piece of decoration that’s latched onto visuals; it, in fact, is basically the backbone that determines a lot of essential things like visual mood and timing. For instance, in the sequence of the Creature Commandos scene (https://www.youtube.com/watch?v=hJOAVQe3Vks, 1:12 to 1:25), the drum hits existing in that part of the song weren’t added there just to match the gunshots in the animation; the visuals were built around the drums being hit in that part of the song. In this case, the music structures the animation, not the other way around.
After realizing that, I started to see how this same idea applies to video games, albeit to a much more multifaceted and nuanced point. Since video games as a medium of media can be utilized and take form in many many many different ways, I guess I’m more interested in how music and sounds do more than to set the mood of a particular part of a video game.
The most obvious example of this is combat music in action games, specifically games under the 2-D side-scroller genre. Now, of course, most action combat music scores are meant to be highly energetic and hype to match the combat sequence. But, in terms of structuring play, when the soundtrack suddenly goes up intensely in tempo, the game is basically telling you, “HEY, YOU’RE FIGHTING NOW,” without having to blast some text indicating so. And as a response, you, as the player, are more focused on the threat(s) at hand, prioritizing them rather than what you were doing before the music started to change. Then, after that music starts to change back to how it was before the fighting started, you instinctively know that you’re currently safe to do whatever you were doing before, be it exploring, opening chests, breaking pots, or whatever. In this instance, music acts as a sort of UI that can’t be seen, but still does its job of informing the player on what’s going on inside the game.
Going into more of that idea of music acting as an invisible yet vital UI function, music plays an important role in dictating the timing of combat as well. Think of games like Hollow Knight, Blasphemous, or any Metroidvania for that matter. A lot of the enemies, from the basics of mobs to the bosses, have a musical cue that serves as a timing cue. Of course, there’s usually something visual that goes along with the cues, but it is the discordant strings before a strike, a melody indcating a time of respite for you to either attack or heal, or the rise in tempo before a long-winded Dark Souls-esque attack that will make you watch a 2-minute animation of you getting beaten to a pulp if you don’t dodge it, that teaches you, ‘hey you should probably do something right now if you don’t want to take damage.’
Additionally music can be used to indicate an entire alteration of a game state, for example, in Cup Head, music is often changed during a boss fight to indicate a change in boss phases, which directly assists you, the player, in managing yourself in-game so that you can dodge the attack that comes immediately after phase 1, or giving yourself a mental timer so that you know your progress in the fight, etc. Or the entire game can be built around the soundtrack(s), like Geometry Dash or Just Shapes and Beats; these are games that are basically concerts poorly hidden with platforming/bullet-hell game mechanics. Here, the entire game’s so to speak “laws of physics” are determined by a given level’s soundtrack (I recommend looking at some gameplay of Just Shapes and Beats, you wont regret it).
Though it wouldn’t be right for me to say that music is strictly the only thing that determines how a player plays a game, there’s literally no limit, depending on how the game is made. (Level design, enemy design, health indicators, movement, etc). But I am saying that music is often underappreciated and overlooked when it comes to analyzing these things in video games
I completely agree with your observation that music is an important way for games to tell the player things without outright showing text on screen. You brought up music that signals to the player a shift in game state such as when a boss enters a second phase. I think Hades is an amazing example of this; besides the fact that the soundtrack is absolutely amazing, the music that plays when you are fighting Hades and he enters a different phase does exactly what you said, it tells the player that they need to lock in asap. The player eventually gets used to doing this, until they try Extreme Measures 4, where Hades suddenly gets an entirely new phase, and the music changes to a track the player has never heard before and which kicks the beat and pressure up tenfold. These unexpected moments of panic would not be anywhere near as impactful if it wasn’t for the music.
I really agree with your point that music is often an underrecognized yet critically important element of UI in videogames. It’s interesting to think that part of the reason why music is overlooked in play experiences is because its role is to accentuate players’ immersion within a game — that is to say, if it’s unnoticeable, it’s doing its job right. Moreover, musical cues and boss music are meant to amplify the emotions we feel in the in-game environment, often during critical moments too, so it’s understandable that our preoccupation with the visual and mechanical elements of the game sidelines our ability to appreciate the music’s immersive qualities. Still, even though we can’t fully recognize or focus on music while playing games, I think that the large fanbases around game soundtracks is indicative of how integral music’s role is to a game’s lasting power and impact, as it illustrates how people not only associate game music with the atmosphere and environment of the games themselves, but also with their personal experiences playing through said games.