Game on: Researching and composing for video games

An ANU School of Music PhD candidate who is researching audio and gaming is also the composer and sound designer for a promising new video game.

For his PhD research, Thomas Connell is investigating how sound design and audio communication affect players' experiences of a game. 

“So one of the things I'm looking at is, how can you use sound design to change the player's perception or change the player's experience of a video game?” Tom asks. 

“Sounds can tell us information that we don't get from anywhere else, and that stuff is really interesting because it can affect players' performance more than music can.”

His influences are broad: from theories about acousmatics (sounds heard where the source is obscured), to observations about fantastical elements of games that are accompanied by sounds that do conform to reality, to his own experiences as a gamer.

“It builds off a bit from what I did in my Honours year, where I made a classification system of sounds that you can find in a video game,” he says.

In terms of audio communication and gameplay, that refers to the way people use programs such as Discord to talk to other people while they’re playing a game, or a game’s own in-built communication tools.

There's a game called Sea of Thieves, Tom says, which has an in-built communication system. 

“It follows the physics of the game. So if you want to talk to someone that is far away, you have to use a megaphone; it’s like a mouth trumpet or something.”

He also cites the galaxy explorer game Elite Dangerous. You can communicate across any distance, but voices are filtered to sound like they’re coming through your spaceship's radio.

“Whereas if you're using, say, just Discord, because it's an external program, it doesn't obey any of those physical rules. And I think that's something that people can miss out on a little bit,” Tom says.

Tom will soon run a focus group to explore how players’ experiences and performance are affected by communicating through each of those means. One survey finding he’s made so far though is that people’s immersion in a game is negatively impacted by audio communication if they’re playing a single player game. If they’re playing a multiplayer game, however, and are communicating with other people playing the same thing, it’s the complete opposite.

“The use of audio communication breaks down the barriers which affect the ontology of a game,” Tom explains. 

“If a player must pause whatever they’re doing and open up a chat box and type a message and then have to wait for a response, they may become bored with the wait time or may lose out on critical time they need to perform an action. In addition, the physical act of having to pause and type is another barrier when a player must remove their hands from the control keys and utilise the entire keyboard, moving from one “work world” to another.”

Tom’ dream is to one day work for Skywalker Sound, which does sound design and music for Star Wars properties as well as Disney films. Sound design for films and video games, he explains, is very similar. The latter involves the extra step of implementing the sounds and music. For now, he’s working on Ailuri – a single player, 2D hand-drawn game about a red panda trying to save the world from environmental destruction.

That opportunity came about when Tom’s Honours supervisor (and Head of the ANU School of Music) Professor Kim Cunio reached out to the Academy of Interactive Entertainment (AIE) in Sydney in 2018. AIE students had made games they needed sound design and music for, and the School of Music had students with the right skills for the job.

“I saw Ailuri and went, ‘That's the one. That's the one I want to work on’,” Tom recalls. “I love red pandas. I mean who doesn't? And I felt like the music I make would work well for the game.”

The music Tom composed for Ailuri is inspired by musicians such as Saint-Saëns and Tchaikovsky, reflecting his classical background and training. He has not only created the music and sounds for the game though; he has also implemented that audio, applying skills he’s learned through his two School of Music degrees.

“I didn't just give them a bunch of audio files and say, ‘Here you go. Just put them in do what you want’. I actually went – ‘Hang on. I'm going to help with getting it working properly’.”

Last year, a Kickstarter was launched for Ailuri. The team blew through their target of $18,000, raising more than $73,000. They’re now on track to release the game in September.

For now, Tom is seeking more gamers to participate in his survey to how they understand, listen, and relate to sound and music in video games. You can find that here. 

Learn more about studying Composition for Film and Video Games at the ANU School of Music.


Written by Evana Ho