Feeling the vibes: The surprising tool making music more accessible

Wellington music producer Jesse Austin-Stewart doesn’t want to be known as ‘the PlayStation controller guy’. Instead, he’s excited about this tool’s ability to make music more accessible to the Deaf and hard-of-hearing communities. 

A disabled person himself, Austin-Stewart's tunes are, in his own words, designed to be “felt, not heard” by holding a PlayStation controller and feeling vibrations move across the controller to each hand.

He came up with the idea back in 2022 while completing his PhD in music at Massey University. While most of his PhD was focused on studying spatial audio, or surround sound, he was interested in looking at things from a “sound art” perspective.

“I was curious how we could make spatial music within sound art more accessible through education and also in terms of disability and hearing,” says Austin-Stewart.

One of Austin-Stewart’s first inventions was a wooden plank on which you rest your arm and feel vibration at different points on your arm. However, this device “costs lots of money to make” and, as he was the only one making them, it would be a lot of work to put into production.

This issue was resolved after finding an article on The Verge about using the PlayStation controller as an audio interface with Spotify, causing the controller to vibrate.

Austin-Stewart credits this PlayStation controller breakthrough as one of his "proudest moments”.

“Everyone can buy a PlayStation controller much more easily than they can buy this custom thing that I made.”

Since coming up with this concept, he’s composed an entire EP, Music For PlayStation, which consists of five tracks written to be felt through the PlayStation controller.

One challenge that he faced was trying to make his tracks feel musical, despite the lack of instrumentation.

“When you're listening to a song, you have the benefit of different instruments making different palettes of sounds.”

Austin-Stewart had to figure out how to apply that to the PlayStation controller and found that rhythmic variation was the solution to spicing up his music. 

“The rhythm was a very important part of keeping things interesting, the length of the notes being really important in terms of how it feels to hold the controller.”

One of the people who inspired his creation was American sound artist Christine Sun Kim, who has written music for subwoofers.

“The subs are turned up really loud, so it's too low for us to hear, but it is moving things in the space,” he explains. “You can see the windows move, people's glasses and things like that.”

Austin-Stewart’s idea is the first to utilise a PlayStation controller. “It has been done before, but not with a PlayStation controller or, more importantly, relatively cheap technologies,” Austin-Stewart says.

Moving forward, he is keen to get more exposure on his Music For PlayStation EP. He also wants to encourage composers to write their own pieces for the PlayStation controller as well. 

“This project has been really fun and exciting for me and I'm keen to keep engaging with it, but I'm also keen to have other people explore it in ways that they find interesting and that are meaningful to them.”

Austin-Stewart hopes that it will allow people to “explore music in ways they haven't been able to before”, and would also like to see more disabled people getting involved with the creation process.

“Agency is critical,” he says. “So continuing to have advocates involved in discussions and decision-making processes is integral.”

Subscribe to our weekly newsletter, The D*List Delivered!