Playing With Sound

Playing With Sound

Music and games have a long and intertwined history, from simple bleeps and bloops through to detailed synaesthetic soundscapes. Patrick Stafford wanders through 20 years of music game development to create the perfect mix.

Patrick Stafford >> gets down with rhythm games

Music and games have always shared a close relationship. From the basic blips of Pong to more modern, sweeping orchestral compositions, interactive entertainment has always been accompanied by music and sound.

It’s really no surprise that games started experimenting with how to manipulate sound, and eventually made that activity part of the games themselves. Dance Dance Revolution is the iconic example of a music-based game, but the past 20 years have been filled with titles dedicated to sound and music composition.

But surely, the peak has past. Although music games based on plastic peripherals like Guitar Hero have been exceedingly popular, the sales seen at the 2007-08 height of the genre’s popularity have not been replicated.

Has the music revolution finally stopped?

“That’s a hard question to answer,” says Harmonix chief creative officer Greg LoPiccolo. Harmonix is arguably responsible for creating the renewed interest in music games, spawning among several other titles, Guitar Hero.

“All these games blew up much bigger than we thought they would, and they also shrank much quicker than we thought they would as well.”

“We have no idea why they got big so quickly. There are a lot of theories, but we just don’t know. We’re very often surprised by what happens.”

Of course, music games haven’t died. They’re enjoying a sound second wave of success through dance titles, made possible due to the advancements in motion technology. Yet the evolution begs some key questions – if music games haven’t died, where are they going?

The music of gameplay » The videogame industry has been a lucrative one for composers and sound designers. Humming tunes from favorite levels and recalling iconic sounds is a fun activity among gamers. Even today, full orchestras – including some recently in Australia – have dedicated entire concerts to full arrangements of game themes.

In a testament to the key relationship between games and sounds, many of these orchestras begin their sessions by playing the “blips” from Pong.

So it was only a matter of time before music games first hit the shelves. It’s a rich area to explore, and natural – everyone loves music, and creating your own can be a game in itself. And while DDR was certainly the first game to popularise the genre, it’s far from the first to exist.

In the 1980s, some basic games started toying with the idea of rhythm – Spinnaker’s Aerobics on the Atari first pioneered the concept, albeit in a very basic form. It wasn’t until 1996, when NanaOn-Sha delivered what many believe to be the first comprehensive rhythm game in PaRappa the Rapper, that the concept started taking hold.

Soon afterwards, others like Konami’s Beatmania followed closely, experimenting with DJ sounds well before DJ Hero in the late 2000s.

Yet music games extend back even further. Even Nintendo’s own Mario Paint garnered a lot of attention for its fairly comprehensive composition feature, which allowed players to create songs on a scroll of sheet music by arranging Mario-based sound effects on each note.

As games technology grew into the 2000s, the ability to create and manipulate noise advanced alongside it. Harmonix entered the arena with its games Frequency and Amplitude, both early versions of Guitar Hero’s basic mechanics.

In the early 2000s, these games started exploring what could be accomplished with peripherals. The London Studio hit SingStar started shipping with microphones, allowing players to match their pitch against the console.

Of course, this led to the rise of Harmonix’ Guitar Hero, which has been one of the best-selling game series of all time with more than 40 million sales worldwide. The brand was ultimately culled by Activision after too many versions created fatigue, but its success can’t be denied.

Yet music games as a whole haven’t died at all. As motion technology has advanced, dance games are the order of the day. Harmonix is back in the spotlight with its Dance Central series, while Ubisoft’s Just Dance has earned the company plenty of success as well – the game is in its fourth iteration.

Despite a short period of over-zealous development and an emphasis on party games, the music genre continues to sell in the millions. But their success is puzzling, considering very few of these games actually teach any sort of musical skill.

A part of us all »

“Music is so important – it’s something that’s ingrained with all of us,” says David Wesley, a professor at Northeastern University. Wesley has authored a book about marketing tactics used in the game industry and has a keen interest in music titles.

“Miyamoto was a musician, artist and computer programmer. It was the concept of music being fundamental to the experience of gaming that helped make Nintendo as successful as it was,” he said.

Greg LoPiccolo says that as a result, exploring music through games is simply a natural extension of the medium. People are already interested in music, and they love games – it’s combining two major interests.

“It’s a great medium to explore with videogames,” he says. “There’s plenty of conceptual overlap and how you think about gameplay. There’s already a rhythmic component to a lot of non-music games already.”

Matt Burns, a former Halo producer and now independent developer of iOS music game Starbloom, says while the category of music games as a whole is experiencing fewer sales, creatively the scene is exploding. Mobile app stores have allowed the form to flourish.

“It’s definitely growing even bigger, because platforms like mobiles are great for trying out little ideas and experimenting in ways that are different.”

Gotta keep movin’ on »

As technology allowed music games to move from SingStar to Guitar Hero, the genre flourishes with the abundance of touch screens. One of the biggest surprises when the Apple App Store first launched in 2008 was the success of Tap Tap Revenge, a rhythm based game that ended up making the developer’s Australian co-founder, Andrew Lacy, quite a lot of money.

Others have followed. Child of Eden was launched by Tetsuya Mizuguchi in 2011, and used the motion controls in both the Xbox and PlayStation so players could interact with the screen and control rhythm in ways they hadn’t been able to before.

“The better the technology gets, the more interesting things we can do music-wise,” says Burns. “The huge focus has been on graphics, and it allows us to do cool things with music. If we want, we can use that power to generate music in real time and make the music more reactive to the input.”

That technology has now advanced once again. With full motion movement input devices now commonplace, the next evolution – dance has become popular again. And no plastic guitar required.

The melody of success »

The basic attraction of music games is actually quite simple – as David Wesley explains, it all harks back to basic psychology.

Having grown up around music, we’re accustomed to hearing songs played on the radio and on television. A game like Guitar Hero widens its appeal by including as many songs as possible. If you only like a few of the songs in the game, you’re still going to have a good time.

Of course, there’s also the indulgent aspect. The game is inviting you to become an expert, even though you’ve spent absolutely no time required in becoming good at playing an instrument.

Wesley points to a piece of research conducted by the Canadian Centre of Arts and Technology at the University of Waterloo, which experimented with slot machines to find out what made casual games so interesting.

“They require little or no training or previous experience; require little time commitment although players can continue to play for hours; are quick and easy to play and offer instant rewards for play in terms of feedback.”

That last point is crucial, Wesley says. Any rhythm game you know of, whether it be Guitar Hero, Dance Dance Revolution or anything else, all offer constant encouragement. Lights and statements like “good work” pop on the screen to drive you. Those types feedback loops exist in other games, but they’re much more obvious in rhythm games where the object is to hit as many notes correctly as possibly.

“Even if you’re a beginner, you see these lights and signs saying, “keep at it”, and you’re going to keep going it. It’s why people keep playing slot machines,” says Wesley.

“You play them enough to get a reward. It’s the same with music and dance games.”

This encouragement also has a secondary marketing tactic: it attracts groups who aren’t usually interested in games at all. Some older players won’t look twice at Call of Duty or complex strategy games, but pick up a plastic guitar and they’re good to go.

Dance games have gone even further with this concept. The fact you don’t need anything at all to start removes a large barrier to entry. All you have to do is move.

“One of the things that makes these games successful is that they appeal to a broader demographic,” says Wesley. “You’ll find people who are senior citizens and stay at home mothers who play these games, because they see different benefits than the traditional first person shooter.”

Although music games may have been criticized for making kids feel as though they’ve mastered the instrument without actually learning any music, there are definitely real world benefits to some of the more successful offerings.

Back in 2007, several schools in the United States started exploring how to tackle childhood obesity with Dance Dance Revolution. An extensive story in the New York Times chronicled the efforts of schools in several education systems to get kids dancing with the games in order to crack down on obesity, along with the possibility of diseases like diabetes and hypertension.

In a fairly obvious study from the Mayo Clinic which sparked the program, researchers found children playing Dance Dance Revolution used up more energy. The school system of West Virginia had a goal of installing the game in all 765 of its schools by the end of 2010. Some of the reports are astonishing – kids are losing dozens of pounds by dancing away.

“[Older players] don’t feel as guilty playing these games because they feel as though there’s a benefit,” says Wesley.

Dance the night away »

This is exactly why the current dance games have become so popular. They blend actual exercise with the constant feedback of a rhythm game.

In fact, music games in general have mostly been focused on rhythm. Guitar Hero requires no musical skill at all. All you need to do is have a sense of timing, and hit the right buttons and the right point.

“Even Guitar Hero was still essentially a rhythm based experience, just with more buttons. It’s only very recently that you can say that the genre has truly evolved to be about making real music,” says former Hyper editor, and musician Eliot Fish.

Even Harmonix, riding an extensive wave of success with its Dance Central titles, agrees.

“DDR proved the idea of a dance game was a viable prospect, and it’s been the foundation for some creative expressions. It’s amazing what people do with it. But in game specifics, it’s a stepping game,” says LoPiccolo.

Just Dance was arguably the first popular dance game to actually use dance moves rather than just moving your feet. The use of motion control allows players to move and spin their way into a sequence of movements that actually replicates a dance.

“DDR is less about socialising,” says Just Dance 4 creative director Alkis Argyriadis. “You’re focusing just on the steps and the screen.”

“I think with our game, you have to interact with your partner and you’re talking with your friends and so on. I think it’s totally different.”

These developers argue their games offer something a little more professional. The fact you’re actually moving around and completing the dance moves goes a long way to teaching a skill than a game like Guitar Hero.

“It’s a much more powerful teaching tool than people give it credit for,” says Greg LoPiccolo. “The toolset that’s there, and the quality of the sensing has really improved.”

This is actually where many developers believe music games are heading, to a place where there is an even closer relationship between what the player does and actual improvement in musical skill. You can spend 100 hours on Guitar Hero and know nothing about music, but spend 100 hours on Just Dance and the player will definitely see an improvement.

“The music genre has really rapidly splintered off into interesting little niches,” says Eliot Fish.

“It might be a difficult balance for a developer to strike when you’re involving the necessity for actual player competence with an instrument. I think that’s why games that involve more of the rhythm side of things tend to be more fun for non-musicians, as it’s a pure button coordination thing that delivers some sort of satisfying musical outcome, by the triggering of beats and sounds.”

As Argyriadis says, “you get into a flow when you’re doing very well”.

However, despite the challenges, there are some developers trying to make games that improve musical talent.  Ubisoft’s Rocksmith is a game that actually requires the input of a real electric guitar, and attempts to teach its users how to play. It’s breaking the mold of what is usually accepted as a music game – and the developers say it’s something they’ve struggled with.

“It’s been a challenge to get people to understand that Rocksmith is not something you bring to a party and get everyone to play immediately,” says producer Nao Higo.

“It’s more of a personal journey, where you play by yourself at first to hone your technique, and once you get to a point where you’re able to play songs at a decent level, then it becomes fun to play with other people.”

Rocksmith goes beyond even what Dance Central and Just Dance are doing, attempting to create a real learning experience where the practice is paramount. Higo says the game is a form of the “evolution of learning”.

This is actually where the rest of the industry wants to go, to a place where music composition is the norm. Developers recognise the beauty of music is being able to create your own – and they want to give other players that chance.

“The next stage for a music game is not saying I’ve learned to play some song by a band, but to take these fundamental musical concepts you use and play with them,” says Matt Burns. “That was one of the great things about Mario Paint – it’s just a natural evolution and you don’t need musical training.”

Greg LoPiccolo won’t say what Harmonix is working on right now, but alludes to the fact the company has allowed elements of musical composition in the past – like with Rock Band’s keyboards practice mode.

A game based on composition, he says, would be a “holy grail”.

“That would be where the point of the game is to create music, and collaborate on music, so that part of playing well is putting your own stamp on musical material.”

Whether the idea succeeds or not, these developers are in agreement the peripheral music game scene may have died for a while. “If it ever comes back,” says Higo “I don’t see it reaching the peak attained in 2008.”

Ultimately, the power of music lies in being able to create. Despite the genre’s past and present successes, music games still have a long way to go.


Most Read Articles

iOS 12 hands-on: 6 things we love (and 3 we don’t) about Apple’s latest OS

iOS 12 hands-on: 6 things we love (and 3 we don’t) about Apple’s latest OS

Sony celebrates 500m PlayStations sold with translucent PS4 Pro

Sony celebrates 500m PlayStations sold with translucent PS4 Pro

Is a Nintendo Switch Virtual Console coming as Nintendo bans retro ROM resources?

Is a Nintendo Switch Virtual Console coming as Nintendo bans retro ROM resources?

Apple AirPods 2: Wireless charging case revealed in iOS 12 beta images

Apple AirPods 2: Wireless charging case revealed in iOS 12 beta images

Would you like to receive

Our Newsletter?