A Portuguese university professor Nuno Fonseca once started applying a computer graphics technique to simulate sounds of explosions, sandstorms, or battlefields. Now, the software that his startup Sound Particles developed is used by the creators of such blockbuster films as Aquaman, Wonder Woman, and Game of Thrones.
I met Mr. Nuno Fonseca during this year’s virtual Web Summit, where he was networking with other professionals and hunting for investors, as he is looking forward to scaling his business. He mentioned that he is running a startup and that the software he created is being used in major Hollywood studios and productions, such as Game of Thrones and Star Wars.
“The idea initially started around 2006, when I realized the most interesting visual effects that I was seeing in the movies were using particle systems, a technique widely used in computer graphics for many years”, he told me.
Naturally, I was intrigued by what a small Portuguese startup has to offer to Hollywood. So, we sat down (virtually, of course) with Nuno Fonseca to discuss his innovation.
“Most of the times you see an explosion in a movie, chances are that it is not an actual explosion but a computer-generated effect,” he told CyberNews. The effect is generated by using particle systems – a computer graphics technique that creates thousands or even millions of small objects to simulate fire, rain, sandstorms, dust, smoke, and explosions.
“This works by combining a huge number of small particles (pixels or small groups of pixels) with certain initial characteristics such as color, movement, that change over time. So roughly speaking, for the explosion example, you might have a certain amount of particles concentrated in a particular position that initially are yellow in color but, as time goes by, they move towards random directions while their color changes to orange/red and eventually becomes black/grey to simulate the smoke,” he explained.
How does it work?
Usually, this technique is used to create a visual effect, but a Portuguese professor came up with the idea to apply the same concept to sound and use thousands of different sounds to create the movie soundscape faster. The premise is the same: taking small elements to create something that’s complex.
“Imagine that you are a Hollywood sound artist trying to create the sounds of a battlefield with thousands of soldiers fighting. The traditional workflow would be to start adding sounds of gunshots or explosions one-by-one and creating layers of sounds until you reach a point of complexity that represents what is going on in the picture. However, this procedure takes a really long time and is very tedious,” Nuno Fonseca explained.
With Sound Particles software, it’s possible to create a sonic particle system that resembles a battlefield, which is composed of thousands of particles, each with random movements, and import 50 war-related sounds.
“Then, each particle would randomly select and play one of those 50 sounds, while our engine renders the sound of each one of the thousand particles from a certain perspective. This means that, within a few minutes, you can create something that would otherwise take much longer and consequently involve much more money. This is why Hollywood studios are moving towards this type of approach,” he explained.
His goal is to eventually do to audio what Pixar did to animation – make it 3D.
“Right now, the animation is all made in 3D. However, the sound/audio workflows that create the sounds for those animations are almost always made in 2D. For us, it makes no sense at all. Thus, our main goal is to completely change the audio workflows that are currently being implemented and shift them 100% to 3D. It is what Pixar made a few decades ago with animation, and that is exactly what we want to do with sound,” Nuno Fonseca said.
Sound Particles’ software was already used in productions like Game of Thrones, Maleficent 2, and Aquaman. Even though technology saves time and effort, it still requires a lot of human assistance.
What about the quality of sound?
The only process that doesn’t require human assistance is the render itself. Sound effects professionals still need to select and import the source files and set up the right parameters for the specific project they are working on.
“For instance, for the battlefield example, the project would have completely different settings and source files when compared to a project that was meant to create the sound of an earthquake,” Fonseca told CyberNews.
The source files are created by professionals who work on movies or other types of production. They may be recorded or synthesized previously and then imported into Sound Particles. Then, users can set up the particle systems as they wish, and Sound Particles will render all the sounds together.
As it supposedly saves time and effort, I wondered whether it impacts the quality of sound. Fonseca insisted that it doesn’t compromise it.
“Additionally, since Sound Particles uses particle system techniques where everything is associated with some kind of randomness, the results sound very organic and natural,” he said.
Fonseca says that there’s a difference between a human-made and a software-made battlefield sound.
“With Sound Particles, you can create soundscapes with millions of sounds. Whereas an average audio professional wouldn’t be capable of generating such a complex sound (maybe they would be capable of doing it, but they would spend days/weeks of work),” he said.
