Larian Studios speeds game development by moving data 5x faster between locations

Introduction

Best known for its work on the critically acclaimed Divinity role-playing game (RPG) series, Larian Studios is an award-winning videogame developer headquartered in Ghent, Belgium. Thanks in large part to the ongoing success of its Divinity titles, the studio has expanded in recent years. It now has satellite offices in Dublin, Ireland; Quebec, Canada; Saint Petersburg, Russia; as well as a new one in Kuala Lumpur, Malaysia. All five locations are connected via a tightly encrypted VPN and share game data and other files over 1Gbps+ WAN connections.
‍
As part of the studio's core development workflow, nightly builds and software assets have to be in sync. Maintaining a high rate of transfer for terabytes of data across long geographic distances is a top priority. In 2019, Larian adopted Resilio Platform after testing several competing solutions.

Challenges

#1: Migrating Seamlessly to a More Capable Sync Solution

Prior to moving to Resilio Platform, Larian had been using the built-in synchronization functionality provided by its existing storage solutions.
‍
"We were using the basic tools offered by our NAS and our servers to sync data. That basically meant rsync and other default options that you'll find in Linux," says Lead Tools Programmer Sebastiaan Sprengers. "And we found that speed wasn't that great. In Europe, it was fine, but once we started syncing to our offices in Quebec or Saint Petersburg, we saw that it wasn't ideal for big files. We just had issues with connection drops, high latency and packet loss."
‍
This clearly highlighted a need for a faster and more robust data-synchronization solution. But Larian didn't want to be forced to change or add to the storage infrastructure it already had in place. Any new solution would therefore need to offer more than superior performance. It would also have to integrate smoothly—and affordably—with QNAP and Synology enterprise NAS devices as well as Larian's Linux-based servers.

#2: Keeping Pace in a Dynamic Industry

Larian hasn't just expanded to new international locations. Its third-party partner network has grown too. For more recent titles like Baldur's Gate III, the studio has been working with UK-based voiceover artists and motion-capture experts in addition to augmenting its in-house QA teams.
‍
That's increased the daily amount of data transferred both internally and externally. It's also added to the number of users who are contributing to and drawing from those shared file pools. On top of that, the international collaboration has led to tighter turnaround times: New audio files and game builds need to be processed and tested as soon as they're ready.

Our QA teams are spread out Worldwide and if they can’t get the new version with the latest changes, sometimes we would lose a full day of QA," says Producer Octaaf Fieremans. "And when you're working against deadlines or submissions for platforms like PlayStation or Xbox One, that can really hurt you."

Solution

"Once we looked at the functionality and the price and how it was set up, Resilio came out as the best choice," says Sprengers. "It was easy to set up with our in-place hardware. We could install it on our current NAS and servers, and basically just plug into what we already have."
‍
Along with ease of migration, Resilio brought massive speed gains. Sprengers estimates that it was "four to five times faster" than the previous methods in transferring data to Quebec and other distant locations. That in turn has accelerated the entire development process.

"The nightly builds that we pump out involve a large amount of data. Getting that synced to every QA team in each studio proved to be quite difficult with the software that we had. With Resilio, as soon as our build servers have a build ready, it's using the entire throughput of our gigabit lines to distribute it to the other studios. So QA can start in the morning on a fresh build," says Fieremans.

That speed and resiliency have also smoothed Larian's path to expansion. When it opened its Kuala Lumpur office, Resilio Platform was immediately able to overcome the routing and ISP issues that had a detrimental effect on other network activity.
‍
Resilio Platform has also become the lynchpin in the studio's workflow with its outsourcing partners. "We've set up Resilio on our FTP server, so we can have externals get data from us or put data on there," Fieremans says. "We then use Resilio to sync that to a local file server in one of our offices. People can easily access the data locally instead of downloading from the FTP server. Which is a big plus."

Impact

Without Resilio Platform, Larian faced two possibilities. The studio could continue to use slow and unreliable generic sync solutions or it could install a new, expensive sync solution with added infrastructure. "Other solutions had too many features that we didn't need or required new servers. We weren't looking into completely changing our infrastructure or setting up servers everywhere all over the world," says Sprengers.

"For us, the main thing was fast transfer of data and obviously maintaining the integrity of all the data that we sync to our offices. And definitely the cost is very important. We were happy to find a solution where we could use our in-place architecture to keep down the cost."– Sebastiaan Sprengers, Lead Tools Programmer @Larian Studios.

Resilio Platform has given Larian that cost-effective means to synchronize vast amounts of data between its own offices as well as outside partners at speeds close to their maximum available bandwidth. That's helped to optimize the studio's most critical workflows, enabling Larian to be more agile as a whole and continue its long record of success.