IO Interactive Has Been Creating Hitman Best Sellers on Helix Core for More Than 10 Years
IO Interactive is an independent video game development studio. If you are not familiar with their name, you are probably familiar with their work. From 2000-2018, IO Interactive (also called IOI) released 12 video games, including their best-selling series, Hitman.
IOI moved to Perforce in 2008. Now, a senior engineer at IOI describes Helix Core as “the most important service we maintain.”
A Version Control System for Game Development
Managing source code and binary files in the same repo
One repo means easier workflows, which is faster and less expensive
A single source of truth for more than 80 million files
When a developer says, “Sync to this changelist,” we know everything will work. We know with confidence that the binaries and source code belong together at a specific slice of time. So we know they work together.”
Binary Assets Can Co-Exist With Source Code
Like most AAA game development studios, IO Interactive (or IOI) has an enormous number of files. They have more than 80 million files in source control and another million in output files. Files range from a few bytes to more than 100 GB. And file types include everything — binaries, text files, configuration files, markup, JSON, various script and source code, build files, and packages.
Another thing they have in common with most AAA game dev studios — they store binary files and source code together in Helix Core.
Before IOI started managing source code in Helix Core, they stored binary media files and graphic art files in Alienbrain, which is a digital asset manager. But once the source code was in Helix Core, it became obvious Alienbrain was too slow to justify its continued use. The team at IOI knew that having binaries in the same repo as the source code would make the workflows easier, which, ultimately, means cheaper. Plus, storing everything in the same repo just made more sense.
Helix Core Is More Than a Digital Asset Manager
IOI has internal code data dependencies between the binary files and the code, so for releases, they have to be together. They have to be identifiable with a changelist or some other common identifier.
Although there are challenges with using data dependencies, when binaries were in Alienbrain, dependencies would have been impossible. The source code and binaries didn’t even know about each other.
Alienbrain markets itself as a digital asset manager (or DAM) for artists. Although it’s possible for DAMs to have version control, it’s not automatic. Moreover, it’s challenging to match specific versions of source code from the VCS to the correct version of binary assets.
Because IOI has data dependencies in their code, it’s important that the team knows with certainty what version of the code needs what version of the data. If either is out of sync, the game build crashes. When the binaries were in Alienbrain, we needed to combine the code with the data in a separate process. This made it asynchronous and increased the possibility of introducing errors.
Now, with Helix Core, most binary files are submitted in the same changelist as the corresponding source code. That means a developer can say, “Sync to this changelist,” and everything works.
“The binaries and the source code belonged together at a specific slice of time. We know they work together,” says Daniel Heeris, Senior Technical Support Engineer at IO Interactive
Using Continuous Integration and Continuous Development to Release Every Month
Although IOI told us that their entire organization isn’t DevOps yet (who is?!), they have many teams that have achieved DevOps, and the methodologies help deliver quality software in a timely fashion.
Continuous development of new content and new features has become increasingly important in game development for keeping audiences engaged. As developers complete their work items, that code moves to QA. When approval comes from QA, they move to a release branch. There, they fix known issues and send builds back to QA. The cycle is repeated until they’re confident they have a release candidate. (At IOI, a release candidate is something that could pass the submission test for platform providers.)
If any issues come back after the release candidate is submitted, they fix it, and then create release candidate #2. That cycle is repeated until they are approved. Then, they push that through. The end product is the release of updates every month.
Fixing Issues on Main
When the team finds issues in the branch, the issues are fixed on main. Developers make integration requests in Jira, and the release manager determines if the changelists should be brought in.
The team created an automated tool to integrate changelists that have been approved. The tool automatically resolves some known types of conflict, and so it’s able to merge files that can’t normally be merged. Then, the changes are integrated from main to the release branch, and the tool also automatically closes the related Jira ticket.
Creating the Release Candidate
Once they have a good batch of fixes, they fire out another build. Because they do Continuous Integration, they’re constantly building. They send a new build to QA. QA lets them know whether the build is ready. This cycle repeats until the release candidate is ready to ship, and the Hitman magic continues!
Helix Core Is the Industry Standard for Game Developers
Learn why 95% of the top game dev studios use Helix Core for version control.