Helix Core Introduces S3 Storage Support

As organizations continue to migrate to the cloud and search for ways to reduce costs, Perforce has released Helix Core 2023.2 – giving users the ability to connect their archive depots to S3-backed (or S3-compatible) cloud object storage. S3 storage support offers users a durable and convenient solution that grows automatically and indefinitely. 

In the following article I explain S3 storage, diving into the benefits of using S3 storage, and how to get started using S3 storage within Helix Core. Read along or jump to the section that interests you the most:

What is S3 Storage?

S3 is an object storage service originally created by AWS for low-cost, resilient data storage. Today S3-compatible storage is available from dozens of different providers, including Azure, Google Cloud, Digital Ocean, Tencent Cloud, and many more. Starting with Helix Core 2023.2, users can utilize S3 storage to store archive depots directly from Helix Core.

Within Helix Core, you can reduce storage costs by storing your archive depots in Amazon S3 (the Simple Storage Service of Amazon Web Services), or an S3-compatible service. 

Common S3-compatible services include Azure, Digital Ocean, Minio, Google Cloud, etc. Note: This list is not meant to be exhaustive, and you should check with your cloud provider to see if it offers S3-compatible services.

📘 Related Resource: Perforce Cloud Hosting + Deployment Options 

Archive depots enable users to store file data located on their server to a bulk storage location (such as S3 storage) – freeing up disk space by relocating files that are rarely accessed. They are typically used for near-line or offline storage of infrequently accessed content. It’s important to note that metadata does not get moved to the archive depot location, just the file data.

What are the Benefits of Using S3 Storage?

There are several reasons why your team might decide to use S3 storage for archive depots within Helix Core.

S3 Storage Frees Up Storage in Your Server

By using S3 storage, only your metadata stays on your server, enabling you to view the history of your files while keeping the actual files in S3 object storage. 

S3 Storage is Cost Effective

Compared to block storage or the standard disk storage, S3 storage is a much more cost-effective option, saving your team both money and resources. Additionally, depending on your cloud provider, S3 storage has extra redundancy and backups built into the service.

S3 Storage Grows Infinitely

Another benefit of S3 storage is that it grows as your project does – removing the need for your team to manually add storage. 

S3 Storage is Always Available

Compared to archiving to physical disks, S3 storage is always available via your cloud provider, so it is a quicker and simpler operation to unarchive (restore) files or projects if they are needed in the future. 

S3 Storage Easily Integrates with Helix Core

Whether you've already deployed Helix Core to AWS (or another cloud provider that has S3-compatible storage) or your Helix Core server is on premise and you just use cloud S3 to store archived depots, it’s easy to get started. 

The main difference in using archive storage on the same cloud provider as your server is that you are more likely to have a faster connection and fewer egress fees. However, this can vary depending on your provider.

I’ll explain how you can get started further down in the article.

S3 Storage Trade-Off

Before deciding to store your data in S3 storage, it is important to consider the following trade-off: Compared to locally attached storage, S3 storage takes longer to load your data. For now, Helix Core is limiting S3 storage to archive depots to minimize the impact of this trade off. 

Potential Use Cases of S3 Storage within Helix Core

To best utilize S3 storage, I recommend users store the following kinds of data in their archive depots:

• Files that don’t need to be accessed frequently but are connected to a current project.
• Versioned files of a project that are no longer undergoing active development or maintenance. 
• Projects that were completed but may need to be updated in the future.

These types of files are particularly relevant for people within both the gaming and virtual production industries. Some potential use cases can include past seasons of a show, a movie or video game that will be made into a sequel, and older video game releases which don’t require frequent updates. For example, if your team has been working on a game for 10 years and there are over 100 revisions for each file, you could archive all the older versions of those files.

How to Use S3 Storage within Helix Core

To start storing archive depots in S3, users will specify the following while creating a depot:

• The location of the S3 bucket. 
• The credentials needed to access the bucket.

For more in-depth instructions for setting up your archive depots, you can refer to the Helix Core Server Administrator Guide: Configuring Your S3 Backed Archive Depot.

Perforce Solutions Engineer, Jase Lindgren provides detailed instructions for how to configure S3 storage for AWS + Helix Core and Digital Ocean + Helix Core in the following tutorials:

How to Set Up S3 Archive Storage for AWS + Helix Core

How to Set Up S3 Archive Storage for Digital Ocean + Helix Core

Stay Updated on What’s New in Helix Core

The What’s New Page for Helix Core has regular updates about the latest product features available in Helix Core, as well as the details for how to get started. Keep an eye out for the release of Helix Core 2024.1 slated for later this year to get further feature updates.

Get Started with Helix Core

If your team isn't already using our industry-leading version control, you can get started free for up to 5 users – with no limits on features or functionality. 

Helix Core is the leading version control system for teams who need to accelerate innovation at scale. Store and track changes to all your digital assets, from source code to binary to IPs. Connect your teams and empower them to move faster and build better.

TRY HELIX CORE FREE