As organizations continue to migrate to the cloud and search for ways to reduce costs, Perforce has released Helix Core 2023.2 – giving users the ability to connect their archive depots to S3-backed (or S3-compatible) cloud object storage. S3 storage support offers users a durable and convenient solution that grows automatically and indefinitely.
In the following article I explain S3 storage, diving into the benefits of using S3 storage, and how to get started using S3 storage within Helix Core. Read along or jump to the section that interests you the most:
What is S3 Storage?
S3 is an object storage service originally created by AWS for low-cost, resilient data storage. Today S3-compatible storage is available from dozens of different providers, including Azure, Google Cloud, Digital Ocean, Tencent Cloud, and many more. Starting with Helix Core 2023.2, users can utilize S3 storage to store archive depots directly from Helix Core.
Within Helix Core, you can reduce storage costs by storing your archive depots in Amazon S3 (the Simple Storage Service of Amazon Web Services), or an S3-compatible service.
Common S3-compatible services include Azure, Digital Ocean, Minio, Google Cloud, etc. Note: This list is not meant to be exhaustive, and you should check with your cloud provider to see if it offers S3-compatible services.
📘 Related Resource: Perforce Cloud Hosting + Deployment Options
Archive depots enable users to store file data located on their server to a bulk storage location (such as S3 storage) – freeing up disk space by relocating files that are rarely accessed. They are typically used for near-line or offline storage of infrequently accessed content. It’s important to note that metadata does not get moved to the archive depot location, just the file data.
Back to topWhat are the Benefits of Using S3 Storage?
There are several reasons why your team might decide to use S3 storage for archive depots within Helix Core.
S3 Storage Frees Up Storage in Your Server
By using S3 storage, only your metadata stays on your server, enabling you to view the history of your files while keeping the actual files in S3 object storage.
S3 Storage is Cost Effective
Compared to block storage or the standard disk storage, S3 storage is a much more cost-effective option, saving your team both money and resources. Additionally, depending on your cloud provider, S3 storage has extra redundancy and backups built into the service.
S3 Storage Grows Infinitely
Another benefit of S3 storage is that it grows as your project does – removing the need for your team to manually add storage.
S3 Storage is Always Available
Compared to archiving to physical disks, S3 storage is always available via your cloud provider, so it is a quicker and simpler operation to unarchive (restore) files or projects if they are needed in the future.
S3 Storage Easily Integrates with Helix Core
Whether you've already deployed Helix Core to AWS (or another cloud provider that has S3-compatible storage) or your Helix Core server is on premise and you just use cloud S3 to store archived depots, it’s easy to get started.
The main difference in using archive storage on the same cloud provider as your server is that you are more likely to have a faster connection and fewer egress fees. However, this can vary depending on your provider.
I’ll explain how you can get started further down in the article.
Back to topS3 Storage Trade-Off
Before deciding to store your data in S3 storage, it is important to consider the following trade-off: Compared to locally attached storage, S3 storage takes longer to load your data. For now, Helix Core is limiting S3 storage to archive depots to minimize the impact of this trade off.
Back to topPotential Use Cases of S3 Storage within Helix Core
To best utilize S3 storage, I recommend users store the following kinds of data in their archive depots:
- Files that don’t need to be accessed frequently but are connected to a current project.
- Versioned files of a project that are no longer undergoing active development or maintenance.
- Projects that were completed but may need to be updated in the future.
These types of files are particularly relevant for people within both the gaming and virtual production industries. Some potential use cases can include past seasons of a show, a movie or video game that will be made into a sequel, and older video game releases which don’t require frequent updates. For example, if your team has been working on a game for 10 years and there are over 100 revisions for each file, you could archive all the older versions of those files.
Back to topHow to Use S3 Storage within Helix Core
To start storing archive depots in S3, users will specify the following while creating a depot:
- The location of the S3 bucket.
- The credentials needed to access the bucket.
For more in-depth instructions for setting up your archive depots, you can refer to the Helix Core Server Administrator Guide: Configuring Your S3 Backed Archive Depot.
Perforce Solutions Engineer, Jase Lindgren provides detailed instructions for how to configure S3 storage for AWS + Helix Core and Digital Ocean + Helix Core in the following tutorials:
How to Set Up S3 Archive Storage for AWS + Helix Core
How to Set Up S3 Archive Storage for Digital Ocean + Helix Core
Stay Updated on What’s New in Helix Core
The What’s New Page for Helix Core has regular updates about the latest product features available in Helix Core, as well as the details for how to get started. Keep an eye out for the release of Helix Core 2024.1 slated for later this year to get further feature updates.
Get Started with Helix Core
If your team isn't already using our industry-leading version control, you can get started free for up to 5 users – with no limits on features or functionality.
Helix Core is the leading version control system for teams who need to accelerate innovation at scale. Store and track changes to all your digital assets, from source code to binary to IPs. Connect your teams and empower them to move faster and build better.