Git LFS (Git Large File Storage) Overview
Git LFS (Git Large File Storage) is used by many teams to manage and store large files. Here, we break down what Git LFS is, what it does, when to use it, and more.
Read along or jump to the section that interests you the most.
What Is Git LFS (Git Large File Storage)?
Git LFS, or Git large file storage, is an open source Git extension used to manage large files and binary files in a separate ”LFS store.” Git large file storage keeps Git repositories at a manageable size.
Most projects today have both code and binary assets. Storing large binary files in Git repositories can be a bottleneck for Git users.
Back to topHow Does Git LFS Work?
Git LFS uses pointers (references) to files instead of storing the actual files or binary large objects (blobs) — a type of data that stores binaries as one entity — in the Git repository itself.
So, instead of writing large files/blobs to a Git repository, you write a pointer file. The files/blobs themselves are written to a separate server, referred to as the LFS store. Doing this makes it possible to version large files and manage blobs while freeing up space in Git repositories.
Getting started with Git LFS is straightforward. You download the extension and configure your file types.
Back to topShould I Use Git LFS?
You should use Git LFS if you need to manage large files or binary files when using Git. (However, you may not want to use Git LFS if you have artists and designers on your team who need to version their large binary art files. More on that in the next section.)
The reason you should use Git LFS or an alternative if you manage large or binary files is that Git is decentralized. This means every developer has the full change history on their computer. Changes in large binary files cause Git repositories to grow by the size of that file every time the file is changed (and that change is committed). That means it will take a long time to get the files. And if you do, it will be difficult to version and merge the binaries.
So, every time the files grow, the Git repository grows. This causes slowdowns when Git users need to retrieve and clone a repository.
Git LFS was created to solve these problems. But it has problems of its own.
Back to topProblems with Git LFS
Git LFS works, but many teams who use it find it hard to manage. Here are some reasons to consider finding an alternative to Git LFS:
Setting Up Git LFS Is Time-Consuming
To use Git LFS, every user must have it installed on their server and workstation. This is time-consuming to do and a burden to admins. Then, once it’s installed, there is no visibility and little control over it.
Maintaining Git LFS Takes Extra Steps
It takes extra steps to maintain Git Large File Storage because you have to set it up for every Git repo, i.e. every Git project. That means for every repo, you have to install Git LFS, tell LFS to track a file type, then add the tracking information to the repo so that when you commit a file of that type, it will be placed in an LFS repository. This is challenging for any users who do not already know Git well.
Git LFS Is Not Designed for Artists
Git LFS helps software developers, since it makes cloning and branching much easier. But for most teams who work with artists or designers, it is not a good solution for these key reasons:
- It does not integrate with popular art and design programs.
- Non-coders must still pay the performance penalty of large binaries if they have to pull their assets from it.
- It is a command line-driven tool, so users must learn some commands to get or commit assets. Many artists struggle with this or will not do it. There are some graphical tools for Git LFS, but game engines and design tools do not have good integration with Git, if they have it at all.
- Being a command line-driven tool also makes it tricky to find the right version of a file, which makes it difficult for artists to iterate on specific assets.
For these reasons, Git LFS is not an ideal solution for game development or virtual production teams.
The above issues can slow down your team’s performance. So, even though Git itself is free, the consequences of using it when your team needs a faster, more scalable solution can be costly.
Back to topAlternatives to Git Large File Storage
Git LFS isn’t the only way to manage large files in Git. Alternatives include other open source or third-party fixes, such as:
- git-annex
- git-bigfiles
- git-fat
- git-media
- git-bigstore
- git-sym
These options still have the same issues as Git LFS: They are command line driven tools, they don't integrate with art or design tools, you still need to get and send the files if you are using them (meaning you still have to wait), and finding a recent version of a file is challenging. There’s a better way to manage large files and binary files.
Back to topThe Best Version Control for Large File Storage: Helix Core
Today’s projects are bigger, with more files and mixed assets than ever before. Git and Git LFS alone can’t manage that. But Helix Core can.
Helix Core — version control software from Perforce — is the best version control for large file management for a number of reasons:
Binary File Handling
It’s ideal for managing large binary files. With Helix Core, large file storage is a native capability, not an add-on. It allows you to store binaries alongside your source code. In fact, all of your largest files — binary files, source code, art files, video files, images, libraries, and build artifacts — can live together in a single repository. Storing all assets in one repository allows large, distributed teams to move faster.
Scale
Helix Core is designed to scale as teams grow globally. Git is usually accessed via a single server that could be thousands of miles away from the user. With Git LFS, the user still must reach across the network to get the binary files they need. Thanks to distributed architecture, Helix Core can be accessed via edge servers that put the assets closer to the user, meaning they get them faster.
Security
In Helix Core, you can set permissions down to a single file and IP address, so team members and outside contributors can access only the files they need, or that you decide they need. This reduces the data size and protects your intellectual property. You can’t do this in native Git, and even if you use a Git manager like GitHub, you still only set permissions per repo or branch, not individual files.
Collaboration
Helix Core is better for teams who collaborate on digital assets because it offers better file locking than Git LFS. LFS’ “file locking” functionality is really only a “repo lock” that prevents others from pushing their changes to the master repository if their push or merge contains files that are locked by another user.
Helix Core, on the other hand, uses an actual file locking strategy. If you try to submit a pending changelist with locked files, you will get an error. If that happens, you can easily remove the locked file from your changelist and submit everything else. In addition, global lock visibility helps prevent conflicts from occurring in the first place. No conflicts, no throw-away changes.
See for yourself why Helix Core is the tool of choice to manage large binary files.
➡️ manage Large Files in Helix Core
If you want to migrate your git LFS repo to Helix Core with history, even if it has hundreds of GB of binary data stored, please contact Professional Services for options.
Back to topGit Large File Storage Alternative: By the Way, Perforce Has Git, Too
Do you have teams who need to use Git? Perforce has multiple options. If you use Helix Core in addition to Git, your Git assets can simply be mirrored into Helix Core using the Git Connector, which is free to use for licensed Helix Core customers. In addition, Perforce offers a code review tool for Git assets stored in Helix Core through Helix TeamHub. Helix TeamHub is also free to use for licensed Helix Core customers when configured in this manner.
Helix TeamHub can be used independently of Helix Core as well, either in the cloud or on-premises. When used this way, it can not only be used to host Git repositories, but also SVN, Mercurial, WebDav, Ivy, Maven, and docker repositories. When using the Perforce-hosted cloud offering, your team can start free for up to 5 users and 1GB, then pay as you grow.
Related Content:
Back to top