Checkpoint files

A checkpoint A backup copy of the underlying metadata at a particular moment in time. A checkpoint can recreate db.user, db.protect, and other db.* files. See also metadata. is a file that contains the information necessary to fully restore the metadata The data stored by Helix Core Server that describes the file revisions in the depot, where they get their content from (see lazy copy), and the current state of client workspaces, protections, groups, users, labels, streams, and branches. Metadata is stored in the server database and is separate from the archive files that users submit. in the Helix Core Server (p4d) database, such as changelists, branches, users, labels, protections, and jobs.

When you create a checkpoint, the metadata database is locked, enabling you to take an internally consistent snapshot of that database with transactional integrity. You can create a checkpoint while p4d is running. However, checkpointing can affect performance because all the database tables are locked while the command runs. You might want to schedule checkpointing to occur when the load from users is light.

To ensure that you have an incremental metadata backup between checkpoints, perform journal A file containing a record of every change made to the metadata since the time of the last checkpoint. This file grows as each transaction is logged. file rotation often. Consider whether weekly checkpoints and daily journal rotation might be appropriate for your organization. Regular checkpointing is important to keep the journal from getting too long. Making a checkpoint immediately before backing up your system is good practice. For more information about journal rotation, see Journal files.

Checkpoints and versioned files

Checkpoints do not contain the content of versioned files Source files stored in the depot, including one or more revisions to each file. Also known as archive files, archives, and depot files. Versioned files typically use the naming convention 'filename,v' or '1.changelist.gz'., only the information required to restore the database of metadata. Versioned files need to be backed up using the utilities available for your operating system.

To guarantee database integrity upon restoration, the checkpoint must be as old as, or older than, the versioned files in the depot. This means that the server database must be checkpointed, and the checkpoint generation must be complete, before beginning the backup of the versioned files.

Create a checkpoint

Checkpoints are not created automatically. Someone or something must run the checkpoint command on the Helix Core Server machine.

You can create a checkpoint by running:

            p4d -r serverRootDirectory -jc

Where serverRootDirectory overrides the default, which is the current working directory or $P4ROOT if set.

The resulting checkpoint file is also created in that same directory. (If that directory does not have the database files, they will be created.)

Because the purpose of the checkpoint (and journal) is to enable you to restore the server database, it is safer to store the checkpoint file in a different location than the server database.To customize the location and name of the checkpoint and journal files, see Use a prefix.

A checkpoint can also be created by using p4 admin checkpoint, which does not require a login account on the Helix Core Server machine.

For more details about options, see Checkpoint and journal options in Helix Core Server (p4d) Reference.

Checkpointing the commit or master server

Most aspects of the checkpoint process are the same on all platforms. This table outlines the process and indicates where there are differences.

UNIX Windows

When you run the command to create a checkpoint, p4d creates a checkpoint file named checkpoint.n, where n is the new value of the journal counter..

You can also provide a journal prefix for p4d -jc or set the journalPrefix configurable to assign a location for rotated journals and checkpoints.

p4d renames the journal file to a file named journal.n-1 in the P4ROOT directory.

p4d copies the journal file to a file named journal.n-1 in the P4ROOT directory.

On Windows, the default requires a copy and rename operation. To enable atomic rename, which might improve performance on Windows, set the filesys.atomic.rename configurable. Atomic rename is applicable to NTFS file systems where the journal and rotated journal are on the same disk volume.

Journals rotated to a different disk volume will be copied and renamed.

Providing a journal prefix for p4d -jc or setting the journalPrefix configurable allows you to set a prefix that is applied to checkpoint and rotated journal file names and that can specify the directory in which to store checkpoint and journal files.

p4d truncates the current journal so that, going forward, the current journal will not have any content that is older than the most recent checkpoint.

p4d writes the MD5 checksum of the checkpoint to a separate file, checkpoint.n.md5, and the lastCheckpointAction counter is updated to reflect successful completion. This can be used to verify that the associated checkpoint (checkpoint.n) has all the server metadata of the time the checkpoint was taken.

The workspace have list An internal list indicates which files and revisions the client workspace has sync'd from the depot. See 'p4 have' in Helix Core Command-Line (P4) Reference. for a client workspace that is readonly or partitioned is neither journaled nor checkpointed. Consider using your system backup utilities if you have such clients.

When verifying the MD5 signature of a compressed checkpoint, the checkpoint must first be uncompressed into a form that reflects the line ending convention native to the system that produced the checkpoint:

LF line endings on UNIX

CR/LF line endings on Windows

You can verify the integrity of a checkpoint by using the p4d -jv command.

Checkpointing and types of servers

Replicas, Edge Servers and standby servers cannot be checkpointed by using the p4d -jc command. Instead, use the p4 admin checkpoint command. This schedules checkpointing of the replicas, Edge Servers, and standby servers to occur when the master server performs its next journal rotation.

Distribution servers can be checkpointed with either the p4d -jc command or the p4 admin checkpoint command.

To create a checkpoint without being logged in to the Helix Core Server machine:

$ p4 admin checkpoint [-z | -Z] [prefix]

p4d -jc and p4 admin checkpoint

This table summarizes the use cases for two approaches to creating checkpoints.

p4d -jc p4 admin checkpoint

For master, commit, and distribution servers

p4d -jc	p4 admin checkpoint
For master, commit, and distribution servers	Must be used for replicas, Edge Servers, and standby servers but can be used for all server types
Can be used by a system administrator who is logged in to the machine that runs Helix Core Server. This system administrator does not need to have a user account on Helix Core Server.	Can be run from a client machine or on the server machine. Only a Helix Core user with the access level of `super` or `operator` can run `p4 admin checkpoint` and `p4 admin journal`
Does not require Helix Core Server to be running.	Requires the user to be logged in to Helix Core Server, but this user does not need a login account to the machine on which Helix Core Server is running.

Must be used for replicas, Edge Servers, and standby servers but can be used for all server types

Can be used by a system administrator who is logged in to the machine that runs Helix Core Server.

This system administrator does not need to have a user account on Helix Core Server.

Can be run from a client machine or on the server machine.

Only a Helix Core user with the access level of super or operator can run p4 admin checkpoint and p4 admin journal

Does not require Helix Core Server to be running. Requires the user to be logged in to Helix Core Server, but this user does not need a login account to the machine on which Helix Core Server is running.

Use a prefix

If no prefix is specified, checkpoint.n and journal.n-1 are the default filenames.

To specify a prefix or directory location for the checkpoint and journal, use the -jc option. For example, you might create a checkpoint with:

p4d -jc prefix

In this case,

your checkpoint files are named prefix.ckp.n, where prefix is as specified on the command line and n is a sequence number
your journal files are named prefix.jnl.n-1, where
- prefix is as specified on the command line
- n-1 is a sequence number that is one less than the journal counter for the checkpoint number

A prefix can specify a directory

You can store checkpoints and journals in the directory of your choice by specifying the directory as part of the prefix. For example:

            p4d -r serverRootDirectory -J /path/to/journal -z -jc /path/to/checkpoints/mybackup

returns

Checkpointing to /Users/bruges/server151/checkpoints/mybackup.ckp.299.gz...
MD5 (/Users/bruges/server151/checkpoints/mybackup.ckp.299) = 5D7D8E548D080B16ECB66AD6CE0F2E5D
Rotating journal to /Users/bruges/server151/checkpoints/mybackup.jnl.298.gz...

You can also specify the prefix for a server with:

p4 configure set journalPrefix=prefix

Setting the journalPrefix configurable overrides the default name and directory for rotated journals and checkpoints. For an example, see journalPrefix in the Helix Core Command-Line (P4) Reference.

Multi-server and replicated environments

Setting the configurable for the journal prefix is a best practice for multi-server and replicated environments. For example, you might avoid confusion if you specify that an indicative term, such as "edge-London" or "standby-Paris", be part of the name.

Automated checkpoints

You can set up an automated program to create your checkpoints on a regular schedule. Be sure to always check the program’s output to ensure that checkpoint creation was started. Compare the checkpoint’s actual MD5 checksum with that recorded in the .md5 file, and back up the .md5 file along with the checkpoint. After successful creation, a checkpoint file can be compressed, archived, or moved onto another disk. At that time or shortly thereafter, back up the versioned files stored in the depot subdirectories.

Restore a database to an older checkpoint

The sequence numbers reflect the roll-forward nature of the journal. To restore the database to an older checkpoint, match the sequence numbers of the checkpoint and the journal A file containing a record of every change made to the metadata since the time of the last checkpoint. This file grows as each transaction is logged..

For example, to restore the state of Helix Core Server to what it was when checkpoint.6 was taken, restore checkpoint.5 and then load journal.5 which contains all the changes made between checkpoint.5 and checkpoint.6. In most cases, you only want to restore the current database, which is reflected by the highest-numbered checkpoint.n rolled forward with the changes in the current journal.

If the checkpoint command fails

Checkpoint failure is usually a symptom of a resource problem that can put your database at risk if not handled correctly. If the checkpoint command fails, request assistance from the Perforce Support Center without delay.