p4 storage

Display, verify, or update physical archive storage.

Syntax

p4 [g-opts] storage [-v [-q]] [-c change] [-T tags -F filters] [-m max] archive...
p4 [g-opts] storage -u [-c change] [-T tags -F filters] [-m max] archive...
p4 storage -d [-c change] [-y] [-D secs] [-t target] [-q] archive...
p4 storage -w
p4 storage -U [-q] [-n threads] archive...
p4 storage -l start|pause|restart|wait|status|cancel //depotDirectory/...

Syntax conventions

Description

A user with admin access can display information about the server archive files and their usage with

p4 storage [-v [-q]] [-c change] [-T tags -F filters] [-m max] archive...

The -l option provides a group of commands to locate orphaned archive files and mark them for removal by the storage -d command.

Tip

86400 seconds (24 hours) is the default delay before file removals, but you can change this. See -d under Options.

See also "Reclaiming disk space by removing orphaned archive files" under Managing disk space in the Helix Core Server Administrator Guide.

The fields that storage displays are:

lbrFile	the path of the archive file
lbrRev	the revision of the archive file
lbrType	the file type of the archive file, such as text+C See File types.
lbrRefCount	how many revisions refer to the archive file
headTime	timestamp of when the record was written
filesize	size of the revision contained in the archive file
digest	MD5 digest of the revision contained in the archive file
serverSize	Size of the archive file on the server at the time that revision was submitted or shelved
compCksum	MD5 of the archive file containing the revision on the server if the file is compressed and `submit.storagefields` is `1`

Additional fields when the -v flag is specified:

actualSize	actual size of revision contained in the archive file
actualDigest	actual MD5 digest of revision contained in the archive file
status	outcome of verify, OK, BAD or MISSING

If upgrading from a version prior to 2019.1

If you are upgrading your Helix Core Server from a version prior to 2019.1 and your Perforce database has many keyed revisions, you might want to avoid the risk of delay incurred by calculating storage digest updates during the upgrade process. If so, consider using the -U option.

Options

-v

Verifies that the specified archive files are intact. Computes and displays the digest of each archive.

If an archive cannot be reproduced (for example, if the file is missing from the archive), the archive's output line ends with MISSING!
If there is a saved digest, p4 storage compares it with the computed one. If they differ, the output line ends with BAD!

p4 storage -v is similar to p4 verify except that:

for storage -v the archive... argument is matched against the lbrPath of the storage tables
for p4 verify the archive... is matched against the depotpath field of the revision tables

-u

Computes and saves the digest only for revisions that have no saved digest.

-q

Minimizes the output of the storage -v command, displaying only errors from mismatched digests or unreproducible revisions.

-d

Deletes unreferenced archives. The database row describing the archive is removed as well as the revision of the underlying archive if both of the following are true:

the specified archive has a reference count of 0
86400 seconds (default value for 24 hours) has elapsed since the storage record was identified by the scanning task
p4 storage -l start //depotDirectory/...
This expiry time can be adjusted using the lbr.storage.delay configurable and is intended to protect the storage records and archive files of in-progress submits and in-progress shelves from being considered for deletion. To override this configurable, specify the -D secs parameter on the command line. For example, a one minute delay would be p4 storage -d -D 60 -y //depotDirectory/...

Without the -y as confirmation, the command merely lists the candidate files and revisions it would remove.

With the -y as confirmation, files and revisions as well as zero referenced storage records are removed.

Archives with a reference count of 0 are rare, but can be created by:

incomplete p4 submit or p4 shelve commands, system crashes, other similar errors
p4 storage -l start //depotDirectory/...

The listing of the candidate revisions can be suppressed with the -q option.

-t target

Filesystem target directory to which orphaned archive files are copied prior to deletion from the directory indicated by //depotDirectory/...

-c Applies to the specified changelist.

-T

Applies to the specified tags, which can be specified using a comma- or space-delimited list.

Example: -T "lbrFile, lbrRev"

-F

Applies to the specified filter. Lists only records satisfying the filter expression. This filter syntax is similar to the one used for jobs -e jobview and is used to evaluate the contents of the fields in the preceding list. Filtering is case-sensitive.

Example: -v -F "lbrRefCount = 1 & status=OK"

-m Limits output to the specified number of records

-w

Provides a message that indicates when the upgrade process is complete by displaying "The storage upgrade process is complete". For details, see Helix Core Server Administrator Guide on upgrading to 2019.1 or later.

-U [-q] [-n threads]

To avoid the delay incurred by calculating the digest during the upgrade, consider using the -U option after a storage upgrade has been performed with the lbr.storage.skipkeyed configurable set to skip keyword revisions.

This option reads all the storage records. When it detects an unset digest field, it recomputes the digest and size fields, then updates the record. Invoking p4 storage -U -q -n threads suppresses progress messages.

The -n threads option specifies the number of threads to use: 0 for no threads, or n to use the specified number of threads. If n is not specified, the command uses the lbr.storage.threads configurable to determine the number of threads.

-l

Allows the administrator to create, pause, restart and cancel background processes that can locate and mark orphan files.

This process looks for files in the specified //depotDirectory and its subdirectories that match the naming convention for the server's archive file format. Each file that is found, along with its revision, is checked for a matching entry in the db.storage table. If no such entry is found, a new record is created that describes the file and revision combination with a reference count of 0.

A later call to p4 storage -d will delete both those records with zero references and the associated the archive revision.

p4 storage -l has subcommands:

start

Start a new background depot scanning task in the specified //depotDirectory to scan it and all its subdirectories. The special //depotDirectory argument of //... is handled by searching for all the local depots, including local stream depots, listed in the depot table and scanning their top and subdirectories.

There is no limit on the number of background processes that can be started, except the limits of the hardware and operating system. Each directory in the archive can only be the subject of a single scan.

If the server shuts down while a scan is unfinished, the scan is marked paused and the administrator can manually restart the scan again when the server is started.

pause

Suspend the named background scan process

restart

Resume the named paused background scan process.

If the server shuts down while a scan is unfinished, the scan is marked paused so that when the server is running again, the administrator can manually restart the scan.

wait

Wait until the named background scan process completes or fails due to an error.

The wait subcommand:

returns no output. It only returns when the scanner is no longer active, which means its state is done, error, or paused.
waits on a single scanner task. For example, with //... there is a single scanner task that searches all the local depots

status

Report on all the background scan processes if the special argument of //... is specified, or report on the named background scan process. For each scan, the following fields are displayed:

scanName	Name of the scan, the argument passed to the start subcommand, which is the path to scan. See the Example of a possible workflow
state	done, paused, run, busy, or error
dirsProc	The number of directories that have been processed
filesProc	The number of files that have been processed
zeroRecs	The number of zero referenced storage records created. This is the number of orphan revisions that have no matching storage record.
dirsErrs	The number of directories skipped due to errors
errmsg	The error message if the status is error

cancel

Remove the scan from the system. A scan remains in the system until it is canceled. To re-run a scan, first pause the scan (if it is active), then cancel the scan (even if it has completed or resulted in an error).

Warning

Scanning requires that each file only be addressed by a single path under the scanning directory. This requirement might be broken by the presence of symlinks. By default the scanner returns an error if it detects a symlink. To allow symlinks, set the lbr.storage.allowsymlink configurable to 1. This should only be done if all the symlinks only reference files and directories that are not under any scanned directory.
The scanner does not run if the depot is shared between servers.

g-opts

See Global options.

Example of a possible workflow

Locating the orphaned files

To start the scan:

p4 storage -l start //depotDirectory/...

To get the status of the scan:

p4 storage -l status //depotDirectory/...

To pause the scan:

p4 storage -l pause //depotDirectory/...

To end the pause and resume the scan:

p4 storage -l restart //depotDirectory/...

To cancel the scan:

p4 storage -l cancel //depotDirectory/...

Tip

A scan remains in the system until it is canceled. To re-run a scan, first pause the scan (if it is active), then cancel the scan (even if it has completed or resulted in an error).

Reclaim disk space on the server

To save the orphaned files to your target directory and delete the zero references along with associated archive files from the depot directory specified:

p4 storage -d -y -t /myTargetDirectory //depotdirectory/...

where /myTargetDirectory is any operating system target directory, and, for Windows, might be something like D:\myCopyOfOrphanedFiles

Usage notes

Can File Arguments Use Revision Specifier?	Can File Arguments Use Revision Range?	Minimal Access Level Required
No	No	`admin` or super to use the -U option