The following chapter briefly outlines some of the factors that can affect the performance of a Perforce server, provides a few tips on diagnosing network-related difficulties, and offers some suggestions on decreasing server load for larger installations.

Tuning for Performance

Memory

• Determining memory requirements for large queries is fairly straightforward: the server requires about 1KB/file of RAM to avoid paging; 10,000 files will require 10MB of RAM.

• To cache db.rev, the size of the db.rev file in an existing installation can be observed and used as an estimate. New installations of Perforce can expect db.rev to require about 150-200 bytes per revision, and roughly 3 revisions per file, or about 0.5KB of RAM per file.

For most installations, a system with enough RAM for 1.5KB per file in the depot will suffice.

Filesystem performance

The only disk-intensive activity is file check-in, where the Perforce server must write and rename files in the archive. Server performance is heavily-dependent upon the operating system's filesystem implementation, and in particular, whether directory updates are synchronous.

Although Perforce does not recommend any specific filesystem, Linux servers are generally fastest, (owing to Linux's asynchronous directory updating), but may have poor recovery if power is cut at the wrong time. The BSD filesystem (also used in Solaris) is relatively slow, but much more reliable. NTFS performance falls somewhere in between these two ranges. The filesystems used by IRIX and OSF have demonstrated an excellent combination of both speed and robustness.

Issues with NFS file locking may arise if a Perforce server's database and versioned files are stored on an NFS-mounted volume. Database updates over NFS can be a problem, and file locking is generally slow; if the journal is NFS-mounted, all operations will be slower. Consequently, we strongly recommend that the Perforce database, depot, and journal files be stored on disks local to the machine running the Perforce server process.

Diskspace allocation

• Number and size of client workspaces

• Size of server database

• Size of server's archive of all versioned files

The client file space required is the size of the files that your users will need in their client workspaces at any one time.

The server's database size can be calculated with a fair level of accuracy; as a rough estimate, it will require 0.5KB per user per file. (i.e. A system with 10,000 files and 50 users will require 250M of diskspace for the database). The database can be expected to grow over time as histories of the individual files grow.

The size of the server's archive of versioned files will depend on the size of the original files stored and will grow as revisions are added. For most sites, allocate at least three times the size of the original files.

If you anticipate your database growing into the gigabyte range, you should ensure that your platform has adequate support for large filesystems. See "Allocate diskspace for anticipated growth" on page 12.

The db.have file holds label contents and the list of files opened in client workspaces, and tends to grow the faster than other files in the database. If you are experiencing issues related to the size of your db.have file and are unable to quickly switch to a server with adequate support for large files, deleting unused clients and labels and reducing the scope of client views can help alleviate the problem.

Network

Perforce uses a TCP connection for each client interaction with the server. The server's port address is defined by P4PORT, but the TCP/IP implementation picks a client port number. After the command completes and the connection is closed, the port is left in a state called TIME_WAIT for two minutes. While the port number ranges from 1025 to 32767, generally only a few hundred or thousand can be in use simultaneously. It is therefore possible to occupy all available ports by invoking a Perforce client command many times in rapid succession, such as with a script.

Before release 99.2, both the server and client side of the connection remained in TIME_WAIT, which meant that a script running on one user's machine could deprive other users of service by tying up all available ports on the server side. As of Release 99.2, only the client side goes into TIME_WAIT, leaving the Perforce server free to handle other clients.

CPU

In general, CPU power is not a major consideration when determining the platform on which to install a Perforce server.

Diagnosing Slow Response Times

1. misconfigured domain name system (DNS)

2. misconfigured Windows networking

3. difficulty accessing the p4 executable on a networked file system

Hostname vs. IP address

P4PORT=host.domain:1666

P4PORT=1.2.3.4:1666

On most systems, you can determine the IP address of a host by invoking:

ping hostname

Try p4 info vs. P4Win

If the former is fast and the latter is slow, you have a DNS-related problem. (When the Perforce server receives a p4 info request, it does a reverse name lookup in order to send back the client and server hostnames along with other configuration information. When it receives a P4win "Show Connection Info" request, however, it simply returns the IP addresses.)

Windows wildcards

p4 files //depot/*

p4 files "//depot/*"

DNS lookups and the hosts file

The corresponding UNIX file is /etc/hosts.

Location of the "p4" executable

p4 -V

Preventing Server Swamp

The number of files affected by a single command is largely determined by:

• p4 command line arguments (or selected folders in the case of GUI operations).

Without arguments, most commands will operate on, or at least refer to, all files in the view.

• Client views, branch views, label views, and protections.

Because commands without arguments operate on all files in the view, it follows that the use of unrestricted views and unlimited protections can result in commands operating on all files in the depot.

At sites with very large depots, unrestricted views and unqualified commands will make a Perforce server work much harder than it needs to. Users and administrators can ease load on their servers by:

• Using "tight" views

• Assigning protections

• Limiting maxresults

• Writing efficient scripts

Using tight views

In the loose view, the entire depot was mapped into the client workspace; for most users, this can be "tightened" considerably. The following view, for example, is restricted to specific areas of the depot:

Client views, in particular, but also branch views and label views, should also be set up to give users just enough scope to do the work they need to do.

Client, branch, and label views are set by the Perforce superuser or by individual users with the p4 client, p4 branch, and p4 label commands respectively.

Two of the techniques for script optimization (i.e. "Using branch views" on page 67 and "The temporary client trick" on page 67) rely on similar techniques. By limiting the size of the view available to the command, fewer commands need to be run, and those that are run require fewer resources.

Assigning protections

Unlike client, branch, and label views, however, the views used by protections can be set only by Perforce superusers. (Protections also control read and write permission to depot files, but the permission levels themselves have no impact on server performance.) By assigning protections in Perforce, a Perforce superuser can effectively limit the size of a user's view, even if the user is using "loose" client specifications.

Protections can be assigned to either users or groups. For example:

Perforce groups are created by superusers with the p4 group command. Not only do they make it easier to assign protections, but they provide a useful fail-safe mechanism called maxresults, described in the next section.

Limiting "maxresults"

To set this limit, fill in the Maxresults: field in the p4 group form. The number in this field specifies the maximum number of rows of data that can be operated on by a single command.

If a user is listed in multiple groups, the highest of the maxresults limits (but not including the default "unlimited" setting!) for those groups is taken as the user's maxresults value.

Example:  Effect of setting maxresults:

As an administrator, you wish members of the group rocketdev to be limited to operations of 20,000 files or less:

Group:        rocketdev Maxresults:   20000 Users:                 bill         ruth         sandy

Suppose that Ruth has an unrestricted ("loose") client view. When she types:

p4 sync

her sync command will be rejected if the depot contains more than 20,000 files. She can work around this limitation either by restricting her client view, or, if she needs all of the files in the view, by syncing smaller sets of files at a time, e.g.:

p4 sync //depot/projA/...
p4 sync //depot/projB/...

Either way, she'll get her files, but without tying up the server to process a single extremely large command.

To leave unlimited the number of result lines processed for a particular group, set the Maxresults: value for that group to unlimited.

Since maxresults values can make life difficult for your users, do not use them unless you find that certain operations are slowing down your server. The Maxresults: value should never be less than 10,000, since certain operations performed by P4Win, the Perforce Windows client, may require a Maxresults: value of between 5,000 and 8,000.

For more information, including a comparison of Perforce commands and the number of files they affect, type:

p4 help maxresults

Maxresults for users in multiple groups

A side effect of this is that you can't create a group that assigns "unlimited" maxresults values to superusers, because if any of the users in it also belonged to another group, the limit in that group would also apply to them. You can get around this by assigning a very high maxresults limit to your superusers group. For example:

(The largest possible maxresults is platform-dependent; on most platforms, this is a 32-bit integer.)

Scripting efficiently

This section points out some common efficiency problems and solutions.

Iterating through files

To this end, scripts should never iterate through files running Perforce commands when they can accomplish the same thing by running one Perforce command on a list of files and iterating through the command results.

For example, consider an approach like this:

for i in `p4 diff2 path1/... path2/...`
do
    [process diff output]
done

for i in `p4 files path1/...`
do
    p4 diff2 path1/$i path2/$i
    [process diff output]
done

Using list input files

For example, if your script currently does something like:

for components in foo bar ola
do
    p4 edit ${component}.h
done

for components in foo bar ola
do
    echo ${component}.h >> LISTFILE
done
p4 -x LISTFILE edit

Using branch views

p4 diff2 pathA/src/...   pathB/src/...
p4 diff2 pathA/tests/... pathB/tests/...
p4 diff2 pathA/doc/...   pathB/doc/...

and running this p4 command instead:

p4 diff2 -b pathA-pathB

Limiting label references

For example, this:

p4 files path/...@label | egrep "path/foo.h|path/bar.h|path/ola.c"

p4 files path/foo.h@label path/bar.h@label path/ola.h@label

p4 files path/foo.h@label
p4 files path/bar.h@label
p4 files path/ola.h@label

The temporary client trick

For instance, suppose your script runs these commands:

p4 sync pathA/src/...@label
p4 sync pathB/tests/...@label
p4 sync pathC/doc/...@label

and running:

p4 -c XY-temp sync @label

Checkpoints for Database Tree Rebalancing

Eventually, the tree may become sufficiently unbalanced that performance is negatively affected. The Perforce checkpoint and restore processes will re-create the trees in a balanced manner; consequently, you may see some increase in server performance following a restoration from a checkpoint.

Rebalancing the trees is normally only useful if the database files have become more than about 10 times the size of the checkpoint. Given the length of time required for the trees to become unbalanced during normal Perforce use, we expect that the majority of sites will never need to restore the database from a checkpoint (i.e. rebalance the trees) for performance reasons.