Release Notes for P4P, the Perforce Proxy Version 2008.1 Introduction P4P, the Perforce Proxy, improves Perforce performance in WAN topographies by mediating between Perforce clients and servers to cache frequently transmitted file revisions. By intercepting requests for cached file revisions, P4P reduces demand on the Perforce server. To improve the performance obtained by Perforce clients that access a central Perforce server across a WAN, set up P4P on the network close to the clients and configure the clients to access P4P, then configure P4P to access the central Perforce server. With this configuration, file revisions are sent once from the central server and cached by the proxy. Subsequent requests for the same revisions are delivered from the proxy, reducing both network traffic across the WAN and CPU load on the central server. On a LAN, you can obtain performance improvements by setting up several proxies to divert the workload from the server's CPU and disks. Unless stated otherwise you can intermix any release Perforce client with any release Perforce server, but any functionality new to 2008.1 requires you to upgrade the client, the proxy and/or the server. The Proxy requires a server upgrade to 2002.2. You might see improved proxy performance if you upgrade your server to 2006.2 Additional information is available in the Perforce System Administrator's Guide, available from the Perforce Web site. System Requirements * Perforce server: 2002.2 or higher * Perforce clients: any version, any client program * OS requirements: sufficient disk space for storing file revisions (P4P does not store server metadata) Starting P4P To start P4P, you invoke the p4p executable. For example, the following command line starts a proxy that communicates with a central Perforce server located on a host named 'centralserver', listening on port 1666. Perforce clients connect to P4P on port 1666 on the machine where the proxy runs. P4P file revisions will be stored under a directory named /local/proxyroot. p4p -p 1666 -t centralserver:1666 -r /local/proxyroot The following command-line flags are supported. * Proxy-specific options -c Do not compress files transmitted from the Perforce server to P4P -d Run as a daemon (fork first, then run) -f Run as single-threaded server -i Run for inetd (socket on stdin/stdout) -q Suppress startup messages -s Run as an NT service (or invoke as p4ps) -e size Only cache files larger than size bytes (default $P4PFSIZE or zero) * General options -h or -? Display this message -L log File to which messages are logged (default $P4LOG, or stderr if $P4LOG is not set) -p port Port on which P4P accepts Perforce client requests (default $P4PORT or 1666 if $P4PORT is not set) -r root Proxy cache directory (default $P4PCACHE or directory where p4p is started if $P4PCACHE is not set) -t port Port that P4P uses to connect to a server (default $P4TARGET or perforce:1666 if $P4PORT is not set) -v level Specifies debug mode (see Perforce System Administrator's guide for details; default $P4DEBUG or none if $P4DEBUG is not set) -V Display the version of the Perforce proxy Administering P4P * No backups required You do not need to back up the P4P cache directory. If necessary, P4P reconstructs it based on Perforce server metadata. * Loading the cache directory to obtain best initial performance P4P stores file revisions only when one of its clients requests them. File revisions are not "prefetched." Note that the performance gain that is provided by P4P only comes into effect after file revisions are cached. After starting P4P, you can load the cache directory by creating a client and syncing it to the head revisions, enabling other clients that subsequently connect to immediately obtain the performance improvements provided by P4P. * Managing disk space consumption P4P saves file revisions in its cache directory. Note that P4P's disk space consumption only increases. P4P does not delete its cache files or otherwise manage its consumption of disk space. To recover disk space, remove files under the proxy's root. It is safe to delete the proxy's cache files while the proxy is running. * Protections The p4 protect command can distinguish connections coming from a proxy if the string 'proxy-' is prepended to the IP addresses of the true client and used in the protections table. For example, 'proxy-*' applies to all connections from all proxies and 'proxy-10.0.0.5' identifies a host with an IP address of 10.0.0.5 and connecting to p4d through a proxy. * Remote Depots The proxy is designed to operate between a p4 client and a p4 server (p4d). It does not work between two p4 servers in a remote depot configuration. Attempting to connect to a remote depot via a proxy will result in the proxy logging an error about this configuration (i.e. can not proxy for remote depots) and an error at the p4 server about unable to access remote depot and an error at the client. -------------------------------------------------------------------------- The change history below indicates client/server/proxy requirements as follows: * -- requires new p4 client program ** -- requires new p4d server program *** -- requires new p4p proxy program New features in 2008.1 #149641 * ** *** TCP connections between Perforce processes; clients, proxies and servers, now discover network buffering levels and better use that buffer space. Performance should be improved especially for long proxy to server connections. #149297 * 'p4 set' will now report on the P4POPTIONS variable. (Bug #22108) #148921 ** *** Uncompress of binary files can be done on the client instead of the server when the proxy is in use. Both the new server and new proxy are required. (Bug #16398, #28490) Bugs fixed in 2008.1 #152854 ** The proxy was storing revisions which were lazy copied between different local depots in multiple locations. Fixed. (Bug #28635) #148708 *** Compression from the server to the proxy would start late and potentially most of a command may be uncompressed. Now, compression will be enabled earlier. (Bug #28575) #140557 *** Under rare conditions 'p4 -x ' could hang if run through the proxy. Among the requirements are many files (in the thousands), and a command which has heavy client interaction such as 'add' or 'diff -s?'. (Bug #27131) New features in 2007.3 #130638 *** Proxy Load functionality. Running the command: "p4 -Zproxyload sync" will update the proxy's archive cache without sending files to the client. This is intended to be used via a dedicated client whose revision have list will indicate if proxy revisions might need to be updated. (Bug #10600) #130380 ** *** Tagged output for 'p4 info' run through the proxy now shows the proxy version string. (Bug #20634) #129045 ** *** Proxy will now cache files as they are submitted through the proxy. A new cache validation system based on MD5 signatures is being used. If a new proxy is working with an old server the old timestamp validation is used. Recommend running "p4 verify" to assure that the p4d server has MD5 signatures computed as the proxy will not validate it's cache for revisions which the p4d server does not have an MD5 signature. (Bug #10866) Bugs fixed in 2007.3 #135086 *** The Windows version of the proxy could leak threads if two sessions wrote to the error log concurrently. Fixed. (Bug #26223) #129045 ** "p4 sync -p" run through the proxy would update the have list when the proxy needs to bring over a revision from the server. Fixed. (Bug #25274) #129045 ** *** The proxy can effectively cache files with a zero modtime. (Bug #23408) #129045 *** "p4 -Zproxyverbose print" will now report if the proxy cache was used to satisfy the command. (Bug #22802) New features in 2007.2 #119109 * ** *** Changes to better support the new utf16 filetype #113338 *** Changes to network buffering which improve performance of many short commands. Bugs fixed in 2006.2 #112395 *** 'p4 print' without the '-o' flag might fail with an internal error in some cases. Fixed. (Bug #23236) #104828 ** 'p4 resolve -at' after a 'p4 integ' between files where only one of the files has keyword expansion as part of its type caused problems because keyword expansion would be done based on the source file's type, but after submit, sync would expand keywords based on the target file's type leading to differences which would be reported with 'p4 diff -se'. With this fix, in this case, 'p4 resolve -at' will expand keywords based on the target file's type. The proxy was also cacheing the wrong keyword expansion which could appear in other clients later sync operations. (Bug #20584) New features in 2006.1 #95339 ** *** p4 print without the -o option can constructively use the proxy's cache now. For this to work both the server and the proxy must be upgraded. (Bug #19759) Bugs fixed since 2006.1/102189 (first release) #113894 *** 'p4 print' without the '-o' flag would cause the proxy to leak handles on NT systems. Fixed. (Bug #22915) #107500 *** The proxy would incorrectly keyword expand (or not) if a revision was lazy copied from a source and the keyword expansion type modifier was changed. Fixed. (Bug #21905) #105760 *** p4 print without the -o option would add garbage characters at the end of the file if the file was text and did not end with a newline. Fixed. (Bug #21935) New features in 2005.2 #74564 ** *** The proxy can be configured to only cache file revsions larger than a certain size set by the '-e' flag or the P4PFSIZE environment variable. You must have a p4d server at the same version level or greater for this new feature to function. (Bug #16399) Bugs fixed in 2005.2 #84680 ** The server proxy protocol was improved for long latency links to need less return data. #75261 *** Proxy options can be set with the enviroment variable P4POPTIONS. That variable can be set to a string of command line options. This allows an NT service to set proxy options which do not have other environment options. (Bug #13809) -------------------------------------------------------------------------- Bugs fixed in 2005.1 #90416 *** The previous change 70964 to fix problems with the Date and DateTime keywords was not correct. Fixed. (Bug #13481) #70975 *** Starting p4p as a service on NT would fail if upper case was used for the executable name. Fixed. (Bug #11840) #70964 ** *** The proxy would expand the Date and DateTime keywords in text+k files using the proxy's timezone not the server's resulting in p4 diff -se reporting that the files did not match the depot. Fixed if you install both the new proxy and new server. (Bug #13481) #68597 ** *** The 2003.2 proxy would crash with the 2004.2 server and vice versa. The 2004.2 server now talks an older protocol, which means that keyword expanded files will not be cached with the 2003.2 proxy talking to the 2004.2 server. The 2004.2 proxy will also no longer crash with the 2003.2 or older servers. (Bug #15387) -------------------------------------------------------------------------- Bugs Fixed in 2004.1 #50042 *** The proxy did not implement the -i flag which allows it to be started from an inetd process even though it was documented. Now implemented. (Bug #12136) -------------------------------------------------------------------------- New features in 2003.2 #46146 ** *** The compress option in client specifications now effects compression only between the proxy and the client if the proxy is in use. Previously a client with compression enabled would cause all communication to be compressed, i.e. both between the server and proxy; and between the proxy and the client. Now, compression between the server and the proxy is exclusively controlled by the proxy's -c flag. (Bug #11194) Bugs Fixed in 2003.2 #46044 ** *** Text files with keyword expansion (ktext, text+k) were not always handled properly when the proxy was in use. This is fixed. (A new p4p or p4d is required, but performance is best if both the new p4p and p4d are in use.) (Bug #11212) #44303 *** Warning message text changed when the proxy can not update its cache to indicate the proxy's cache file name. (Bug #11033) -------------------------------------------------------------------------- Bugs fixed in 2003.1 #43921 *** The proxy's RCS based cache files could be corrupted if concurrent updates to that RCS file occurred due to multiple clients requesting a revision from that file. Fixed. Related, the executable bit will no longer be set on RCS files. (Bug #10512) #43265 *** The proxy would terminate operations early if it had a problem updating its cache. Now, errors are logged and reported to the client in this case, but processing contines. In this case the client will be updated but the proxy's cache is not, and the proxy's cache should be investigated for corruption or file system overflow. (Bug #10918) #43075 *** Performance of the proxy sending many small files was poor due to not keeping network connections full. This change improves performance by better keeping data in motion. (Bug #10892) -------------------------------------------------------------------------- Bug fixed in 2002.2 #37999 *** Configuring the proxy with an identical P4PORT and P4TARGET (or -p and -t) ports would cause the server to fork (or spawn new threads) until the machine's resources are totally consumed. (Bug #9709)