Chapter 6
Scripting Perforce:
Triggers and Daemons
There are two primary methods of scripting Perforce:
- Perforce triggers are user-written scripts that are called by a Perforce server whenever certain operations (such as changelist submission or changes to forms) are performed. If the script returns a value of 0, the operation continues; if the script returns any other value, the operation fails. Upon failure, the script's standard output (not error output) is sent to the Perforce client program as an error message.
- Daemons run at predetermined times, looking for changes to the Perforce metadata. When a daemon determines that the state of the depot has changed in some specified way, it runs other commands. For example, a daemon might look for newly submitted changelists and send email to users interested in tracking changes to those files. Perforce provides a number of tools that make writing daemons easier.
This chapter assumes that you know how to write scripts.
Triggers
Triggers can be useful in many situations. Consider the following common uses:
- To validate changelist contents beyond the mechanisms afforded by the Perforce protections table. For example, you can use a pre-submit trigger to ensure that whenever file1 is submitted in a changelist, file2 is also submitted.
- To validate file contents as part of changelist submission. For example, you can use a mid-submit trigger to ensure that, when file1 and file2 are submitted, both files refer to the same set of header files.
- To start build processes after successful changelist submission.
- To validate specifications, or to provide customized versions of Perforce specification forms. For example, you can use specification triggers to generate a customized default workspace view in the p4 client command, or to ensure that users enter a meaningful workspace description.
- To notify other users of attempts to change forms such as the user form or the job specification form, or to trigger process control tools following updates to Perforce metadata.
Warning!
|
When writing trigger scripts, Perforce commands that write data to the depot are dangerous and should be avoided. In particular, do not run the p4 submit command from within a trigger script.
|
Example:
A basic trigger.
The development group wants to ensure that whenever an .exe file is submitted to the depot,
a set of release notes for the program are submitted at the same time.
You write a trigger script that takes a changelist number as its only argument, performs a p4
opened on the changelist, parses the results to find the files included in the changelist, and
ensures that for every executable file that's been submitted, a RELNOTES file in the same
directory has also been submitted. If the changelist includes a RELNOTES file, the script
terminates with an exit status of 0; otherwise the exit status is set to 1.
After writing the script, you add it to the trigger table by editing the p4 trigger form as
follows:
Triggers: rnotes submit //depot/....exe "/usr/bin/test.pl %changelist%"
|
Whenever a file ending in .exe is submitted, this trigger is fired. If the trigger script fails, it
returns a nonzero exit status, and the user's submit fails.
The trigger table
After you have written a trigger script, create the trigger by issuing the p4 triggers command. The p4 triggers form looks like this:
Triggers: relnotes_check submit //depot/bld/... "/usr/bin/relcheck.pl %user%" verify_jobs submit //depot/... "/usr/bin/job.py %change%"
|
You must be a Perforce superuser to run p4 triggers.
Trigger table fields
Each line in the trigger table has four fields:
Field
|
Meaning
|
---|
name
|
The user-defined name of the trigger.
|
type
|
There are six trigger types. The first three trigger types (submit, content, and commit) are fired when users submit changelists, and are referred to as changelist submission triggers. The remaining trigger types (save, out, and in) are fired when users generate or modify form specifications, and are referred to as specification triggers.
- submit: Execute a changelist trigger after changelist creation, but before file transfer. Trigger may not access file contents.
- content: Execute a changelist trigger after changelist creation and file transfer, but before file commit.
- To obtain file contents, use commands such as p4 diff2, p4 files, p4 fstat, and p4 print with the revision specifier @=change, where change is the changelist number of the pending changelist as passed to the script in the %changelist% variable.
- commit: Execute a changelist trigger after changelist creation, file transfer, and changelist commit.
- save: Execute specification trigger after its contents are parsed, but before its contents are stored in the Perforce database. Trigger may not modify form specified in %formfile% variable.
- out: Execute specification trigger upon generation of form to end user. Trigger may modify form.
- in: Execute specification trigger on edited form before contents are parsed and validated by the Perforce server. Trigger may modify form.
|
path
|
For changelist submission triggers (submit, content, or commit), a file pattern in depot syntax. When a user submits a changelist that contains any files that match this file pattern, the script linked to this trigger is run. Use exclusionary mappings to prevent triggers from running on specified files.
For specification triggers (save, out, or in), the name of the type of form, such as branch, client, and so on. Triggers that fire on the p4 triggers command are ignored.
|
command
|
The command for the Perforce server to run when a matching path applies for the trigger type. Specify the command in a way that allows the Perforce server account to locate and run the command. The command must be quoted, and can take the variables specified in "Trigger script variables" on page 94 as arguments.
For submit and content triggers, changelist submission continues if the trigger script exits with 0, or fails if the script exits with a nonzero value. For commit triggers, changelist submission succeeds regardless of the trigger script's exit code, but subsequent commit triggers do not fire if the script exits with a nonzero value.
For in, out, and save triggers, the data in the specification becomes part of the Perforce database if the script exits with 0. Otherwise, the database is not updated.
|
Trigger script variables
Use the following variables in the command field to pass data to a trigger script:
Argument
|
Description
|
Available for type
|
---|
%changelist%
|
The number of the changelist being submitted. (The abbreviated form %change% is equivalent.)
|
submit, content, and commit
|
%client%
|
Triggering user's client workspace name.
|
all
|
%clienthost%
|
Hostname of the client.
|
all
|
%clientip%
|
The IP address of the client.
|
all
|
%serverhost%
|
Hostname of the Perforce server.
|
all
|
%serverip%
|
The IP address of the server.
|
all
|
%serverport%
|
The IP address and port of the Perforce server, in the format ip_address:port.
|
all
|
%serverroot%
|
The P4ROOT directory of the Perforce server.
|
all
|
%user%
|
Perforce username of the triggering user.
|
all
|
%formfile%
|
Path to temporary specification file. To modify the form from an in or out trigger, overwrite this file. The file is read-only for triggers of type save.
|
save, out, and in
|
%formname%
|
Name of form (for instance, a branch name or a changelist number).
|
save, out, and in
|
%formtype%
|
Type of form (for instance, branch, change, and so on).
|
save, out, and in
|
Triggering on changelists
To configure Perforce to run trigger scripts when users submit changelists, use changelist submission triggers: these are triggers of type submit, content, and commit.
For changelist submission triggers, the path column of each trigger line is a file pattern in depot syntax. If a changelist being submitted contains any files in this path, the trigger fires. To prevent changes to a file from firing a trigger, use an exclusionary mapping in the path.
Submit triggers
Use the submit trigger type to create triggers that fire after changelist creation, but before files are transferred to the server. Because submit triggers fire before files are transferred to the server, submit triggers cannot access file contents. Submit triggers are useful for integration with reporting tools or systems that do not require access to file contents.
Example:
The following submit trigger is an MS-DOS batch file that rejects a changelist if the
submitter has not assigned a job to the changelist. This trigger fires only on changelist
submission attempts that affect at least one file in the //depot/qa branch.
@echo off if not x%1==x goto doit echo Usage is %0[change#]
:doit p4 describe -s %1|findstr "Jobs:\n\n\t" > nul if errorlevel 1 echo No jobs found for changelist %1 p4 describe -s %1|findstr "Jobs:\n\n\t" > nul
|
To use the trigger, add the following line to your triggers table:
sample1 submit //depot/qa/... "jobcheck.bat %changelist%"
|
Every time a changelist is submitted that affects any files under //depot/qa, the
jobcheck.bat file is called. If the string "Jobs:" (followed by two newlines and a tab
character) is detected, the script assumes that a job has been attached to the changelist and
permits changelist submission to continue. Otherwise, the submit is rejected.
The second findstr command ensures that the final error level of the trigger script is the
same as the error level that determines whether to output the error message.
Content triggers
Use the content trigger type to create triggers that fire after changelist creation and file transfer, but prior to committing the submit to the database. Content triggers can access file contents by using the p4 diff2, p4 files, p4 fstat, and p4 print commands with the @=change revision specifier, where change is the number of the pending changelist as passed to the trigger script in the %changelist% variable.
Use content triggers to validate file contents as part of changelist submission, and to abort changelist submission if the validation fails.
Example:
The following content trigger is a Bourne shell script that ensures that every file in
every changelist contains a copyright notice for the current year.
The script assumes the existence of a client workspace called copychecker that includes all
of //depot/src. This workspace does not have to be synced.
#!/bin/sh
# Set target string, files to search, location of p4 executable...
TARGET="Copyright `date +%Y` Example Company" DEPOT_PATH="//depot/src/..." CHANGE=$1 P4CMD="/usr/local/bin/p4 -p 1666 -c copychecker" XIT=0 echo ""
# For each file, strip off #version and other non-filename info # Use sed to swap spaces w/"%" to obtain single arguments for "for"
for FILE in `$P4CMD files $DEPOT_PATH@=$CHANGE | \ sed -e 's/\(.*\)\#[0-9]* - .*$/\1/' -e 's/ /%/g'` do # Undo the replacement to obtain filename... FILE="`echo $FILE | sed -e 's/%/ /g'`"
# ...and use @= specifier to access file contents: # p4 print -q //depot/src/file.c@=12345 if $P4CMD print -q "$FILE@=$CHANGE" | grep "$TARGET" > /dev/null then else echo "Submit fails: '$TARGET' not found in $FILE" XIT=1 fi done
exit $XIT
|
To use the trigger, add the following line to your triggers table:
sample2 content //depot/src/... "copydate.sh %change%"
|
The trigger fires when any changelist with at least one file in //depot/src is submitted. The
corresponding DEPOT_PATH defined in the script ensures that of all the files in the triggering
changelist, only those files actually under //depot/src are checked.
Commit triggers
Use the commit trigger type to create triggers that fire after changelist creation, file transfer, and changelist commission to the database. Use commit triggers for processes that assume (or require) the successful submission of a changelist.
Example:
The following commit trigger sends emails to other users who have files open in the
submitted changelist:
#!/bin/sh # mailopens.sh - Notify users when open files are updated changelist=$1 workspace=$2 user=$3 p4 fstat @$changelist,@$changelist | while read line do # Parse out the name/value pair. name=`echo $line | sed 's/[\. ]\+\([^ ]\+\) .\+/\1/'` value=`echo $line | sed 's/[\. ]\+[^ ]\+ \(.\+\)/\1/'` if [ "$name" = "depotFile" ] then # Line is "... depotFile <depotFile>". Parse to get depotFile. depotfile=$value elif [ "`echo $name | cut -b-9`" = "otherOpen" -a \ "$name" != "otherOpen" ] then # Line is "... ... otherOpen[0-9]+ <otherUser@otherWorkspace>". # Parse to get otherUser and otherWorkspace. otheruser=`echo $value | sed 's/\(.\+\)@.\+/\1/'` otherworkspace=`echo $value | sed 's/.\+@\(.\+\)/\1/'` # Get email address of the other user from p4 user -o. othermail=`p4 user -o $otheruser | grep Email: \ | grep -v \# | cut -b8-`
# Mail other user that a file they have open has been updated mail -s "$depotfile was just submitted" $othermail <<EOM The Perforce file: $depotfile was just submitted in changelist $changelist by Perforce user $user from the $workspace workspace. You have been sent this message because you have this file open in the $otherworkspace workspace. EOM fi done exit 0
|
To use the trigger, add the following line to your triggers table:
sample3 commit //... "mailopens.sh %changelist% %client% %user%"
|
Whenever a user submits a changelist, any users with open files affected by that changelist
receive an email notification.
Triggering on specifications
To configure Perforce to run trigger scripts when users edit specifications, use specification triggers: these are triggers of type save, in, and out.
Use specification triggers to generate customized specifications for users, validate customized specifications, to notify other users of attempted changes to specification forms, and to otherwise interact with process control and management tools.
Save triggers
Save triggers are called when users send changed specifications to the server, and are called after the specification has been parsed by the server, but before the changed specification is stored in the Perforce metadata.
Example:
To prohibit certain users from modifying their client workspaces, add the users to a
group called lockedws, and use the following save trigger.
This trigger denies attempts to change client specifications for users in the lockedws group,
outputs an error message containing the user name, IP address of the user's workstation, and
the name of the client workspace on which a modification was attempted, and notifies an
administrator.
#!/bin/bash NOAUTH=lockedws USERNAME=$1 WSNAME=$2 IPADDR=$3
GROUPS=`p4 groups "$1"`
if echo "$GROUPS" | grep -qs $NOAUTH then echo "$USERNAME ($IPADDR) in $NOAUTH may not change $WSNAME" mail -s "User $1 workspace mod denial" [email protected] exit 1 else exit 0 fi
|
The save trigger fires on client specifications only, and appears in the trigger table as
follows:
sample5 save client "ws_lock.sh %user% %client% %clientip%"
|
Users whose names appear in the output of p4 groups lockedws have changes to their
client workspaces parsed by the server, and even if those changes are syntactically correct, the
attempted change to the workspace is denied and an administrator is notified of the attempt.
Out triggers
Out triggers are called whenever the Perforce Server generates a specification for display to the user. For example, the command p4 job -o fires an out trigger on the job path.
Warning!
|
Never use a Perforce command in an out trigger that fires the same out trigger, or infinite recursion will result. For example, never run p4 job -o from within an out trigger script that fires on job specifications.
|
Example:
The default Perforce client workspace view maps the entire depot //depot/... to the
user's client workspace. To prevent novice users from attempting to sync the entire depot, this
Perl script changes the default workspace view used by p4 client to map only the current
release codeline of //depot/releases/main/...
#!/usr/bin/perl # default_ws.pl - Customize the default client workspace view.
$p4 = "p4 -p localhost:1666"; $formname = $ARGV[0]; # from %formname% in trigger table $formfile = $ARGV[1]; # from %formfile% in trigger table
# Default server-generated workspace view and modified view $defaultin = "\t//depot/... //$formname/...\n"; $defaultout = "\t//depot/releases/main/... //$formname/...\n";
# Check "p4 clients" to be sure this is a new workspace. # If it's an existing workspace, exit without modifying the view. open CLIENTS, "$p4 clients |" or die "Couldn't get workspace list"; while ( <CLIENTS> ) { if ( /^Client $formname .*/ ) { exit 0; } }
# Build a modified workspace spec based on contents of %formfile% $modifiedform = ""; open FORM, $formfile or die "Trigger couldn't read form tempfile"; while ( <FORM> ) { ## Do the substitution as appropriate. if ( m:$defaultin: ) { $_ = "$defaultout"; } $modifiedform .= $_; }
# Write the modified spec back to the %formfile%, open MODFORM, ">$formfile" or die "Couldn't write form tempfile"; print MODFORM $modifiedform; exit 0;
|
The out trigger fires on client specifications only, and appears as follows:
sample3 out client "default_ws.pl %formname% %formfile%"
|
New users creating client workspaces are presented with your customized default view.
In triggers
In triggers are called when users submit specifications, and before the specification is parsed by the Perforce server.
Example:
All users authorized to edit jobs have been placed in a designated group called
jobbers. The following Python script runs p4 group -o jobbers with the -G (Python
marshaled objects) flag to determine if the user who triggered the script is in the jobbers
group.
import sys, os, marshal
# Configure for your environment tuser = "triggerman" # trigger username auth_group = "jobbers" # Perforce group authorized to edit jobs
# Get trigger input args user = sys.argv[1]
# Get authorized user list # Use global -G flag to get output as marshaled Python dictionary CMD = "p4 -G -u %s -p 1666 group -o %s" % \ (tuser, auth_group) result = {} result = marshal.load(os.popen(CMD, 'r'))
auth_users = [] for k in result.keys(): if k[:4] == 'User': # user key format: User0, User1, ... u = result[k] auth_users.append(u)
# Compare current user to authorized users. if not user in auth_users: print "\n\t>>> You don't have permission to edit jobs." print "\n\t>>> You must be a member of '%s'.\n" % auth_group sys.exit(1) else: # authorized user -- OK to create/edit jobs sys.exit(0)
|
The in trigger fires on job specifications only, and appears in the trigger table as follows:
sample3 in job "python jobgroup.py %user%"
|
If the user is in the jobbers group, the in trigger succeeds and the changed job is passed to
the Perforce server for parsing. Otherwise, an error message is displayed and changes to the
job are rejected.
Using multiple triggers
Triggers are run in the order in which they appear in the triggers table. If you have multiple triggers of the same type that fire on the same path, each is run in the order in which it appears in the triggers table. If one of these triggers fails, no further triggers are executed.
Example:
Multiple triggers on the same file:
All *.c files must pass through the scripts check1.sh, check2.sh, and check3.sh:
Triggers: check1 submit //depot/src/*.c "/usr/bin/check1.sh %change%" check2 submit //depot/src/*.c "/usr/bin/check2.sh %change%" check3 submit //depot/src/*.c "/usr/bin/check3.sh %change%"
|
If any trigger fails (for instance, check1.sh), the submit fails immediately and none of the
subsequent triggers (that is, check2.sh and check3.sh) are called. Each time a trigger
succeeds, the next matching trigger is run.
To link multiple file specifications to the same trigger (and trigger type), list the trigger multiple times in the trigger table.
Example:
Activating the same trigger for multiple filespecs:
Triggers: bugcheck submit //depot/*.c "/usr/bin/checkit.pl %change%" bugcheck submit //depot/*.h "/usr/bin/checkit.pl %change%" bugcheck submit //depot/*.cpp "/usr/bin/checkit.pl %change%"
|
In this case, the bugcheck trigger runs on the *.c files, the *.h files, and the *.cpp files.
Multiple changelist submission triggers of different types that fire on the same path fire in the following order:
- submit (fired on changelist submission, before file transmission)
- content triggers (after changelist submission and file transmission)
- commit triggers (fired any automatic changelist renumbering by the server).
Similarly, specification triggers of different types are fired in the following order
- out (form generation)
- in (changed form is transmitted to the server)
- save (validated form is ready for storage in the Perforce database).
Writing triggers to support multiple Perforce Servers
To call the same trigger script from more than one Perforce Server, use the %serverhost%, %serverip%, and %serverport% variables to make your trigger script more portable.
For instance, if you have a script that uses hardcoded port numbers and addresses...
#!/bin/sh # Usage: jobcheck.sh changelist CHANGE=$1 P4CMD="/usr/local/bin/p4 -p 192.168.0.12:1666" $P4CMD describe -s $1 | grep "Jobs fixed...\n\n\t" > /dev/null
|
...and you call it with the following line in the trigger table...
sample1 submit //depot/qa/... "jobcheck.sh %change%"
|
...you can improve portability by changing the script as follows...
#!/bin/sh # Usage: jobcheck.sh changelist server:port CHANGE=$1 P4PORT=$2 P4CMD="/usr/local/bin/p4 -p $P4PORT" $P4CMD describe -s $1 | grep "Jobs fixed...\n\n\t" > /dev/null
|
...and passing the server-specific data as an argument to the trigger script:
sample2 submit //depot/qa/... "jobcheck.sh %change% %serverport%"
|
For a complete list of variables that apply for each trigger type, see "Trigger script variables" on page 94.
Triggers and security
Warning!
|
Because triggers are spawned by the p4d process, never run p4d as root on UNIX systems.
|
Triggers and Windows
By default, the Perforce service runs under the Windows local System account.
Because Windows requires a real account name and password to access files on a network drive, if the trigger script resides on a network drive, you must configure the service to use a real userid and password to access the script.
For details, see "Installing the Perforce service on a network drive" on page 125.
Daemons
Daemons are processes that are called periodically or run continuously in the background. Daemons that use Perforce usually work by examining the server metadata as often as needed and taking action as often as necessary.
Typical daemon applications include:
- A change review daemon that wakes up every ten minutes to see if any changelists have been submitted to the production depot. If any changelists have been submitted, the daemon sends email to those users who have "subscribed" to any of the files included in those changelists. The message informs them that the files they're interested in have changed.
- A jobs daemon that generates a report at the end of each day to create a report on open jobs. It shows the number of jobs in each category, the severity each job, and more. The report is mailed to all interested users.
- A Web daemon that looks for changes to files in a particular depot subdirectory. If new file revisions are found there, they are synced to a client workspace that contains the live web pages.
Daemons can be used for almost any task that needs to occur when Perforce metadata has changed. Unlike triggers, which are used primarily for submission validation, daemons can also be used to write information (that is, submit files) to a depot.
Perforce's change review daemon
The Perforce change review daemon (p4review.py) is available from the Perforce Supporting Programs page:
The review daemon runs under Python, available at http://www.python.org/. Before running the review daemon, please be sure to read and follow the configuration instructions included in the daemon itself.
Users subscribe to files by calling p4 user, entering their email addresses in the Email: field, and entering any number of file patterns corresponding to files in which they're interested in to the Reviews: field.
User: sarahm Email: [email protected] Update: 1997/04/29 11:52:08 Access: 1997/04/29 11:52:08 FullName: Sarah MacLonnogan Reviews: //depot/doc/... //depot.../README
|
The change review daemon monitors the files were included in each newly submitted changelist and emails all users who have subscribed to any files included in a changelist, letting those users know that the file(s) in question have changed.
By including the special path //depot/jobs in the Reviews: field, users can also receive mail from the Perforce change review daemon whenever job data is updated.
The change review daemon implements the following scheme:
- p4 counter is used to read and change a variable, called a counter, in the Perforce metadata. The counter used by this daemon, review, stores the number of the latest changelist that's been reviewed.
- The Perforce depot is polled for submitted, unreviewed changelists with the p4 review -t review command.
- p4 reviews generates a list of reviewers for each of these changelists.
- The Python mail module mails the p4 describe changelist description to each reviewer.
- The first three steps are repeated every three minutes, or at some other interval configured the time of installation.
The command used in the fourth step (p4 describe) is a straightforward reporting command. The other commands (p4 review, p4 reviews, and p4 counter) are used almost exclusively by review daemons.
Creating other daemons
You can use p4review.py (see "Perforce's change review daemon" on page 103) as a starting point to create your own daemons, changing it as needed. As an example, another daemon might upload Perforce job information into an external bug tracking system after changelist submission. It would use the p4 review command with a new review counter to list new changelists, and use p4 fixes to get the list of jobs fixed by the newly submitted changelists. This information might then be fed to the external system, notifying it that certain jobs have been completed.
If you write a daemon of your own and would like to share it with other users, you can submit it into the Perforce Public Depot. For more information, go to http://www.perforce.com and follow the "Perforce Public Depot" link.
Commands used by daemons
Certain Perforce commands are used almost exclusively by review daemons.
These commands are:
Command
|
Usage
|
---|
p4 counter name [value]
|
When a value argument is not included, p4 counter returns the value of the variable name.
When a value argument appears, p4 counter sets the value of the variable name to value.
Requires at least review access to run.
WARNING: The review counters journal, job, and change are used internally by Perforce; use of any of these three names as review numbers could corrupt the Perforce database.
For Release 99.2 and above, Perforce will not let you change the values of journal, job, and change.
Counters are represented internally as signed ints. For most platforms, the largest value that can be stored in a counter is 231 - 1, or 2147483647. A server running on a 64-bit platform can store counters up to 263 - 1, or 9223372036854775807
|
p4 counters
|
List all counters and their values.
|
p4 review -c change#
|
For all changelists between change# and the latest submitted changelist, this command lists the changelists' numbers, creators, and creators' email addresses.
Requires at least review access to run.
|
p4 reviews -c change# filespec
|
Lists all users who have subscribed to review the named files or any files in the specified changelist.
|
p4 changes -m 1 -s submitted
|
Output a single line showing the changelist number of the last submitted changelist, as opposed to the highest changelist number known to the Perforce server.
|
Daemons and counters
If you're writing a change review daemon or other daemon that deals with submitted changelists, you may also wish to keep track of the changelist number of the last submitted changelist, which is the second field in the output of a p4 changes -m 1 -s submitted command.
This is not the same as the output of p4 counter change. The last changelist number known to the Perforce server (the output of p4 counter change) includes pending changelists created by users, but not yet submitted to the depot.
Scripting and buffering
Depending on your platform, the output of individual p4 commands may be fully-buffered (output flushed only after a given number of bytes generated), line-buffered (as on a tty, one line sent per linefeed), or unbuffered.
In general, stdout to a file or pipe is fully-buffered, and stdout to a tty is line-buffered. If your trigger or daemon requires line-buffering (or no buffering), you can disable buffering by supplying the -v0 debug flag to the p4 command in question.
If you're using pipes to transfer standard output from a Perforce command (with or without the -v0 flag), you may also experience buffering issues introduced by the kernel, as the -v0 flag can only unbuffer the output of the command itself.
Please send comments and questions about this manual to
[email protected].
Copyright 1999-2004 Perforce Software. All rights reserved.
Last updated: 08/19/04