Triggers for external file transfer

Helix Core Server can be integrated with third-party WAN acceleration software to provide extremely fast transfer of archive files in a high latency network using Helix Core Server multi-server architecture. This feature supports external archive transfer with two approaches:

Replica archive pull threads

To use external transfer with replica archive pull threads:

1. Set the following server configurables:

p4 configure set replica#pull.trigger.dir=/tmp/trigger
p4 configure set replica#lbr.replica.notransfer=1
p4 configure set lbr.autocompress=1

The pull.trigger.dir configurable specifies the location where the pull thread writes the temporary file to pass as %archiveList% to the pull-archive trigger.

Note

If a replica has the lbr.replication configurable set to cache and the user performs p4 sync or p4 print for a file that is not on the replica, the replica synchronously fetches the file directly. This is known as an "inline archive transfer". To instead force "external archive transfers" by the use of your pull-archive trigger, set lbr.replica.notransfer to 1.

To make new files of type text eligible for external archive transfers, set lbr.autocompress to 1. This change only applies to text files that are created after you set lbr.autocompress to 1.

2. Define a pull-archive trigger in the trigger table:

externalPull pull-archive pull "pull.sh %archiveList%"

to specify the trigger script that performs the archive transfers, where %archiveList% represents the name of a temporary file containing the list of files to transfer.

3. Configure replica archive pull threads with the --trigger option:

p4 configure set replica#startup.2="pull -u -i 1 --trigger --batch=10"

Note

The optional --min-size and --max-size options enable you to partition archive pull threads for files of different sizes. For example,

p4 configure set replica#startup.3="pull -u -i 1 --trigger --batch=5 --min-size=8192"

specifies that the trigger ignores small files.

The size unit is bytes, but K, M, G, and T modifiers can also be used.

Important

To have small files handled by the standard archive pull threads and larger files handled by external file transfer, configure standard archive pull threads along with archive pull threads that use external transfer. For example,

p4 configure set replica#startup.4="pull -u -i 1 --batch=1000 --min-size=1 --max-size=8K"

specifies that small file transfers occur without using external software.

Tip

For high-latency configurations, a larger --batch value might improve archive transfer speed for large numbers of small files.

Edge server submits

To use external transfer for submits from an edge server to a commit server:

  1. Set rpl.submit.nocopy=1 to disable default submit archive copy:

    p4 configure set rpl.submit.nocopy=1​

  2. Define a edge-content trigger:

    edgeTransfer edge-content //... "submit.sh %changelist% %serverroot%"

  3. If the edge-content trigger needs to write temporary files, set the pull.trigger.dir configurable for the edge server:

    p4 configure set edge#pull.trigger.dir=/tmp/edge-trigger

    and update the trigger table entry for the edge-content trigger with %triggerdir% to pass the configured temporary location to the trigger:

    edgeTransfer edge-content //... "submit.sh %changelist% %serverroot% %triggerdir%"
Tip

For sample triggers and additional details, see the Support Knowledgebase article, "External Archive Transfer using pull-archive and edge-content triggers".