Using alternative file transfer from submit and pull in distributed environment.
Requirements:
submit from an edge server uses alternative file transfer from edge to
commit server using an edge-content trigger.
pull from an edge server uses alternative file transfer using
pull-archive triggers.
Setup for submit.
p4 configure set lbr.autocompress=1
p4 configure set rpl.submit.nocopy=1
p4 configure set pull.trigger.dir=<location to write temp files>
A user must be set up with the public keys and anything else needed
to perform the copy. In the case of using aspera, this would require
ssl public keys or the path to a key, and a license or you set the
Aspera passord in the environment.
edge-content trigger: this will invoke the copy commands using a
list of files generated from its 'fstat -Ob' command.
For the fstat command, it needs a user with admin privilege.
For the copy command, it needs the user, target host, target path,
source path. The changelist must be passed as an argument to the
trigger, as well as any of the other above-mentioned parameters
if they are not hard-coded in the trigger script.
The trigger variable %triggerdir% has the value of pull.trigger.dir.
How this works:
The edge-content trigger invokes 'p4 fstat -Ob @=<shelvedChange>'
in order to get a list of files to transfer. In addition, it filters
out any lazy copies since those don't require transfer.
It will invoke the copy command (in this case ascp) to transfer the
files from the shelved change from the edge server to the commit server.
If a file transfer failure is detected from the trigger, the error returned
will cause the submit to fail.
Upon success the submit continues. When rmtSubmitShelf is run, it
will Stat() the files on the commit server to verify that the transfer
occurred instead of performing the transfer.
If any files are not found, an error is returned from submit.
Setup for pull.
p4 configure set lbr.autocompress=1
p4 configure set pull.trigger.dir=<location to write temp files>
p4 configure set lbr.replica.notransfer=1
pull-archive trigger: This will invoke the copy command by reading the list of
files from the file whose filename was passed as a parameter to the trigger
script. For the copy command, it needs the user, target host, target path,
source path.
The trigger variable %triggerdir% has the value of pull.trigger.dir.
The trigger variable %archiveList% has the name of the file containing
the files to be copied.
How this works:
There may be several startup.* configurables using 'pull --trigger'.
When file content should be pulled, each pull will try
to grab a batch of work. Then it goes through the list of files to
be pulled, creates a temp file and writes the files to be transfered
to the temp file. If any min-size or max-size parameters are specified,
the pull will use them to filter out files not meeting the constraint.
It then invokes the pull-archive trigger with the file containing its
file list. The pull-archive trigger invokes the copy command
(in this case ascp) to transfer the files from the commit to the edge.
Transfer errors are reported to the log file when rpl=4. Files that
failed to transfer will be detected and transferred by regular pull
threads (not --trigger). Files are checked for transfer error by
using Stat() unless the trigger script reported an error. If the
trigger reported an error, then the checksums are compared to
determine if transfer failed.
When lbr.replica.notransfer is configured, sync commands accessing files
being pulled by the pull-archive triggers will wait for the pull transfer
to finish rather than initiating another fetch mechanism.