ReplicaTester.py

This is a utility to check the status of replication in your Perforce environment. The –h argument provides basic usage information:

ReplicaTester.py -h
Usage: ReplicaTester.py [-h] [-C] [-l logfile] [-v] [-t type] -c configFile
    -h = show help (this message)
    -C = generate sample configuration file
    -v = verbose (debug mode)
    -l logfile = specify logfile (default is STDOUT)
    -c configFile = the XML configuration file
    -t type = type of test to perform [ping|meta|file|all]
        ping - simple p4 ping test to each replica
        pull - run p4 pull on each replica
        meta - metadata (p4 key) update
        file - distribution of file
        all - perform each of these in turn (default)

** NOTE: Tests are run in parallel and could result in increased network activity. 
Please use responsibly.

** NOTE: for file testing, a client workspace MUST be already defined. This 
script will not create one for you.

Tests

Prior to running any tests, the script attempts to do a tcp ping on the server/port to ensure that it is reachable. Unreachable servers are marked as ‘down’ and not used in subsequent tests. These servers will appear in the output with the line:

[ERROR] stg-perforce(Replica): unreachable

The following four tests can be run using the script’s –t option. If not specified, the script runs ‘all’, which is

• Ping Test (ping) – this test performs a ‘p4 ping’ on each replica server listed in the configuration file.

• Pull Test (pull) – this test runs the ‘p4 pull –lj’ command on each replica server and returns the difference between the replica’s sequence number and the master’s sequence number.

• Metadata Test (meta) – this test sets a key on the master server and then checks each replica server to see when that key gets replicated. If the key does not get replicated within the provided number of attempts, then the test bails out and reports that the key did not get replicated.

• File Test (file) – this test creates a binary file of a specified size, submits it to the master server, and then checks each replica server to see when or if the file is replicated. If the file is not seen on the replica server within the specified number of attempts, the test bails out and reports that the file did not get replicated. The filetype in perforce is set to binary+S, so only one revision of the file is kept on the server or replica (this filetype prevents this test from taking up too much disk space).

Setup

There are a few things that need to be done before running the script. First, an XML configuration file must be created. A generic configuration file can be created by passing the –C (capital C) option to the script and redirecting the output to a file. Here is a sample configuration file:

<?xml version="1.0" encoding="utf-8"?>
<config>
    <p4>/usr/local/bin/p4</p4>
    <!-- 
        test file size is specified in bytes
        1KB = 1024
        1MB = 1048576
        10MB = 10485760
    -->
    <fileSize>10485760</fileSize>
    <!-- test file name (will be created in the client workspace -->
    <depotFile>//depot/testing/replicationTestFile</depotFile>
    <!-- name for key used in metadata test -->
    <metaTestKey>REPLICATION_TEST_KEY</metaTestKey>
    <!-- admin username and password -->
    <adminUser>p4admin</adminUser>
    <adminPass>somepass</adminPass>
    <!-- client workspace (must already exist) -->
    <client>replication_test</client>
    <!-- master server entry
        - name = display name
        - timeout = tcp ping timeout (seconds)
        - text = P4PORT
    -->
    <master name="perforce2(Master)" timeout="1">perforce2:1667</master>
    <!-- list of replica servers to check -->
    <replicaList>
        <!-- replica server entry
            - name = display name
            - attempts = maximum number of attempts to check the server 
              (valid for meta and file tests)
            - text = P4PORT
        -->
        <replica name="stg-perforce(Replica)" timeout="10"
        attempts="1000">10.160.39.173:1668</replica>
    </replicaList>
</config>

Next, a client workspace must be created to run the ‘file’ test. This client must include the replication test file specified in the configuration file. It is probably a good idea to specifically map that file and no other. The root for this workspace can be in a temporary area on the machine running the script (e.g. in /tmp). Here is a sample client workspace:

Client: replication_test
Owner: p4admin
Description: Created by p4admin.
Root: /tmp/replication_test
Options: noallwrite noclobber compress unlocked nomodtime normdir
SubmitOptions: leaveunchanged
LineEnd: local
View:
  //depot/testing/replicationTestFile //replication_test/replicationTestFile

When run, the script will print output to STDOUT, unless a log file is specified (using the –l option). Here is some sample output:

ReplicaTester.py -c config.xml 
[INFO] --- pingTest start
[INFO] stg-perforce(Replica): count:10 time: .030s
[INFO] --- pingTest complete
[INFO] --- pullTest start
[INFO] stg-perforce(Replica): in sync with master
[INFO] --- pullTest complete
[INFO] --- metaTest start
[INFO] stg-perforce(Replica): key updated 0.796012 sec
[INFO] --- metaTest complete
[INFO] --- fileTest start
[INFO] stg-perforce(Replica): file updated 2.284463 sec
[INFO] --- fileTest complete