cli_tests.cfg #1

# Test Data File for Command Line tests.
#
# Each test is a one-line entry of the form:
#
# <TestGroup>|<TestCmd>|<ExitCode>|<TestLog>|<ExpectedStringRegex>|<Comments>
# 
# Where:
# <TestGroup> is an arbitrary name used to apply to a group of tests, allowing that group
# of tests to be executed independently of other configured tests. By default, all
# tests are executed; call run_cli_tests.sh with '-g <TestGroup>' to indicate that the
# specified group of tests are to be run. This is intended to speed up interactive testing
# for a focused group of tests.
#
# <TestCmd> is the Perforce command to execute.  If it starts with 'p4 ', the
# 'p4 ' wll be substituted at runtime with 'p4 -p <port> -u <user> -c <client>',
# where the port/user/client values are dependint test configuration data in
# test_sdp.*.cfg  files.
#
# Alternately, the <TestCmd> can start with 'p4@port@user@client ', to execute
# commands as a user or in a workspace defined in the test entry.  The user
# is typically 'bruno' or some other user from the Perforce Sample Depot data set.
# 
# If the <TestCmd> starts with 'p4 ', a default port (1666), user (bruno), and
# client (bruno_ws) are used. If the <TestCMd> starts with 'p4@P4PORT@P4USER@P4CLIENT ',
# the command is executed with '-p P4PORT -u P4USER -c P4CLIENT '.

# If the <TestCmd> does not start with 'p4 ' or 'p4@P4PORT@P4USER@P4CLIENT ', it is
# deemed to be an arbitrary command and it is executed as is, with no '-p/-u/-c'
# options. To avoid having options in a p4 command, inject a space before the 'p4' in
# the TestCmd. For exampe, start the line as something like "YourTestGroup| p4 info";
# the space between the 'p4' and the '|' will disable the addition of the options
# '-p P4PORT -u P4USER -c P4CLIENT'.
#
# The <TestCmd> value can also be the special value 'no_cmd', meaning that no
# test command is executed. This allows additional checks of logs to be done based
# on the execution of the most recent executed test.
#
# <ExitCode> is the expected exit code, which must be a whole number (integer 0 or
# greater). If the actual test exist code is not match, the test is deemed to have
# failed.  A special value of 'U' meaning Undefined can be given rather than a
# whole number to indicate that any exit code is accepted.  (The test may still fail
# depending on the <Output>). A special value of 'N' can be given to indicate that
# any non-zero exit code is accepted.
#
# <TestLog> Specify the absolute path to the log containing the expected string,
# or the special value 'output' to scan the output of the command rather than
# a log file.
#
# <ExpectedStringRegex> is a string of text expected of the executed command in
# either the ouptutor a given logfile. The string can be a regular expression.
#
# <Comments> are short comments for the human reviewer of the test, describing what
# is expected in the output, what is being tested, etc.

# Sample: Run 'p4 -s info -s' command. Expect a zero exit code with "Server version"
# in the output:
Info|p4 -s info -s|0|output|Server version|Basic p4 info command.

# Sample: Run 'p4 -s infox -s' (which does not exist). Expect a 1 exit code with "Unknown command"
# in the output:
#
Info|p4 -s infox -s|1|output|error: Unknown command.|Broken p4 info command.

# These next two entries are sample bogus test entries.  These will abort the
# sdp_test_suite.sh test run unless '-f' is used, in which case they'll just be
# ignored:
### p4 -s info -s|BogusExitCode|/p4/1/logs/log|blah|Sample bad test data entry.
### p4x -s info -s|U|/p4/1/logs/log|Blah|Sample bad test data entry.

# This is a real test:
Basic|p4login 1 -v|0|output|^Success:|Login test.
Basic|p4 -s configure show server.depot.root|0|output|^exit: 0$|Check server.depot.root value.
Basic|p4@1666@earl@bruno_ws flush //depot/Jam/MAIN/src/Build.com|0|/p4/1/logs/log|user-flush|Check expected P4LOG writes.

# This example with 'U' (undefined) for expected the exit code and an empty value
# for the expected string would be for informational purposes only, since the
# test would never fail:
Info|p4 info|U||Sample test that should always succeed.

# Test Preflight check with root-owned state.xu file.
P4DPreflight|sudo chown root:root /p4/1/root/state.xu|0|||Test Setup: chown file be bogusly root-owned.
P4DPreflight|sudo systemctl restart p4d_1|U|||Test Setup: Start service
P4DPreflight|sleep 1|0|output||Test Setup: Sleep a bit.
P4DPreflight|no_cmd|U|/p4/1/logs/p4d_init.log|Error: /p4/1/root/state.xu is owned by root, expected perforce.|Should have refused to start service.
P4DPreflight|sudo chown perforce:perforce /p4/1/root/state.xu|0|||Test Cleanup: chown file back to normal.
P4DPreflight|sudo systemctl restart p4d_1|U|||Test Cleanup: Leaving the service running.
P4DPreflight|sleep 3|0|output||Test Cleanup: Sleep a bit to give service time to start.

# Test verify_sdp.sh 'commit_defined' and 'server_type_known' features.
VerifySDP|/p4/common/bin/verify_sdp.sh -only commit_defined -d|0|/p4/1/logs/verify_sdp.log|Verified: Exactly one.*master.1:commit-server|Test verify_sdp.sh 'commit_defined' checks.
VerifySDP|no_cmd|0|/p4/1/logs/verify_sdp.log|No server specs with Services|Test verify_sdp.sh 'commit_defined' checks, part 2.
VerifySDP|p4 --field ServerID=Bogus --field Services=standard server -o Bogus > /tmp/Bogus.server.p4s|0|output||Test Setup: Created bogus server spec file.|0|output||Test Setup: Generate bogus server spec file.
VerifySDP|p4 -s server -i < /tmp/Bogus.server.p4s|0|output||Test Setup: Loaded bogus server spec file.
VerifySDP|/p4/common/bin/verify_sdp.sh -only commit_defined -d|N|/p4/1/logs/verify_sdp.log|Error: One or more server specs with Services|Test verify_sdp.sh 'commit_defined' checks w/bogus extra standard spec. Failure expected.
VerifySDP|p4 -s server -d Bogus|0|output|Server Bogus deleted|Test Cleanup: Removed bogus server spec.
VerifySDP|/p4/common/bin/verify_sdp.sh -only server_type_known -d|0|/p4/1/logs/verify_sdp.log|Exactly one of the run_if_|Test verify_sdp.sh 'server_known_type' checks.
#### Disabling this check for now because it does not behave as intended.
#VerifySDP|cp -f /p4/1/root/server.id /p4/1/root/server.id.TheOriginal && echo fubar > /p4/1/root/server.id|0|output||Test Setup: Move P4ROOT/server.id aside.
#VerifySDP|/p4/common/bin/verify_sdp.sh -only server_type_known -d|N|/p4/1/logs/verify_sdp.log|Error: Exactly one of the run_if_|Test verify_sdp.sh 'server_known_type' checks. Failure expected.
#VerifySDP|cp -f /p4/1/root/server.id.TheOriginal /p4/1/root/server.id|0|output||Test Cleanup: Restore P4ROOT/server.id.

# Test the ccheck.sh Security Check mode.
SecurityCheck|/p4/sdp/test/bsw/set_password_for_bruno.sh|0|||Test Setup: Ensure bruno has a password.
SecurityCheck|p4 configure set server.rolechecks=0|U||Test Setup: Make sure server.rolechecks needs fixing with -fix.
SecurityCheck|p4 configure set security=3|U||Test Setup: Set security=3 so it can be fixed with -fix.
SecurityCheck|ccheck.sh -sec|1|output|Result: FAIL|Initial check fails.
SecurityCheck|no_cmd|U|/p4/1/logs/ccheck.log|is NOT SSL-enabled.|No SSL in this environment.
SecurityCheck|ccheck.sh -sec -no_ssl -fix -y|0|output|TO DO: p4 -s configure set server.rolechecks=|With -fix, see that things needing -FIX are TO DO.
SecurityCheck|no_cmd|U|/p4/1/logs/ccheck.log|For server 'any', configuration variable 'security' set to '4'|Verify security was fixed.
SecurityCheck|p4 configure set security=0|0||Test Setup: Set security=0 so it can only be fixed with -FIX.
SecurityCheck|ccheck.sh -sec -no_ssl -FIX -y|0|output|For server 'any', configuration variable 'security' set to '4'|Fix security with -FIX.
SecurityCheck|no_cmd|U|/p4/1/logs/ccheck.log|For server 'any', configuration variable 'server.rolechecks' set to '1'|Fix server.rolechecks with -FIX.
SecurityCheck|ccheck.sh -sec -no_ssl|0|output|PASS with Grade A|Final check succeeds.
SecurityCheck|p4 configure set security=0|U||Test Cleanup: Set security=0 to avoid breaking other tests.
SecurityCheck|/p4/sdp/test/bsw/fleet_login.sh|0|output|SUCCESS: Login OK on all targeted hosts.|Test Cleanup: Redo login fleet-wide after messing with auth.id in SecurityCheck test group.