RUNTIME_AND_TESTING.md #1

  • //
  • guest/
  • russell_jackson/
  • sdp/
  • Server/
  • Unix/
  • setup/
  • RUNTIME_AND_TESTING.md
  • Markdown
  • View
  • Commits
  • Open Download .zip Download (5 KB)

Python runtime and setup validation

The installer baseline is CPython 3.14.8 with P4Python 2026.1. It accepts Python 3.13.x or 3.14.x with P4Python 2025.2 or 2026.1. These pairs follow the Perforce interpreter compatibility matrix; the Python baseline is the 3.14.8 release. This is the supported configuration target, not a claim that every platform/interpreter pair has been tested in this repository.

install_sdp_python.sh is self-contained. It no longer requires the absent P4U shell libraries. It builds CPython from the official HTTPS source archive, then installs the latest patch wheel in the selected P4Python release series from the configured pip index. A compatible binary wheel is required; the installer does not guess an SDK/OpenSSL source build when no wheel exists. Perforce installation guidance describes the supported packaging routes.

Run it as the SDP OS account on a supported Linux or macOS platform after installing a C/C++ compiler, make, wget, OpenSSL and the development libraries needed by CPython, including OpenSSL and zlib. The installer verifies SSL/zlib support and that P4Python imports successfully. Authentication trigger dependencies (python-ldap and pyrad) are separate; see trigger configuration.

./install_sdp_python.sh -n                  # show the plan without changing files
./install_sdp_python.sh                    # install in /p4/common/python
./install_sdp_python.sh -P 3.14.8 -R r26.1   # legacy release spelling also accepted

-r selects the installation directory. -f permits replacing an existing directory containing bin/python3; shared system directories and symlink targets are rejected. Download and compilation complete before the old installation is removed. Installation failures can still leave a partially replaced interpreter, so schedule replacement when no SDP Python jobs are running. -w /absolute/work/path selects an executable build filesystem when the default temporary directory is mounted noexec; combine it with a later -w KEEP to preserve that build. -w KEEP retains successful build files; failed builds always retain their logs. -L selects a log and -si limits output to that log. -no_ssl is rejected because current P4Python requires SSL. The old P4API/P4Python source archives are no longer downloaded.

The setup archive handling compares GNU tar versions numerically: 1.30 and newer use the newer overwrite option. Versions such as 1.9, 1.100 and 2.0 are no longer ordered as strings.

ccheck.sh uses the connected server's release, compared numerically, for known configurable boundaries. dm.user.hideinvalid and db.journalrotate.warnthresh require 2024.1; server.startup.autorestart requires 2025.1; lbr.unloaddepot.compress requires 2025.2. Older servers log COMPATIBILITY: Skipping ; requires P4D or newer; running . These are explicit omissions of unavailable settings, not proof that an older release has the corresponding feature. Unknown names and failed queries/updates remain errors. When auth.id changes, automation users are reauthenticated only if SDP_AUTOMATION_USERS is configured; explicit p4login -automation still rejects an absent list. configure_new_server.sh stops if the configurables tool is missing or fails, before creating spec/unload depots or reporting successful setup. Authorized archive-root fixes run before depot disk-space validation. The safety check requires a successful server query and valid numeric values for every returned volume, uses the smallest available value, and fails if a requested threshold cannot safely be applied; a preview reports deferred validation when its planned root change has not been applied.

Offline regression coverage is in test/test_setup_improvements.py. It tests release selection, a fully mocked download/build/install, download failure preserving an existing interpreter, numeric tar comparisons, all 365 dates in the edge cron schedule, and disposable-test guards. It does not compile CPython or install a real P4Python wheel. A successful mock suite does not establish real platform support. Live SDP validation requires the explicitly isolated environment described in disposable testing.

The container matrix executes the real installer as the SDP OS account into /opt/sdp-python/baseline, then uses that installed CPython/P4Python runtime for its primary script tests. A separate distro environment runs additional comparisons. Targets are Ubuntu 26.04 LTS and RHEL UBI 8.10, 9.8 and 10.2; UBI supplies RHEL userspace rather than a full RHEL host/kernel test. Versions, installer source hash, build account, logs and per-runtime results are recorded. See container testing; consult the collected results for the combinations that actually completed successfully.

# Python runtime and setup validation

The installer baseline is **CPython 3.14.8 with P4Python 2026.1**. It accepts Python 3.13.x or 3.14.x with P4Python 2025.2 or 2026.1. These pairs follow the [Perforce interpreter compatibility matrix](https://help.perforce.com/helix-core/apis/p4python/current/Content/P4Python/python.requirements.html); the Python baseline is the [3.14.8 release](https://www.python.org/downloads/release/python-3148/). This is the supported configuration target, not a claim that every platform/interpreter pair has been tested in this repository.

`install_sdp_python.sh` is self-contained. It no longer requires the absent P4U shell libraries. It builds CPython from the official HTTPS source archive, then installs the latest patch wheel in the selected P4Python release series from the configured pip index. A compatible binary wheel is required; the installer does not guess an SDK/OpenSSL source build when no wheel exists. [Perforce installation guidance](https://help.perforce.com/helix-core/apis/p4python/current/Content/P4Python/python.installation.html) describes the supported packaging routes.

Run it as the SDP OS account on a supported Linux or macOS platform after installing a C/C++ compiler, make, wget, OpenSSL and the development libraries needed by CPython, including OpenSSL and zlib. The installer verifies SSL/zlib support and that P4Python imports successfully. Authentication trigger dependencies (`python-ldap` and `pyrad`) are separate; see [trigger configuration](../p4/common/bin/triggers/AUTHENTICATION.md).

```bash
./install_sdp_python.sh -n                  # show the plan without changing files
./install_sdp_python.sh                    # install in /p4/common/python
./install_sdp_python.sh -P 3.14.8 -R r26.1   # legacy release spelling also accepted
```

`-r` selects the installation directory. `-f` permits replacing an existing directory containing `bin/python3`; shared system directories and symlink targets are rejected. Download and compilation complete before the old installation is removed. Installation failures can still leave a partially replaced interpreter, so schedule replacement when no SDP Python jobs are running. `-w /absolute/work/path` selects an executable build filesystem when the default temporary directory is mounted `noexec`; combine it with a later `-w KEEP` to preserve that build. `-w KEEP` retains successful build files; failed builds always retain their logs. `-L` selects a log and `-si` limits output to that log. `-no_ssl` is rejected because current P4Python requires SSL. The old P4API/P4Python source archives are no longer downloaded.

The setup archive handling compares GNU tar versions numerically: 1.30 and newer use the newer overwrite option. Versions such as 1.9, 1.100 and 2.0 are no longer ordered as strings.

`ccheck.sh` uses the connected server's release, compared numerically, for known configurable boundaries. `dm.user.hideinvalid` and `db.journalrotate.warnthresh` require [2024.1](https://help.perforce.com/helix-core/server-apps/cmdref/current/Content/CmdRef/whats-new-2024-1.html); `server.startup.autorestart` requires [2025.1](https://help.perforce.com/helix-core/server-apps/cmdref/current/Content/CmdRef/whats-new-2025-1.html); `lbr.unloaddepot.compress` requires [2025.2](https://workshop.perforce.com/files/guest/perforce_software/sdp/dev/Server/setup/configure_new_server.sh?v=76). Older servers log `COMPATIBILITY: Skipping <name>; requires P4D <release> or newer; running <version>.` These are explicit omissions of unavailable settings, not proof that an older release has the corresponding feature. Unknown names and failed queries/updates remain errors. When `auth.id` changes, automation users are reauthenticated only if `SDP_AUTOMATION_USERS` is configured; explicit `p4login -automation` still rejects an absent list. `configure_new_server.sh` stops if the configurables tool is missing or fails, before creating spec/unload depots or reporting successful setup. Authorized archive-root fixes run before depot disk-space validation. The safety check requires a successful server query and valid numeric values for every returned volume, uses the smallest available value, and fails if a requested threshold cannot safely be applied; a preview reports deferred validation when its planned root change has not been applied.

Offline regression coverage is in `test/test_setup_improvements.py`. It tests release selection, a fully mocked download/build/install, download failure preserving an existing interpreter, numeric tar comparisons, all 365 dates in the edge cron schedule, and disposable-test guards. It does not compile CPython or install a real P4Python wheel. A successful mock suite does not establish real platform support. Live SDP validation requires the explicitly isolated environment described in [disposable testing](../../../test/DISPOSABLE_TESTING.md).

The container matrix executes the real installer as the SDP OS account into `/opt/sdp-python/baseline`, then uses that installed CPython/P4Python runtime for its primary script tests. A separate distro environment runs additional comparisons. Targets are Ubuntu 26.04 LTS and RHEL UBI 8.10, 9.8 and 10.2; UBI supplies RHEL userspace rather than a full RHEL host/kernel test. Versions, installer source hash, build account, logs and per-runtime results are recorded. See [container testing](../../../test/containers/README.md); consult the collected results for the combinations that actually completed successfully.
# Change User Description Committed
#1 34004 Russell C. Jackson (Rusty) Improve SDP backup/recovery, Maintenance, logging and cloud operation.

Add owned checkpoint locks, recovery-aware retention, preserved database switches,
bounded cleanup and coordinated AWS/Azure snapshots. Fix service-user password
initialization, edge staging, standby promotion state parsing, proxy cache setup,
typed depot verification and historical-server configuration/error handling.
Remove unused OTP programs; keep Windows removed, server.locks cleanup and the
journal-count condition unchanged. Add no global shell strict modes.

Update operational documentation and add reproducible mkdirs.sh/mkdirs.cfg
container assurance. Validate Ubuntu 26.04 and UBI 8.10/9.8/10.2 across both Python
runtimes: 44 stages; 428 offline tests per runtime/OS, zero offline skips;
19 deployed-archive comparisons per combination. Three six-node topology runs
pass all 30 recovery/failure scenarios. P4D 2023.1 through 2026.1 on UBI 8/9
pass all 14 cells and 98 scenarios. Static, documentation and provenance checks
pass. Containers do not certify every fleet kernel, patch build or integration.

Tested source manifest SHA-256:
e1a23e17b8ced8c1148d9a250589977c6f6e98606bf2a81dc3ff5d7c909e3695
Deployed comparison archive SHA-256:
1bd23217356ee3808e040d39aea5be9519bda7c8f2b4924656a70aff8ad7293a