sdp_upgrade.sh.man.txt #2

USAGE for sdp_upgrade.sh version r26.1.0.BETA.33512:

   sdp_upgrade.sh [-y] [-p] [-L <log>|off] [-D]

   or

sdp_upgrade.sh -h|-man

This script must be executed from the 'sdp_upgrade' directory in the extracted
SDP tarball, which can be in one of two locations.  If the /opt/perforce/p4-sdp/sdp
directory exists, start the upgrade process like this (as the 'root' user):

  cd /opt/perforce/p4-sdp/sdp/Server/Unix/p4/common/sdp_upgrade
  ./sdp_upgrade.sh

Otherwise, start the upgrade like this, as the operating system user account
under which the p4d service runs as (e.g. 'perforce' or 'p4admin' but never
as 'root'):

  cd /mnt/p4depots/downloads/new/sdp/Server/Unix/p4/common/sdp_upgrade
  ./sdp_upgrade.sh

Running sdp_upgrade.sh without '-y' will do a dry run.  Review the output and check
for a SUCCESS indication at the end.  If you get the success message, proceedd
with:

  ./sdp_upgrade.sh -y


DESCRIPTION:

	This script upgrades Perforce P4 Server Deployment Package (SDP) from
	SDP 2020.1 to the version included in the latest SDP version, SDP
	2026.1.

	== Pre-Upgrade Planning ==

	This script will upgrade the SDP if the pre-upgrade starting SDP version
	is SDP 2020.1 or later, including any/all patches of SDP
	2020.1.

	If the current SDP version is older than 2020.1, it must first be upgraded
	to SDP 2020.1 using the SDP Legacy Upgrade Guide.  For upgrading from
	pre-20.1 versions dating back to 2007, in-place or migration-style upgrades
	can be done.  See:

	https://workshop.perforce.com/view/p4-sdp/main/doc/SDP_Legacy_Upgrades.Unix.html

	The SDP should always be upgraded to the latest version first before
	P4 binaries p4d/p4broker/p4p are upgraded using the SDP
	upgrade.sh script.

	Upgrading the SDP first ensures the version of the SDP you have
	is compatible with the latest versions of p4d/p4broker/p4p/p4, and
	will always be compatible with all supported versions of these
	P4 binaries.

	When this script is used, i.e. when the current SDP version is 2020.1
	or newer, the SDP upgrade procedure does not require downtime for any
	running Perforce P4 services, such as p4d, p4broker, or p4p.  This
	script is safe to run in environments where live p4d instances are running,
	and does not require p4d, p4broker, p4p, or any other services to be stopped
	or upgraded.  Upgrade of the SDP is cleanly separate from the upgrade the
	P4 binaries. The upgrade of the SDP can be done immediately prior to
	P4 upgrades, or many days prior.

	There can be multiple SDP instances on a given server machine. This script
	will upgrade the SDP on the machine, and thus after the upgrade all
	instances will immediately use new SDP scripts and updated instance
	configuration files, e.g. the /p4/common/config/p4_N.vars files.  However,
	all instances will continue running the same P4 binaries.  Any live
	running P4 server process on the machine are unaffected by the
	upgrade of SDP.

	This script will upgrade the SDP on a single machine. If your Perforce
	P4 topology has multiple machines, the SDP should be upgraded on all
	machines. The upgrade of SDP on multiple machines can be done in any order,
	as there is no cross-machine dependency requiring the SDP to be the same
	version.  (The order of upgrade of P4 services and binaries such as
	p4d in global topologies with replicas and edge servers does matter, but is
	outside the scope of this script).

	Planning Recap:
	1. The SDP can be upgraded without downtime when this script is used, i.e.
	when the starting SDP version is 2020.1 or later.
	2. Upgrade SDP on all machines, in any order, before upgrading p4d and other
	P4 binaries.

	== Directory Structure ==

	SDP installations exist in one of two structures, detectable by where
	the /p4/sdp and /p4/common symlinks are targeted:

	* Legacy structure: symlinks point under your depot storage mount
	(e.g. /hxdepots/sdp and /hxdepots/p4/common, or /mnt/p4depots/sdp and
	/mnt/p4depots/p4/common, depending on what your site calls that mount).
	* OS Package structure (available since SDP 2024.2, used when the
	initial install was done with install_sdp.sh): symlinks point under
	/opt/perforce/helix-sdp instead (e.g. /opt/perforce/helix-sdp/p4/sdp
	and /opt/perforce/helix-sdp/p4/common), keeping SDP's own files off of
	depot storage (which is often NFS-mounted) even though the depot data
	itself may still be on NFS.

	This script upgrades the SDP scripts/config in place on whichever
	structure you already have -- it does not migrate a machine from one
	structure to the other. A separate 'migrate between structures'
	capability (including a future /opt/perforce/p4-sdp OS package
	structure) is planned but not yet implemented; passing the
	undocumented '-pkg' option will tell you so rather than doing
	anything.

	== NFS Sharing of P4Depots ==

	In some environments, the P4Depots volume is shared across multiple server
	machines with NFS, typically mounted as /p4depots.  This script updates the
	/p4depots/p4/common and /p4depots/sdp directories, both of which are on the
	NFS mount.  Thus upgrading SDP on a single machine will effectively and
	immediately upgrade the SDP on all machines that share /p4depots from the
	same NFS-mounted storage. This is a safe and valid configuration, as
	upgrading the SDP does not affect any live running p4d servers.

	== Acquiring the SDP Package - OS Package Structure ==

	If the /opt/perforce/helix-sdp structure exists on your machine, then
	upgrade using the procedure in this section.  Otherwise see the section
	below "Acquiring the SDP Package - Classic Structure".

	Become the root user first:
	  sudo su -
	  cd /opt/perforce/helix-sdp/downloads
	  [[ -e sdp.Unix.tgz ]] && mv -f sdp.Unix.tgz sdp.Unix.$(date +'%Y-%m-%d-%H%M%S').tgz
	  curl -L -O https://workshop.perforce.com/download/p4-sdp/main/downloads/sdp.Unix.tgz
	  tar -tzf sdp.Unix.tgz 2>&1 | grep -q sdp/Version && echo OK

	If this does not display an OK message, the tarball is not valid. Investigate
	and resolve this issue before proceeding. As root:

	  cd /opt/perforce/helix-sdp
	  [[ -d backup ]] || mkdir backup
	  mv sdp backup/sdp.old.$(date +'%Y-%m-%d-%H%M%S')
	  tar -xzf downloads/sdp.Unix.tgz

	  cd /opt/perforce/helix-sdp/sdp/Server/Unix/p4/common/sdp_upgrade
	  ./sdp_upgrade.sh -man

	== Acquiring the SDP Package - Classic Structure ==

	If the /opt/perforce/helix-sdp structure exists on your machine, then
	upgrade using the procedure above in the OS Package Structure section.

	This script is part of the SDP package (tarball). It must be run from an
	extracted tarball directory.  Acquiring the SDP tarball is a manual operation.

	The SDP tarball must be extracted such that the 'sdp' directory appears as
	<P4Depots>/downloads/new/sdp, where <P4Depots> defaults to /p4depots. To
	determine the value for <P4Depots> at your site you can run the following:
   
     bash -c 'cd /p4/common; d=$(pwd -P); echo ${d%/p4/common}'
   
   On this machine, that value is: /mnt/p4depots

	Following are sample commands to acquire the latest SDP, to be executed
	as the user perforce:

	  cd /mnt/p4depots
	  [[ -d downloads ]] || mkdir downloads
	  cd downloads
	  [[ -d new ]] && mv new old.$(date +'%Y%m%d-%H%M')
	  curl -L -O https://workshop.perforce.com/download/p4-sdp/main/downloads/sdp.Unix.tgz
	  ls -l sdp.Unix.tgz
	  mkdir new
	  cd new
	  tar -xzf ../sdp.Unix.tgz

	After extracting the SDP tarball, cd to the directory where this
	sdp_upgrade.sh script resides, and execute it from there.

	  cd /mnt/p4depots/downloads/new/sdp/Server/Unix/p4/common/sdp_upgrade
	  ./sdp_upgrade.sh -man

	== Preflight Checks ==

	Prior to upgrading, preflight checks are performed to ensure the
	upgrade can be completed successfully. If the preflight checks
	fail, the upgrade will not start.

	Sample Preflight Checks:

	* The existing SDP version is verified to be SDP 2020.1+.
	* Various basic SDP structural checks are done.
	* The /p4/common/bin/p4_vars is checked to confirm it can be upgraded.
	* All /p4/common/config/p4_N.vars files are checked to confirm they can be upgraded.

	== Automated Upgrade Processing ==

	Step 1: Backup /p4/common.

	The existing <P4Depots>/p4/common structure is backed up to:

	<P4Depots>/p4/common.bak.<YYYYMMDD-hhmm>

	Step 2: Update /p4/common.

	The existing SDP /p4/common structure is updated with new
	versions of SDP files.

	Step 3: Generate the SDP Environment File.

	Regenerate the SDP general environment file,
	/p4/common/bin/p4_vars.

	The template is /p4/common/config/p4_vars.template.

	Step 4: Generate the SDP Instance Files.

	Regenerate the SDP instance environment files for all instances based on
	the new template.

	The template is /p4/common/config/instance_vars.template.

	For Steps 3 and 4, the re-generation logic will preserve current
	settings. If upgrading from SDP r20.1, any custom logic that
	exists below the '### MAKE LOCAL CHANGES HERE' tag will be
	split into separate files.  Custom logic in p4_vars will be moved
	to /p4/common/site/config/p4_vars.local. Custom logic in
	p4_N.vars files will be moved to /p4/common/site/config/p4_N.vars.local.

	Note: Despite these changes, the mechanism for loading the SDP shell
	environment remains unchanged since 2007, so it looks like:

	$ source /p4/common/bin/p4_vars N

	Changes to the right-side of assignments for specific are preserved
	for all defined SDP settings.  For p4_vars, preserved settings are:
	  - OSUSER (determined by current owner of /p4/common)
	  - KEEPLOGS
	  - KEEPCKPS
	  - KEEPJNLS
	
	For instance_vars files, preserved settings are:
	  - MAILTO
	  - MAILFROM
	  - P4USER
	  - P4MASTER_ID
	  - SSL_PREFIX
	  - P4PORTNUM
	  - P4BROKERPORTNUM
	  - P4MASTERHOST
	  - PROXY_TARGET
	  - PROXY_PORT
	  - PROXY_MON_LEVEL
	  - PROXY_V_FLAGS
	  - P4DTG_CFG
	  - SNAPSHOT_SCRIPT
	  - SDP_ALWAYS_LOGIN
	  - SDP_AUTOMATION_USERS
	  - SDP_MAX_START_DELAY_P4D
	  - SDP_MAX_START_DELAY_P4BROKER
	  - SDP_MAX_START_DELAY_P4P
	  - SDP_MAX_STOP_DELAY_P4D
	  - SDP_MAX_STOP_DELAY_P4BROKER
	  - SDP_MAX_STOP_DELAY_P4P
	  - VERIFY_SDP_SKIP_TEST_LIST
	  - The 'umask' setting.
	  - KEEPLOGS (if set)
	  - KEEPCKPS (if set)
	  - KEEPJNLS (if set)
	
	Note that the above list excludes any values that are calculated.
	
	Step 5: Remove Deprecated Files.

	Deprecated files will be purged from the SDP structure.  The list of
	files to be cleaned are listed in this file:

	/downloads/new/sdp/Server/Unix/p4/common/sdp_upgrade/deprecated_files.txt

	Paths listed in this file are relative to the '/p4' directory (or
	more accurately the SDP Install Root directory, which is always
	'/p4' except in SDP test production environments).

	Step 6: Update SDP crontabs.

	No crontab updates are required for this SDP upgrade.
	
	== Post-Upgrade Processing ==

	This script provides guidance on any post-processing steps. For some
	releases, this may include upgrades to crontabs.

OPTIONS:
-y	Specify the '-y' option to confirm that the SDP upgrade should be done.

	By default, this script operates in No-Op mode, meaning no actions
	that affect data or structures are taken.  Instead, commands that would
	be run are displayed.  This mode can be educational, showing various
	steps that will occur during an actual upgrade.

 -p	Specify '-p' to halt processing after preflight checks are complete,
	and before actual processing starts. By default, processing starts
	immediately upon successful completion of preflight checks.

 -Od	Specify '-Od' to override the rule preventing downgrades.

	WARNING: This is an advanced option intended for use by or with the
	guidance of Perforce Support or Perforce Consulting.

 -L <log>
	Specify the log file to use.  The default is /tmp/sdp_upgrade.<timestamp>.log

	The special value 'off' disables logging to a file. This cannot be
	specified if '-y' is used.

 -d	Enable debugging verbosity.

 -D	Set extreme debugging verbosity.

HELP OPTIONS:
 -h	Display short help message
 -man	Display man-style help message

FILES AND DIRECTORIES:
	Name: SDPCommon
	Path: /p4/common
	Notes: This sdp_upgrade.sh script updates files in and under this folder.

	Name: P4Depots
	Default Path: /p4depots
	Notes: The folder containing versioned files, checkpoints, and numbered
	journals, and the SDP itself. This is commonly a mount point.

	Name: DownloadsDir
	Default Path: /p4depots/downloads

	Name: SDPInstallRoot
	Path: /p4

	Name: SDPOSPackageRoot
	Path: /opt/perforce/helix-sdp
	Notes: Used instead of the legacy SDPInstallRoot layout when the SDP was
	originally installed with install_sdp.sh (available since SDP 2024.2).
	See Directory Structure above.

EXAMPLES:
	This script must be executed from 'sdp_upgrade' directory in the extracted
	SDP tarball.  Typical operation starts like this:

	cd /p4depots/downloads/new/sdp/Server/Unix/p4/common/sdp_upgrade
	./sdp_upgrade.sh -h

	All following examples assume operation from that directory.

	Example 1: Prelight check only:

	sdp_upgrade.sh -p

	Example 2: Preview mode:

	sdp_upgrade.sh

	Example 3: Live operation:

	sdp_upgrade.sh -y

LOGGING:
	This script generates a log file, ~/sdp_upgrade.<timestamp>.log
	by default. See the '-L' option above.

CUSTOM PRE- AND POST- UPGRADE AUTOMATION HOOKS:
	This script can execute custom pre- and post- upgrade scripts. This
	can be useful to incorporate site-specific elements of an SDP upgrade.

	If the file /p4/common/site/upgrade/pre-sdp_upgrade.sh exists and is
	executable, it will be executed as a pre-upgrade script. If the file
	/p4/common/site/upgrade/post-sdp_upgrade.sh exists and is executable,
	it will be executed as a post-upgrade script.
	
	Pre- and post- upgrade scripts are passed the '-y' flag to confirm
	actual processing is to be done.  Custom scripts are expected to
	operate in preview mode by default, taking no actions that affect data
	(just as this script behaves).  If this sdp_upgrade.sh script is given
	the '-y' flag, that option is passed to the custom script as well,
	indicating active processing should occur.

	Pre- and post- upgrade scripts are expected to exit with a zero exit
	code to indicate success, and non-zero to indicate failure.
	The custom pre-upgrade script is executed after standard preflight
	checks complete successfully.  Preflight checks are executed before
	actual upgrade processing commences. If a custom pre-upgrade script
	indicates a failure, the overall upgrade process aborts.

	The post-upgrade custom script is executed after the main SDP upgrade
	is successful.

	Success or failure of pre- and post- upgrade scripts is reported in
	the log.  These scripts do not require independent logging, as all
	standard and error output is captured in the log of this sdp_upgrade.sh
	script.

	TIP: Be sure to fully test custom scripts in a test environment
	before incorporating them into an upgrade on production systems.

EXIT CODES:
	An exit code of 0 indicates no errors were encountered during the
	upgrade. A non-zero exit code indicates the upgrade was aborted
	or failed.