# SDP and HMS Update Process

[TOC]

## Introduction

These notes describe the process of upgrading the Server Deployment Package (SDP), Helm Management System (HMS), and then upgrading the P4 Server products:

* p4d
* p4broker
* p4p

The SDP can also manage p4dtg server, but upgrade for this is currently manual, and would be done following the model used for the above products with regard to naming of exes and symlink handling.

## An HMS Managed Environment

An HMS-managed environment is one in which a stream depot named `//sdp` is used to manage the local deployment of the SDP, and for which the `origin` remote spec is used to fetch updates to the SDP and HMS from The Public Depot.

In some deployments, the HMS SDP instance is on the same server machine as a customer's own data set.  In other cases, a dedicated server machine (typically a VM) named `hms` (or similar) is used. In this latter scenario, the HMS host is typically setup as a Linux bastion host, a host that can access (via SSH and P4 protocols) all server machines managed by HMS, including all machines on which p4d/p4broker/p4p/p4dtg are run.

HMS management may have been configured initially with the `hms_ts_setup.sh` (HMS "Tight Ship" setup), or manually.  In most cases, there is a separate `/p4/hms` SDP instance. In some cases, the `//sdp` depot is in the customer's primary data set.  Either way, the procedures below applies.

## News

News: In early 2019, HMS returned to being a separate package, layered on but no longer packaged with the SDP. This resulted in a change to the upgrade procedure for SDP and HMS, documented here.

## Process Overview

The SDP is upgraded first, followed by HMS, and lastly followed by other P4 Server products (the upgrade of which is managed with the SDP `upgrade.sh` script).  This is because the latest SDP will be aware of changes to the P4 upgrade procedures and account for them.

## SDP Versioning Structure

The following are streams in the `//sdp` structure:

* `//sdp/workshop_main` contains the SDP package structure, including docs, the Unix/Windows trees, Maintenance and other folders -- everything contained in the SDP tarball.  This stream is reserved for updates from Perforce Software.  This is a release stream parented by `//sdp/main`.
* `//sdp/main` is the local mainline.  It is initially seeded by a copy from `//sdp/workshop_main`.
* A set of `//sdp/deploy_<hostname>` virtual streams, one per machine, map a subset of files from `//sdp/main` into the as-deployed structure. These streams are to be mapped to workspaces with a root directory of `/p4`, to version on-disk structure such as `/p4/common` and `/p4/<instance>/bin`.
* `//hms/workshop_main` contains the HMS package structure, including docs and bootstrapping files.  A subset of these files get merged to `//sdp/main/` for deployment (details below).

## Procedure

### Part 1: Upgrading the SDP

#### STEP 1. Login and Set Context for your HMS server

```
$ ssh -l perforce perforce-01
$ export P4CONFIG=/p4/.p4config.SDP

$ p4 -s info
```

Verify that you are connected to a server, and that the Client root field does not indicate `unknown`.

#### STEP 2. Fetch SDP from The Public Depot

```
$ p4 fetch
```

This uses the remote spec named `origin`, which connects as the `ftp` user to `public.perforce.com:1666`. It fetches changes into the streams `//sdp/workshop_main` and `//hms/workshop_main`.

These two streams are never modified locally by the customer, and instead are reserved for receiving updates to the SDP and HMS from Perforce Software.

Note: In some cases, particularly those where the `//sdp` and `//hms` depots are blended into the customers data set, the remote spec used for fetching updates may be named something other than `origin`, e.g. it may be `public` or `workshop`.  In those cases, the fetch command must reference the named remote spec with the `-r` flag, e.g.:

```
p4 fetch -r public
```

#### STEP 3.  Merge SDP to Local Main

Next, bring the changes from the public SDP into the local //sdp/main stream, from which SDP and HMS files are managed and deployed.

```
$ cd /p4/hms/tmp/sdp_main
$ export P4CONFIG=.p4config
$ p4 -s sync
```

Next, be sure the workspace is clean and ready to use:

```
$ p4 opened
$ p4 status
```

If these do not come back clean, it is likely the result of prior work in progress being done in the workspace and left incomplete.  Figure out what to do to get the workspace back to clean (submitting, reverting, or even shelving changes, and possibly using (with GREAT caution) the `p4 clean` command in this workspace.

```
$ p4 -s merge -S //sdp/workshop_main

$ p4 -s resolve -as
```

That sets up the merge and attempts a simple/safe resolve.  After that, see if any additional files needed handled by doing a resolve preview:

```
$ p4 -s resolve -n
```

If additional files are listed, they must be handled individually and with care.  Typical resolution options include:

* Using `p4 resolve -am` to start merging changes to any files that were updated in the public SDP and also modified locally.
* Using `p4 resolve -at` to overwrite any local changes with the latest version from the public SDP.
* Using `p4 resolve -ay` to ignore changes from the public SDP. This may commonly be used to ignore updates to files that must be modified with HMS to capture your local topology, such as `sdp_hosts.cfg` and `P4Topology.cfg`.

If no site-local custom changes have been made, this process will be very easy, as there will not be actual merge work.  If site-local changes have been made, you'll have the power of the P4 integration engine to make complex work manageable.

When done, submit the changes in this workspace.

```
p4 -s submit -d "Merged updates from public SDP." //sdp/main/...
```

Note: If you are merging from an earlier version of the SDP that included HMS in the same package, you will see this merge deleting HMS-related files.  Don't be alarmed! You can ignore these changes with `p4 resolve -ay`, as those files will be handled in the next step.

#### STEP 4.  Merge HMS to Local Main

Now bring the changes from the public HMS into the local //sdp/main stream.  HMS is overlaid onto the SDP structure. The HMS_to_SDP branch spec maps files from `//hms/workshop_main/p4/common/...` to `//sdp/main/Server/Unix/p4/common/...`.

Continue in the same workspace as used for the previous step.

```
$ p4 -s merge -b HMS_to_SDP

$ p4 -s resolve -as
```

Then go through the rest of the resolve and submit process as with the SDP.  As before with the SDP, if any site-local custom changes were made to files updated in the public release of HMS, code merging must be done.

When done, submit the changes in this workspace.

```
p4 -s submit -d "Merged updates from public HMS." //sdp/main/...
```

#### STEP 5.  Deploy Updates

Finally, deploy the updated files to all hosts:

```
sdp_sync -i hms -a
```

This completes the update of SDP and HMS into your local mainline.