Multiple Helix Core Server instances

A single Swarm server can be connected one or more Helix Core Server (P4D) instances. This section describes how to configure Swarm to connect to multiple Helix Core Servers.

Complete the following steps to connect Swarm to multiple Helix Core Server instances:

• Set up the Swarm configuration file for the Helix Core Servers
• Set the Swarm trigger token and Swarm host variable for each Helix Core Server
• Configure a cron job for each Helix Core Server instance

Set up the Swarm configuration file for the Helix Core Servers

The Swarm Helix Core Server connection details are configured using the p4 item in the <swarm_root>/data/config.php configuration file.

When Swarm is configured for a single Helix Core Server instance the p4 item looks like this:

<?php
    return array(
        'p4' => array(
            'port'     => 'my-helix-core-server:1666',
            'user'     => 'admin_userid',
            'password' => 'admin user ticket or password',
            'sso'      => 'disabled', // ['disabled'|'optional'|'enabled'] default value is 'disabled' 
         ),
    );

Where:

• port is the P4PORT for your Helix Core Server.
• user is a userid that has admin privileges for the Helix Core Server.
• password is the ticket or password for the Helix Core Server user.
• sso if the Helix Core Server is configured for the Helix Authentication Service, set to one of the following:

  • enabled all users must use Helix Authentication Service to log in to Swarm.

  • optional Helix Authentication Service is available for users to log in to Swarm but is not enforced.

  • disabled Helix Authentication Service is not available to Swarm. This is the default value.

Important

From Swarm 2021.1, the sso_enabled configurable is deprecated but remains supported. It is replaced with the more flexible sso configurable. If the sso_enabled configurable and sso configurable are both present in the p4 configuration block, Swarm uses the sso configurable value.

Multiple Helix Core Server instances can be added to the p4 item in config.php but each Helix Core Server instance must have a label to identify the server instance. This enables Swarm to connect to the Helix Core Servers in the p4 item.

The first Helix Core Server in the p4 item is the primary Helix Core Server, this instance renders the global dashboard. If Swarm fails to connect to the primary Helix Core Server, Swarm cannot display any of the Helix Core Server instances in the global dashboard. You can still connect directly to the other P4D instances by including the Helix Core Server label in the URL for each instance, for example: https://swarm.company.com/serverA.

Configure the Swarm configuration file for multiple Helix Core Server instances:

Enter a label for each Helix Core Server instance and enter the server connection details under the server label.

For example:

<?php
    return array(
        'p4' => array(
            'serverA' => array(
                'port'     => 'p4d-A:1666',
                'user'     => 'admin_userid_serverA',
                'password' => 'admin user ticket or password for serverA',
                'sso'      => 'disabled', // ['disabled'|'optional'|'enabled'] default value is 'disabled' 
            ),
            'serverB' => array(
                'port'     => 'p4d-B:1666',
                'user'     => 'admin_userid_serverB',
                'password' => 'admin user ticket or password for serverB',
                'sso'      => 'disabled', // ['disabled'|'optional'|'enabled'] default value is 'disabled' 
            ),		
        );

Where:

• serverA and serverB are labels that identify the individual Helix Core Servers.
  • The labels must be URL-friendly labels.
  • The labels must not contain any spaces or any characters that require URL encoding, and
  • The total length of the server label and Redis namespace combined is limited to ≤ 127 characters. The default namespace is Swarm.
• The port, user, password, and sso items are specific to the Helix Core Server instance they are nested under.

Configure a specific Helix Core Server

The root element in the <swarm_root>/data/config.php configuration file contains default values for all Helix Core Server instances. You can override the default values by using a server specific configuration file. For example, to override the global value of jira configuration for a specific server, you can provide a different value for jira configuration in the server specific configuration file.

To configure a specific Helix Core Server instance, add a config.php file in the <swarm_root>/data/servers/<serverid> directory. For the new Helix Core Server configuration to take effect, the configuration cache must be deleted and reloaded. For more information on how to delete and reload cache, see Cache Info.

Configure event notification for each Helix Core Server

Swarm needs to know about a number of Helix Core Server events to operate correctly, this can be done by using Helix Core Server Extensions (recommended) or Helix Core Server Triggers. Swarm installs include the Swarm Helix Core Server extension file and trigger scripts required for Swarm to get the events it needs from your Helix Core Server.

You must install and configure the Swarm Helix Core Server extension or Triggers to complete the Swarm configuration.

Do one of the following so that Swarm is notified about events on the Helix Core Servers it is connected to:

• Install and configure the Swarm Helix Core Server extension for each Helix Core Server instance, see Install and configure the Swarm Helix Core Server extension (recommended)

• Set the Swarm trigger token and host variable for each Helix Core Server instance, see Set the Swarm trigger token and Swarm host variable for each Helix Core Server

Install and configure the Swarm Helix Core Server extension (recommended)

Prerequisites

To install the Swarm Helix Core Server extension you need:

A compatible version of Helix Core Server for Helix Core Server Extensions:

• Linux: Helix Core Server 2021.2 and later. If you are using an earlier version of Helix Core Server, you must use triggers.

• Windows: Helix Core Server 2021.2 and later. If you are using an earlier version of Helix Core Server, you must use triggers.

You will also need:

• Extensions installed and configured on your Helix Core Server.

• A user with super permissions to install and configure the Swarm Helix Core Server extension.

Swarm Helix Core Server extension installation

The Swarm Helix Core Server extension file is included for all of the Swarm installation types. The helix-swarm.p4-extension file is located in swarm_root/p4-bin/extensions

On each Helix Core Server you are connecting Swarm to, run the following commands as a Helix Core Server user with super permissions:

1. Install the Swarm Helix Core Server extension on the Helix Core Server with:

p4 extension --yes --install helix-swarm.p4-extension

Example response:

Extension 'Perforce::helix-swarm' version '2022.1.20221215' installed successfully.
Perform the following steps to turn on the Extension:

# Create a global configuration if one doesn't already exist.
p4 extension --configure Perforce::helix-swarm

# Create an instance configuration to enable the Extension.
p4 extension --configure Perforce::helix-swarm --name Perforce::helix-swarm-instanceName

For more information, visit: https://www.perforce.com/manuals/extensions/Content/Extensions/Home-extensions.html

2. Create a global configuration if one doesn't already exist:

   p4 extension --configure Perforce::helix-swarm

   The spec file opens in your text editor, for example:

   ExtName:        helix-swarm
   ExtDescription:
           Helix Swarm Extension
    
   ExtVersion:     2022.1.DEV.20220120
    
   ExtUUID:        4532BC59-7BC8-478F-ADF6-0A563C42563D
    
   ExtRev: 1
    
   ExtMaxScriptTime:       unset
    
   ExtMaxScriptMem:        unset
    
   ExtAllowedGroups:
    
   ExtEnabled:     true
    
   ExtP4USER:      sampleExtensionsUser
    
   Name:   helix-swarm
    
   Owner:  super
    
   Update: 2022/01/21 10:43:46
    
   Description:
           The description of your config.
    
   ExtConfig:
           Debug:
                   2
           Swarm-Secure:
                   true
           Swarm-Token:
                   ... SWARM-TOKEN
           Swarm-URL:
                   http://localhost/

3. Edit the spec file to match your system:

   • ExtP4USER: Set to an existing user with super permissions.

   • ExtConfig: Check and edit the values in this block:

     • Debug:

       Debug levels 0 to 3 control the amount of debug information sent to the Helix Core Server Extensions log.

       Important

       A debug level of 10 and higher sends all debug information to the client. This is useful for debugging, but should not be run in a production environment.

     • SSL-CA-File: (optional) Set the absolute path of the Certificate Authority (CA) Privacy-Enhanced Mail (PEM) file to validate the SwarmHelix Core Server's certificate. Ensure that this file is accessible to the Helix Core Server process owner.

       If a suitable certificate is not found in the local SSL certificate store maintained by the operating system or if there are verification issues from using a self-signed certificate then the default CA file is used.

       By default, the local certificate stored in the default CA file is used if the absolute path for the Certificate Authority (CA) is not specified.

     • Swarm-Secure:

       • true will only accept secure SSL certificates.

       • false will accept insecure SSL certificates.

     • Swarm-Token: Set to the Swarm trigger token value.

       Important

       To set multiple Helix Core Server token settings for Extensions, repeat the following steps for each Helix Core Server. The process to obtain an extension token of your Swarm instance is similar to obtaining a trigger token of your Swarm instance.

       To obtain the trigger token of your Swarm instance:

       1. Log in to Swarm as a super user.

       2. Click your userid, found at the right of the main toolbar.

       3. Select About Swarm.

       4. The About Swarm dialog is displayed and Swarm generates an API token if it doesn't already exist.

       5. Copy the trigger token value displayed at the bottom of the dialog and paste the token into the spec file.

     • Swarm-URL: Set to the URL of your Swarm instance.

4. When you have finished editing the spec file, save it in your text editor.

5. Create an instance configuration to enable the extension:

   p4 extension --configure Perforce::helix-swarm --name swarm

   The spec file opens in your text editor, for example:

   ExtName:        helix-swarm
    
   ExtDescription:
           Helix Swarm Extension
    
   ExtVersion:     2022.1.DEV.20220120
    
   ExtUUID:        4532BC59-7BC8-478F-ADF6-0A563C42563D
    
   ExtRev: 1
    
   ExtMaxScriptTime:       unset
   
   ExtMaxScriptMem:        unset
    
   ExtEnabled:     true
    
   ExtDebug:       none
    
   Name:   swarm
    
   Owner:  super
    
   Update: 2022/01/21 10:58:41
   
   Description:
           The description of your config.
   
   ExtConfig: 
           depot-path:
                   //...
           enableStrict:
                   true
           enableWorkflow:
                   true
           httpTimeout:
                   30
           ignoreErrors:
                   false
   

   We recommend that these settings are left at their default values, but the configuration needs to be run to set the default values.

   • enableStrict: Enable strict checking of reviews for workflow. Verifies that the file content in a commit matches the file content of its associated approved review. If one or more files in a commit do not match the content of the file in its associated review, the commit is rejected.

   • enableWorkflow: Enable workflow.

   • httpTimeout: Timeout (in seconds) when communicating with the Swarm server.

   • ignoreErrors: If the server returns an error (timeout, not there), then default to allowing a request. It means workflow rules can be skipped, but if the Swarm server is down it will not block submits.

6. Save the spec file in your text editor.

7. Confirm your Swarm Helix Core Server extension is installed and configured by running the following command on the Helix Core Server:

p4 extension --run swarm ping

• If the Swarm Helix Core Server extension is working, the Helix Core Server responds with OK

• If the Swarm Helix Core Server extension is not working, the Helix Core Server responds with an error message. For example, BAD (Cannot reach https://swarm-example.com/)

For more information about Helix Core Server Extensions, see:

• Extension overview in the Helix Core Extensions Developer Guide.

• p4 extension in the Helix Core Command-Line (P4) Reference.

8. Repeat these steps for each Helix Core Server.
9. When you have configured all of the Helix Core Servers, configure a cron job to start Swarm workers for each Helix Core Server instance, see Configure a cron job for each Helix Core Server instance.

Set the Swarm trigger token and Swarm host variable for each Helix Core Server

Helix Core Server uses a Swarm trigger token to confirm that trigger requests from Swarm are valid. Each Helix Core Server instance must have a valid Swarm trigger token in its swarm-trigger.conf file. The swarm-trigger.conf file also contains the Swarm host URL for the Helix Core Server instance.

Set the Swarm trigger token and Swarm host variable for each Helix Core Server instance:

1. Navigate to the Swarm URL for the instance.

For example: https://swarm.company.com/serverA

2. Log in to Swarm as a super user.
3. Click your userid, in the main toolbar and select About Swarm.

The About Swarm dialog is displayed with a Trigger Token. Swarm generates the trigger token if it doesn't already exist.

4. Copy the trigger token value from the dialog.
5. Open the swarm-trigger.conf file for the Helix Core Server instance.
Tip

For more information about the location of the swarm-trigger.conf file, see Set up Swarm triggers with a Windows or Linux-hosted Helix Core Server.

6. In the swarm-trigger.conf file:
   1. Set the SWARM_HOST URL.
   2. Set the SWARM_TOKEN by pasting the Swarm trigger token you copied in the steps above.

For example:

# SWARM_HOST (required)
# Hostname of your Swarm instance, with leading "http://" or "https://".
SWARM_HOST="https://swarm.company.com/serverA"

# SWARM_TOKEN (required)
# The token used when talking to Swarm to offer some security. To obtain the
# value, log in to Swarm as a super user and select 'About Swarm' to see the
# token value.
SWARM_TOKEN="TRIGGER-TOKEN-FOR-SERVERA"

7. Save the swarm-trigger.conf file.
8. Repeat these steps for each Helix Core Server.
9. When you have configured all of the Helix Core Servers, configure a cron job to start Swarm workers for each Helix Core Server instance, see Configure a cron job for each Helix Core Server instance.

Configure a cron job for each Helix Core Server instance

To automatically spawn workers for each of the servers connected to Swarm, you must manually configure a cron job for each of the servers.

Swarm package installations only

The swarm-cron-hosts.conf file specifies the connection type (HTTP or HTTPS), hostname, port number and, server label for Swarm cron jobs.

If you have installed Swarm via packages, you must specify all of the Helix Core Server URLs within /opt/perforce/etc/swarm-cron-hosts.conf.

1. Edit the swarm-cron-hosts.conf file so that it contains the actual Swarm hostname, ports, and server labels you have configured for Swarm.

The following format is used with one Helix Core Server on each line:

[http[s]://]<swarm-host>[:<port>][/<base-url>]

Default if value not specified:

• [http[s]://] http
• <swarm-host> must be specified
• [:<port>] 80
• [/<base-url>] server label, must be specified

For example, for serverA and serverB configured earlier on a Swarm host of https://swarm.company.com with a default port value of 80. The entries in the swarm-cron-hosts.conf file would be:

https://swarm.company.com/serverA
https://swarm.company.com/serverB					

2. Save the swarm-cron-hosts.conf file.
3. Swarm is now configured to connect to multiple Helix Core Server instances.
Tip

To check or modify Swarm worker configuration, see Worker configuration.

Swarm Tarball installation only

If you have installed Swarm from a tarball or configured cron manually, you need create a file called helix-swarm and add all of the Helix Core Server instances connected to Swarm.

1. Create a file named helix-swarm in /etc/cron.d if it does not already exist.
2. Edit the helix-swarm file so that it has the following content:

#
# Cron job to start Swarm workers every minute
#
* * * * * nobody [ -x /opt/perforce/swarm/p4-bin/scripts/swarm-cron.sh ] && /opt/perforce/swarm/p4-bin/scripts/swarm-cron.sh

3. Save the helix-swarm file.
4. Edit the swarm-cron-hosts.conf file so that it contains the actual Swarm hostname, ports, and server labels you have configured for Swarm.

The following format is used with one Helix Core Server on each line:

[http[s]://]<swarm-host>[:<port>][/<base-url>]

Default if value not specified:

• [http[s]://] http
• <swarm-host> must be specified
• :<port> 80
• [/<base-url>] server label, must be specified

For example, for serverA and serverB configured earlier on a Swarm host of https://swarm.company.com with a default port value of 80. The entries in the swarm-cron-hosts.conf file would be:

https://swarm.company.com/serverA
https://swarm.company.com/serverB					

5. Save the swarm-cron-hosts.conf file.
6. Swarm is now configured to connect to multiple Helix Core Server instances.
Tip

To check or modify Swarm worker configuration, see Worker configuration.

Further information

Users

• To visit the Global Dashboard, enter the basic Swarm URL without a Helix Core Server instance name, for example: https://swarm.company.com.
• To visit a specific Helix Core Server instance in Swarm without going via the global dashboard, include the server name in the URL, for example: https://swarm.company.com/serverA.

Once you are viewing the Helix Core Server in Swarm, Swarm works as a standard single Helix Core ServerSwarm system.

Administrators

On the web server hosting Swarm, Swarm automatically creates a data folder for each Helix Core Server instance in <Swarm_root>/data/servers. This is because each Helix Core Server request is a field.

For example:

# ls -la /opt/perforce/swarm/data/servers/serverA
total 362296			
drwx------ 6 apache apache    4096 Sep  8 17:15 .
drwx------ 6 apache apache    4096 Sep  6 15:41 ..
drwx------ 2 apache apache    4096 Sep 20 18:08 cache
drwx------ 3 apache apache    4096 Sep  7 13:25 clients
-rw-r--r-- 1 apache apache 3709543 Sep 26 14:19 log
-r-------- 1 apache apache      84 Sep  6 15:41 p4trust
drwx------ 4 apache apache    4096 Sep 20 14:27 queue
drwx------ 2 apache apache    4096 Sep 20 11:18 sessions

Developers

To get a list of projects via the Swarm API for serverA, run bash:

• If you are using wget:

  $ wget -u "apiuser:password" https://swarm.company.com/serverA/api/v11/projects

• If you are using curl:

  $ curl -u "apiuser:password" https://swarm.company.com/serverA/api/v11/projects