Upgrading a RHEL package installation

The section describes how to upgrade a Swarm RHEL package installation to a newer release.

Upgrade Swarm

The following process attempts to minimize downtime, but a short period of downtime for Swarm users is unavoidable. There should be no downtime for your Helix Core Server. After a successful upgrade, all Swarm users are logged out.

If you are using Swarm in a production environment, we encourage you to test this upgrade process in a non-production environment first.

1. Install the main Swarm package on the server to host Swarm.

   Follow the instructions for your OS distribution, see RHEL 8 or RHEL 9 below:

   • RHEL 8 (run these commands as root):

     The full PHP and Apache upgrade process described below is only required the first time you upgrade to PHP 8.x. For future Swarm upgrades just run the Swarm upgrade steps, the ImageMagick steps if you want to install ImageMagick, and check that SELinux is working correctly.

     The upgrade process will deploy epel-release-latest-8.noarch.rpm to give Swarm access to the EPEL repository, deploy remi-release-8.rpm to give Swarm access to PHP 8.x and Apache 2.4, upgrade Swarm, stop and disable the Apache 2.2 server, copy and edit some Swarm files so they work with Apache 2.4, and finally start and enable the Apache 2.4 server.

     1. Deploy the epel-release-latest-8.noarch.rpm repository configuration package to give Swarm access to EPEL packages:

     dnf install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm

     2. Deploy the Remi repository configuration package to give Swarm access to PHP 8.x (only required the first time you upgrade to PHP 8.x):

     dnf install https://rpms.remirepo.net/enterprise/remi-release-8.rpm

     Tip

     If you don't deploy the Remi repository, you will see dependency errors when you do the next steps.

     3. Install the yum-utils package to give access to the yum-config-manager command:

     dnf install yum-utils

     4. Enable the optional channel for some dependencies:

     subscription-manager repos --enable=rhel-8-server-optional-rpms

     5. Install the Default/Single version of PHP:
        1. Enable the module stream for PHP 8.3:

        dnf module reset php

        2. Install PHP 8.3:

        dnf module install -y php:remi-8.3

        3. Run an upgrade for PHP, this will also upgrade the Swarm packages:

        dnf update

     6. If you didn't run dnf update in the previous step, run the Swarm package upgrade now:

     yum install helix-swarm helix-swarm-triggers helix-swarm-optional

     Important

     If you are upgrading from Swarm 2019.3 to Swarm 2021.1 or later, remove the Swarm PHP 7.1 files:

     yum remove $(yum list installed | grep php71 | awk '{print $1}')

     7. If you are upgrading from Apache 2.2, disable your Apache 2.2 HTTP server so that is does not automatically start:

     systemctl disable httpd

     8. If you are upgrading from Apache 2.2, stop your Apache 2.2 HTTP server:

     systemctl stop httpd

     9. If you have any special php.ini configuration options set, copy the php.ini file to the /etc/php.d/php.ini/ directory, for example:

     cp /etc/opt/rh/rh-php72/php.d/php.ini /etc/php.d/php.ini

     10. Copy the perforce-swarm-site.conf file to the /etc/httpd/conf.d/ directory if it is not already in there, for example:

     cp /opt/rh/httpd24/root/etc/httpd/conf.d/perforce-swarm-site.conf /etc/httpd/conf.d/perforce-swarm-site.conf

     11. Replace the /log/httpd24 filepath with /log/httpd/ in the perforce-swarm-site.conf file using the sed command:

     sed -i "s#/log/httpd24/#/log/httpd/#g" /etc/httpd/conf.d/perforce-swarm-site.conf

     12. Enable your Apache 2.4 HTTP server so that it automatically starts:

     systemctl enable httpd

     13. Start the Apache 2.4 HTTP server:

     systemctl start httpd

     14. Enable your Apache 2.4 HTTP server so that it automatically starts:

     systemctl enable httpd

     15. Start the Apache 2.4 HTTP server:

     systemctl start httpd

     16. Optional, ImageMagick: when ImageMagick is enabled, Swarm can display the following image formats: BMP, EPS, PSD, TGA, TIFF.
         

         1. Install ImageMagick:

         yum install ImageMagick

         2. Restart the web server to so that ImageMagick is picked up.

         systemctl restart httpd

     17. If you are upgrading from Swarm 2020.1, restart your Redis server:

     systemctl restart redis-server-swarm

     18. Check that SELinux is working correctly for your system. For instructions on configuring SELinux on RHEL, see SELinux configuration.
   • RHEL 9 (run these commands as root):
     1. Run an upgrade for PHP, this will also upgrade the Swarm packages:

     dnf update

     2. If you didn't run dnf update in the previous step, run the Swarm package upgrade now:

     yum install helix-swarm helix-swarm-triggers helix-swarm-optional

     Important

     If you are upgrading from Swarm 2019.3 to Swarm 2021.1 or later, remove the Swarm PHP 7.1 files:

     yum remove $(yum list installed | grep php71 | awk '{print $1}')

     3. Optional, ImageMagick: when ImageMagick is enabled, Swarm can display the following image formats: BMP, EPS, PSD, TGA, TIFF.
        

        1. Install ImageMagick:

        yum install ImageMagick

        2. Restart the web server to so that ImageMagick is picked up.

        systemctl restart httpd

     4. Check that SELinux is working correctly for your system. For instructions on configuring SELinux on RHEL, see SELinux configuration.
2. Swarm generally has several major updates each year, and may occasionally have a patch update between major updates. To determine whether a Swarm update is available.

Run the following command as root:

yum list updates | grep swarm

Configuring Helix Core Server event notification

Swarm needs to know about a number of Helix Core Server events to operate correctly, this can be done by using Helix Core Server Extensions or Helix Core Server Triggers. Swarm installs include the Swarm Helix Core Server extension file and trigger scripts required for Swarm to get the events it needs from your Helix Core Server.

You must install the Swarm Helix Core Server extension or update your Triggers to complete the Swarm upgrade.

Do one of the following so that Swarm is notified about events on the Helix Core Server:

• Install and configure the Swarm Helix Core Server extension, see Installing the Swarm Helix Core Server extension

• Update your Swarm Triggers, see Updating your triggers

Installing the Swarm Helix Core Server extension

Prerequisites

To install the Swarm Helix Core Server extension you need:

A compatible version of Helix Core Server for Helix Core Server Extensions:

• Linux: Helix Core Server 2021.2 and later. If you are using an earlier version of Helix Core Server, you must use triggers.

• Windows: Helix Core Server 2021.2 and later. If you are using an earlier version of Helix Core Server, you must use triggers.

You will also need:

• Helix Core Server Extensions installed and configured on your Helix Core Server.

• Your Swarm user password

• A user with super permissions to install and configure Helix Core Server Extensions.

To install the Swarm Helix Core Server extension

There are two ways to install the Swarm Helix Core Server extension:

• Run the interactive post-installation configuration script. See Install process using the interactive post-installation configuration script.

  Note

  The Swarm post-installation configuration script can be used in a few different ways. The steps below outline the most straightforward configuration using an interactive install, but you can review the options by running:

  sudo /opt/perforce/swarm/sbin/configure-swarm.sh -h

• Alternatively, use the swarm-extctl.sh script file located in /opt/perforce/swarm/sbin directory. The swarm-extctl.sh script file specifically handles the extension upgrade. See Install process using the swarm-extctl.sh script.

Install process using the interactive post-installation configuration script

1. Run the interactive post-installation configuration script to install the Swarm Helix Core Server extension:

   sudo /opt/perforce/swarm/sbin/configure-swarm.sh

   The configuration script displays a summary for your Swarm instance:

   ------------------------------------------------------------
   configure-swarm.sh: Thu Feb 17 14:20:49 PDT 2021: commencing configuration of Swarm
   Summary of arguments passed:
   Interactive?       [yes]
   Force?             [no]
   P4PORT             [(not specified, will prompt)]
   Swarm user         [(not specified, will prompt, will suggest swarm)]
   Swarm password     [(not specified, will prompt)]
   Email host         [(not specified, will prompt)]
   Use Extensions?    [(not specified, will prompt)] 
   Swarm host         [(not specified, will prompt, will suggest myhost)]
   Swarm port         [(default (80))]
   Swarm base URL     [(default (empty))]
   Create Swarm user? [yes]
   Super user         [(not specified, will prompt)]
   Super password     [(not specified, will prompt)]

2. The configuration script prompts you for your Swarm configuration details.

   Enter your Swarm user password when prompted, all of the other details are pre-filled with your existing Swarm configuration. Press the [Enter] key to accept each of them.

3. The script prompts you to use the Swarm Helix Core Server extension.

   Do you want to use Swarm's Helix Core server extension?
   Configuring Server extensions requires super user access to the Helix Server.
   If you install the Swarm server extension, do not install the Swarm triggers.
   Server extensions are supported for:
   * Linux: Helix server 19.2 and later.
   * Windows: Helix server 21.2 and later. 
   
   Use server extensions? (y/n) [n]

Type y to use Helix Core Server Extensions.

4. Sign in as a user with super permissions when prompted.

5. The script checks that Helix Core Server Extensions can be installed.

Note

The script will:

• check your Helix Core Server supports Helix Core Server Extensions

• check if Helix Core Server Extensions are installed and configured on your Helix Core Server. If they are, Swarm does not need to do anything

• check if Swarm Helix Core Server Triggers are installed on the server

If any of these checks fail, Swarm will not install the Swarm Helix Core Server extension and will report the issues on the configuration summary screen.

6. If Swarm Triggers are installed, you are prompted to remove them.

Swarm triggers are installed on this server and must be removed
before installing extensions. You can either have this script
automatically remove them, or you can quit and do it manually. 
Do you want to automatically remove the triggers? (y/n) [n]

• Recommended: Type y to get the script to automatically remove the Swarm Helix Core Server Triggers.

The Swarm Helix Core Server Triggers are not deleted immediately, the triggers to be removed are listed so you can review them before removing them.

• Type n to manually remove the Swarm Helix Core Server Triggers from the trigger spec file.

7. The script requests confirmation that it is okay to remove the triggers:

Are you sure it is okay to remove these triggers? (y/n) [n] 

• Recommended: Type y to automatically remove the triggers if the list of triggers looks correct.

Note

The script will:

• save a timestamped copy of your old trigger spec to /opt/perforce/swarm/triggers.saved.yyyymmdd-hhmmss

• save a timestamped copy of the new trigger spec to /opt/perforce/swarm/triggers.noswarm.yyyymmdd-hhmmss

• remove the Swarm Helix Core Server Triggers from the trigger spec and save it

If either of the trigger specs contain sensitive information, move them to a secure location.

• Type n if you see something wrong with the Swarm Helix Core Server Triggers marked for removal or if you want to remove the triggers from the trigger spec manually. You are offered the option of automatically opening your trigger spec file in your default text editor or opening the file manually in a text editor. Save your changes and rerun the post-installation configuration script to complete the installation of the Swarm Helix Core Serverextension.

8. The script will now install and configure the Swarm Helix Core Server extension for you. When it has completed the configuration, the configuration summary screen is displayed, for example:

...........
- installing instance configuration
Extension config swarm saved
configure-swarm.sh: Thu Feb 17 14:31:36 PDT 2021: completed configuration of Helix Swarm

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::
::  Swarm is now configured and available at:
::
::      http://myhost/
::
::  Ensure that you have configured the Swarm hostname in your
::  network's DNS, or have added an IP address-to-hostname
::  mapping to your computer's hosts configuration so that you
:: can access Swarm. 
:: 
::  You may login as the Swarm user [swarm] using the password
::  you specified.
:: 
::  Server side extensions are installed and configured
::  on your P4D server.
::
::  Documentation for optional post-install configuration, such as
::  configuring Swarm to use HTTPS, operate in a sub-folder, or on a
::  custom port, is available:
::
::  https://www.perforce.com/perforce/doc.current/manuals/swarm/setup.post.html
::
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

9. The Swarm upgrade is now complete.

Install process using the swarm-extctl.sh script

1. Navigate to the the swarm-extctl.sh script file located in /opt/perforce/swarm/sbin directory.

2. Run the following commands as root:

   1. Backup the current configuration. Ensure that the current configuration is correctly backed up before continuing.

      sudo ./swarm-extctl.sh -p <masterserverIp:port> -u <username> save

   2. Remove the existing extension from the master.

      sudo ./swarm-extctl.sh -p <masterserverIp:port> -u <username> delete

   3. Install the latest version of Swarm Helix Core Server extension located on the Swarm server on the master. The following command will check for all saved configurations and apply them during install.

      sudo ./swarm-extctl.sh -p <masterserverIp:port> -u <username> install

3. The Swarm upgrade is now complete.

Next step

Now Validate your upgrade.

Updating your triggers

1. Copy the new Swarm trigger script to your Helix Core Server machine. The trigger script is SWARM_ROOT/p4-bin/scripts/swarm-trigger.pl, and requires installation of Perl 5.08+ (use the latest available) on the Helix Core Server machine. If Swarm is using SSL, then the triggers also require the IO::Socket::SSL Perl module.

   Warning

   Do not overwrite any existing trigger script at this time. Give the script a new name, for example: swarm-trigger-new.pl.

2. Configure the Swarm trigger script by creating, in the same directory on the Helix Core Server machine, swarm-trigger.conf. It should contain:

   Note

   If you already have a swarm-trigger.conf file, no additional configuration is required.

   # SWARM_HOST (required)
   # Hostname of your Swarm instance, with leading "http://" or "https://".
   SWARM_HOST="http://my-swarm-host"
   
   # SWARM_TOKEN (required)
   # The token used when talking to Swarm to offer some security. To obtain the
   # value, log in to Swarm as a super user and select 'About Swarm' to see the
   # token value.
   SWARM_TOKEN="MY-UUID-STYLE-TOKEN"
   
   # ADMIN_USER (optional) Do not use if the Workflow feature is enabled (default)
   # For enforcing reviewed changes, optionally specify the normal Perforce user
   # with admin privileges (to read keys); if not set, will use whatever Perforce
   # user is set in environment.
   ADMIN_USER=
   
   # ADMIN_TICKET_FILE (optional) Do not use if the Workflow feature is enabled (default)
   # For enforcing reviewed changes, optionally specify the location of the
   # p4tickets file if different from the default ($HOME/.p4tickets).
   # Ensure this user is a member of a group with an 'unlimited' or very long
   # timeout; then, manually login as this user from the Perforce server machine to
   # set the ticket.
   ADMIN_TICKET_FILE=				
   										
   # VERIFY_SSL (optional)
   # If HTTPS is being used on the Swarm web server, then this controls whether
   # the SSL certificate is validated or not. By default this is set to 1, which
   # means any SSL certificates must be valid. If the web server is using a self
   # signed certificate, then this must be set to 0.
   # set the ticket.
   VERIFY_SSL=1

   Fill in the required SWARM_HOST and SWARM_TOKEN variables with the configuration from any previous Swarm trigger script, typically swarm-trigger.pl.

   Tip

   The ADMIN_USER and ADMIN_TICKET variables were used by the 'enforce triggers' in Swarm 2019.1 and earlier. They can be removed unless you are explicitly disabling workflow and using the deprecated 'enforce triggers'.

   Note

   Swarm 2015.4 and earlier: Swarm trigger script files were available as shell scripts in these earlier Swarm versions, typically swarm-trigger.sh.

   Swarm must now use a Perl trigger script file, typically swarm-trigger.pl.

3. On Linux: ensure that the script is executable:

   sudo chmod +x swarm-trigger-new.pl

4. Rename the new trigger script:

   On Linux:

   mv swarm-trigger-new.pl swarm-trigger.pl

   On Windows:

   ren swarm-trigger-new.pl swarm-trigger.pl

5. Update the triggers in your Helix Core Server.

   Warning

   • The swarm.shelvedel shelve-delete trigger line was added to Swarm in version 2018.1 and updated in version 2020.1.

     • Upgrading from Swarm 2017.4 and earlier: add the swarm.shelvedel shelve-delete trigger line to the Helix Core Server trigger table if it is not already present, see Update the Helix Core Server triggers table to run the trigger script.
     • Upgrading from Swarm 2018.x and 2019.x: replace the existing swarm.shelvedel shelve-delete trigger line in the Helix Core Server trigger table with the one supplied in the Swarm version you are upgrading to.

   • Workflow feature:

     The Workflow feature is enabled by default in Swarm 2019.2 and later. The trigger lines required when workflow is enabled are different to those required when workflow is disabled:

     • Workflow feature enabled (default):
       • Comment out the swarm.enforce.1, swarm.enforce.2, swarm.strict.1, and swarm.strict.2 trigger lines in the Helix Core Server trigger table if they are present, see Update the Helix Core Server triggers table to run the trigger script.
       • Add the swarm.enforce change-submit, swarm.strict change-content, and swarm.shelvesub shelve-submit trigger lines to the Helix Core Server trigger table if they are not already present, see Update the Helix Core Server triggers table to run the trigger script.
     • Workflow feature disabled:

       Comment out the swarm.enforce change-submit, swarm.strict change-content, and swarm.shelvesub shelve-submit trigger lines in the Helix Core Server trigger table if they are present, see Update the Helix Core Server triggers table to run the trigger script.

   1. Run the Swarm trigger script to capture (using Ctrl+C on Windows and Linux) the trigger lines that should be included in the Perforce trigger table:

      On Linux:

      ./swarm-trigger.pl -o

      On Windows:

      path/to/perl swarm-trigger.pl -o

   2. As a Perforce user with super privileges, update the Perforce trigger table by running p4 triggers command and replacing any swarm.* lines with the previously captured trigger line output (using Ctrl+V on Windows and Linux).
   Important

   If you previously customized the Swarm trigger lines, perhaps to apply various Trigger options, be sure to repeat those customizations within the updated trigger lines.

Next step

Now Validate your upgrade.

Validate your upgrade

Check that your upgraded Swarm instance is working correctly by doing the following:

1. Create a new changelist that:
   1. Contains at least one modified file
   2. Contains the #review keyword in the changelist description
2. Right click on the new changelist in P4V and click Shelve Files...
Important

Do not select Request New Swarm Review... because this method uses the API and will not fully test the Swarm Helix Core Server extension.

This is also true if you are using Swarm Helix Core Server Triggers instead of the Swarm Helix Core Server extension.

3. Check that a new Swarm review is created for the changelist.
• If a Swarm review is created, the Swarm Helix Core Server extension is working. If you are using Swarm Helix Core Server Triggers instead of the Swarm Helix Core Server extension and the review is created, the triggers are working.
• If a Swarm review is not created, see Review not created.

Swarm index upgrade

If you are upgrading from Swarm 2017.2 or earlier you should run the index upgrade, this ensures that the review activity history is displayed in the correct order on the Dashboard, and Reviews list pages.

The index upgrade process can be configured to suit your Swarm system specifications. See Upgrade index for details.

Run the upgrade as an Admin user by visiting the following URL:

http://SWARM-HOST/upgrade

Secure your Swarm installation

To make your Swarm installation more secure apply the following recommendations for HTTP and Swarm implementation through security groups.

HTTP

Here is a list of best practices to use when port 80 is exposed for HTTP traffic:

• Redirect to HTTPS: If Port 80 needs to be open to support legacy systems or specific use cases, ensure that all HTTP traffic is redirected to HTTPS to encrypt data in transit.

• Use HSTS (HTTP Strict Transport Security) headers: Implement HSTS headers to force browsers only to use secure HTTPS connections when interacting with your server.

• Close port 80: If there is no requirement to use HTTP, Port 80 must be closed entirely to prevent any unencrypted data transmission.

• Implement SSL/TLS (secure sockets layer and transport layer security) certificates: Ensure that your server is configured with a valid SSL/TLS certificate to enable secure HTTPS connections.

• Firewall configuration: Configure firewalls to block or filter access to Port 80, particularly from untrusted networks.

• Continuous monitoring and auditing: Regularly monitor network traffic and audit server configurations to ensure that unnecessary ports are not exposed and that data is transmitted securely.

When you implement HTTPS, you must make the following changes:

1. Modify your cron job for the Swarm workers.

   Edit the cron configuration file to point to your HTTPS URL, for example, https://HOSTNAME/. For more information about how to edit the cron configuration file, see Set up a recurring task to spawn workers.

   To verify if the cron configuration file points to your HTTPS URL, run the following curl statement:

   curl https://myswarm.host/queue/worker

2. Modify the Swarm Extension or Trigger configuration.

   If you are using the Swarm extension run the following command and change ExtConfig’s Swarm URL to be your new HTTPS URL:

   p4 extension --configure Perforce:helix-swarm

   If you are using triggers, edit swarm-trigger.pl configuration file and set your SWARM_HOST to be https.

3. Edit the external_url in the SWARM_ROOT/data/config.php file’s environment block to point to your HTTPS URL. This URL is used in emails, Jira links, and Swarm test’s pass-and-fail outgoing URL parameters.

   Tip

   If you make a configuration change, Swarm will not use it until the configuration cache has been reloaded, this forces Swarm to use the new configuration. You must be an admin or super user to reload the Swarm config cache. Navigate to the User id dropdown menu, select System Information, click the Cache Info tab, and click the Reload Configuration button.

4. Modify the Helix Core Server's P4.Swarm.URL property. For more information about Helix Core Server integration, see Client integration.

   If your Apache server is listening on both HTTPS and HTTP in perforce-swarm-site.conf file, you must set the auto_register_urlconfigurable in the p4 block to false and correctly configure the P4.Swarm.URL property .

   If your Apache server is listening only on HTTPS and if the auto_register_urlconfigurable in the p4 block is set to true (default value), an Apache restart will correct the property.

   To get all your current values for P4.Swarm.URL property, run:

   p4 -Ztag property -A -l -n P4.Swarm.URL

   Ensure that the P4.Swarm.URL property points to your HTTPS URL.

5. Modify the URL of all applications. Any other applications that reference the URL should be switched to using the HTTPS URL.

Swarm implementation through security groups

Here is a list of best practices for implementation using security groups or the user's preferred setup:

• Use a trusted proxy: Ensure to only use a trusted proxy, such as allow lists, Content Delivery Networks (CDN), and API Gateways.

• Backend servers and other proxies or load balancers should be disabled: Ensure that direct access to backend servers and other proxies or load balancers is disabled, except through the trusted proxy mentioned above. This will prevent unauthorized access while ensuring that all requests are filtered through the trusted proxy.

• Continuous monitoring and logging of the X-Forwarded-For header: Implement monitoring and logging on the X-Forwarded-For header to track and identify any suspicious activities. This can help in identifying and preventing potential malicious activity or security threats.

• Use a secure protocol: Implement a secure protocol such as HTTPS to encrypt the communications between the client and the load balancers, and between the load balancer and backend server to prevent eavesdropping or tampering with the X-Forwarded-For header.

• Configure X-Forwarded-For header: Configure the processing mode of the X-Forwarded-For header (append, preserve, or remove) based on specific technical or security requirements.

All done!