Failback after failover

After a Failover with master server participation, failback restores the previous role of that master server. After a successful failback,

  • the pre-failover master server is restored to its original role as the master server

  • (optional) the pre-failover standby server is restored to its original role of standby server

Prerequisites for using the failback command

  • Both the master and the standby servers must be version 2022.1 or later.

  • The original p4 failover command must have been performed with master participation.

  • The p4 failback command must be performed with master participation.

  • When the master is an edge server, the failback command requires commit server participation.

  • The server ID for the failed-over server must not be manually altered prior to performing the Procedure for using the failback command.

  • Journals created after failover or failback must still exist in their expected location before the standby-to-be is started as a standby.

Procedure for using the failback command

  1. On the machine where the pre-failover master was running, convert the pre-failover master server to a standby.

    1. Preview the p4d -Fm command described at Failback options:
      p4d -r p4root -Fm masterServerID standbyServerID
      where
      masterServerID and standbyServerID refer to the pre-failover server IDs of the master and standby, and
      the p4root location is the P4ROOT location of the pre-failover master server.

    2. If the preview looks acceptable, include the -y option to perform the operation:
      p4d -r p4root -y -Fm masterServerID standbyServerID

    Note

    This command changes this server to a restricted standby, which allows it to ignore configuration settings that would interfere with its operation as the target for the failback command.

  2. Start the newly converted standby server and wait for its replication to catch up with the master.

  3. (Optional) Change the standby server spec to make it a mandatory standby.

  4. On the restricted standby server, restore the pre-failover master server to its former master status by running the p4 failback command.

  5. (Optional) On the machine where the pre-failover standby was running, restore the pre-failover standby to its former status of standby for the restored master.

    1. Preview the p4d -Fs command described at Failback options:
      p4d -r p4root -Fs masterServerID standbyServerID
      where
      masterServerID and standbyServerID refer to the pre-failover server IDs of the master and standby, and
      the p4root location is the P4ROOT location of the pre-failover standby.

    2. If the preview looks acceptable, include the -y option to perform the operation:
      p4d -r p4root -y -Fs masterServerID standbyServerID

    Note

    This command changes this server to a restricted standby. When this server discovers from replication that p4 failback has been successfully run, it will function as an unrestricted standby for the restored master.

  6. (Optional) Change the standby server spec to make it a mandatory standby.

If p4 failback cannot be used

Failback might still be possible even if the Prerequisites for using the failback command are not fully met.

Prerequisites if p4 failback cannot be used

Ensure that the original commit or master server is:

  • reconfigured as a standby before starting it

  • assigned the serverid of a standby server, and that this serverid is different from the serverid of the current commit or master server

Reseeding the original commit or master server

If p4 failback cannot be used, it is a best practice to reseed the original commit or master from the new commit or master before performing the Failback steps if p4 failback cannot be used. Consider the following:

If the original commit or master did not participate in the failover ... If the original commit or master participated in the failover ...
... the old commit or master's metadata might contain transactions that did not make it to the original standby at the time of failover. Those transactions could reappear at failback. To avoid this possibility, reseed the original commit or master from the new commit or master before performing the failback steps. ... reseeding might not be necessary. However, if you have any doubts about metadata integrity, the safest option is to reseed the original commit or master from the new commit or master before performing the failback steps.

Failback steps if p4 failback cannot be used

At the new commit or master server

  1. Verify that the standby is pulling from new commit or master server by issuing p4 servers -J
  2. Check the result, which might be something like:

    commit '2019/07/09 16:41:36' commit-server 40/13642 40/13642 wadL/1 1

    standby '2019/07/09 16:41:31' standby 40/10000 40/10000 wAdl/4 1

    where 10000 is lower than 13642, which indicates that the standby is not yet fully caught up with the commit or master server.

  3. Wait a moment, then reissue p4 servers -J to verify that standby is fully caught up with the commit or master server. For example:

    commit '2019/07/09 16:41:36' commit-server 40/13642 40/13642 wadL/1 1

    standby '2019/07/09 16:41:36' standby 40/13642 40/13642 wAdl/4 1

At the new standby that was the original commit or master server

  1. Issue the failover command: p4 failover
  2. Follow the steps at Failover.