Setting Up SAP HANA System Replication

Starting with SAP HANA 2.0, additional configuration steps are required to set up SAP HANA system replication, because replication connections now use certificate-based authentication.

System replication with SAP HANA 2.0 requires authentication for the data and log shipping channels. The authentication is done using the certificates in the system PKI SSFS store. An additional manual setup step is required to exchange certificates in the system PKI SSFS store between primary and secondary sites. For more information, see SAP Note: 2369981.

For more information, see SAP Note 2369981 - Required configuration steps for authentication with HANA System Replication.

The copied files become active during system restart. Therefore, it is recommended to copy the files when the secondary SAP HANA system is offline, for example, before registration.

There are two ways to set up SAP HANA system replication in the SAP HANA cockpit:

• Enable the primary system and then register the secondary system from the primary system in one configuration step.

• Enable system replication on the primary system and then register the secondary system in a second step.

The steps to configure the primary and the secondary system using SAP HANA cockpit are outlined in the figure, Enable System Replication.

You have enabled system replication and registered the secondary system with the primary system. The secondary system operates in recovery mode. All secondary system services constantly communicate with their primary counterparts, replicate and persist data and logs, and load data to memory. However, the secondary system does not accept SQL connections.

To set up SAP HANA system replication between two identical SAP HANA systems, you must first enable system replication on the primary system and then register the secondary system.

It is also possible to configure SAP HANA system replication with the command line tool hdbnsutil as <sid>adm at the OS level. The command line tool can be a part of a script, which executes further steps beyond system replication.

1. Create a data backup of the primary system.

2. Enable the primary system and give the primary system a logical name:

   hdbnsutil -sr_enable --name=PRIMARY

3. Stop the secondary system:

   sapcontrol –nr <instance_number> -function StopSystem HDB

4. Register the secondary system (choose replication mode and operation mode):

   Code Snippet

   Copy codeSwitch to dark mode

   12345
   hdbnsutil -sr_register --remoteHost=<primary hostname> 
   --remoteInstance=<instance number> 
   --replicationMode=<sync|syncmem|async> 
   --operationMode=<delta_datashipping|logreplay> 
   --name=SECONDARY

5. Start the secondary system to start replication:

   sapcontrol –nr <instance_number> -function StartSystem HDB

Once the secondary system is started, the replication process starts automatically.

When activated, the full sync option for SAP HANA system replication ensures that a log buffer is shipped to the secondary system before a commit takes place on the local primary system.

The full sync option can be enabled for SYNC replication (that is, not for SYNCMEM). With the full sync option activated, transaction processing occurs on the primary blocks. If the secondary system is not currently connected, the newly created log buffers cannot be shipped to the secondary site. This behavior ensures that no transaction can be locally committed without shipping the log buffers to the secondary site. The full sync option can be switched on and off using the command: hdbnsutil -sr_fullsync --enable|--disable

This command changes the setting of the enable_full_sync parameter in the system_replication section of the global.ini file accordingly. However, in a running system, full sync does not become active immediately. This is done to prevent the system from blocking transactions immediately when setting the parameter to true. Instead, full sync has to first be enabled by the administrator. In a second step, it is internally activated when the secondary is connected and becomes ACTIVE.

In the M_SERVICE_REPLICATION system view, the setting of the full sync option can be viewed in using SQL.

The full sync option can have the following values:

• DISABLED: Full sync is not configured at all. The parameter is enable_full_sync = false in the system_replication section of the global.ini file.

• ENABLED: Full sync is configured, but it is not yet active, so transactions do not block in this state. To become active, the secondary has to connect and REPLICATION_STATUS must be ACTIVE.

• ACTIVE: Full sync mode is configured and active. If the network connection to a connected secondary is closed, transactions on the primary side block in this state.

If full sync is enabled when an active secondary is currently connected, FULL_SYNC is immediately set to ACTIVE.

SAP HANA system replication supports a number of compression methods for log and data shipping.

The following types of compression for log and data shipping are supported:

Log

Log buffer tail compression (by default)

Log buffer content compression

Data

Data page compression

Log buffer tail compression is turned on by default. All log buffers are aligned to 4 KB boundaries by a filler entry. With log buffer tail compression, the filler entry is cut off from the buffer before sending it over the network and added again when the buffer has reached the secondary site. So only the net buffer size is transferred to the secondary site.

The size of the filler entry is less than 4 KB. This is the maximum size reduction per sent log buffer. If the log buffers size is quite large, the compression ratio is quite limited.

Log buffer and page content compression can be activated by parameter settings.

Log buffers and data pages shipped to the secondary site can be compressed using a lossless compression algorithm (LZ4). By default, content compression is turned off. You can turn it on by setting the following configuration parameters on the secondary site in the system_replication section of the global.ini file.

Log and data compression is especially useful when system replication is used over long distances, for example, using the ASYNC replication mode.

The open source compression algorithm LZ4 has been selected because of its speed and compression ratios, and the relatively low time overhead introduced for compression/decompression. Log buffer content compression also works in combination with log buffer tail compression. Therefore, only the content part of the log buffer is compressed, without considering the filler entry.

The activation of the compression reduces the required network bandwidth, but at the same time there is some CPU overhead for compressing and decompressing the information. Using compression is particularly useful in the case of long distances between primary and secondary sites or in the case of bandwidth limitations.

To monitor SAP HANA system replication, you can use the System Replication tile in the SAP HANA Cockpit.

If system replication is configured, the System Replication tile provides information about the type of landscape (2-tier or 3-tier), the replication mode between the primary and the tier-2 secondary, the operation mode, and the overall replication status.

The System Replication tile displays the following states at a glance:

• Not configured (meaning system replication is not configured)

• All services are active and in sync

• All services are active, but not yet in sync

• Errors in replication

To check the status of replication in detail, choose the System Replication tile. The System Replication overview screen displays a graphical representation of the system replication landscape, configuration, and status. At the top, the "chain" of systems with their replication modes is shown, containing further information about the sites and the network connections between them.

The System Replication screen provides the following information:

• The name and role of the system, as well as the selected operation mode.

  For the operation modes logreplay and logreplay_readaccess, a retention time estimation is also displayed. This is an estimation of the time left before the primary system starts to overwrite the RetainedFree marked log segments, and a full data shipping becomes necessary to get the primary and secondary systems back in sync after a disconnect situation. The estimated log full time is an estimation of the time left before the primary system runs into a log full. The value shown in the header shows the situation into which the system could run first: log retention or log full.

• If the SQL ports of the secondary system are open for read access.

• The replication mode used between the systems.

• The current average redo log shipping time and the average size of shipped redo log buffers.

  This describes how long it took on average to send redo log buffers to the secondary site, based on measurements over the last 24 hours.

In addition, detailed information on system replication is provided in the tabs shown in the figure, Details for the Status of a Specific Service.

Selecting one row in the Replicated Services tab shows the details for the corresponding service grouped thematically, as in the following example for the index server. Because this information is context-sensitive, you only see the information required for this system. Therefore, because this example system is running in the logreplay operation mode, no information on delta data shipping is shown here. However, the context-sensitive information about the log replay delay is displayed. The delta between Last Log Position and Replayed Log Position indicates how far the log replay is behind on the secondary.

The SAP HANA Cockpit distinguishes between a primary and a secondary system. On the SAP HANA Cockpit of the secondary system, the System Replication tile provides an initial overview of this site’s state. From the System Replication Overview, you can initiate a takeover.

The Python scripts are located in the directory $DIR_INSTANCE/exe/python_support.

You can also gather information about the overall status of the sites and the system replication using Python scripts.

The landcapeHostConfiguration.py script shows the status of the primary system:

• SAP HANA is OK.

• SAP HANA will be OK after a host auto-failover, for example.

• Not enough instances are started and a takeover would be useful.

The script provides an overall status and a return code to match the overall host status.

A takeover is only recommended when the return code from the script is 1 (error).

1234567
<sid>adm># python $DIR_INSTANCE/exe/python_support/landscapeHostConfiguration.py
| Host  | Host   | Host   |    ...    | NameServer | NameServer  | ...
|       | Active | Status |           | Config Role| Actual Role |
| ----- | ------ | ------ | --------- | ---------- | ----------- | ------
| host1 | yes    | ok     |    ...    | master 1   | master      | ...
| host2 | yes    | ok     |    ...    | master 2   | slave       | ...
overall host status: ok

The following host states are possible:

• OK: System is OK.

• WARNING: A host auto-failover to a standby host is taking place.

• INFORMATION: The landscape is completely functional, but the current (actual) role of the host differs from the configured role.

• ERROR: There are not enough active hosts.

The systemReplicationStatus.py script shows the status of system replication.

Using systemReplicationStatus.py has the advantage of showing whether the secondary systems are in sync or not. This provides more confidence if a takeover is justified because if system replication was never in sync or is outdated, unexpected loss of data might occur.

Example:

123456789101112131415161718
h10adm@wdflbmt7346:/> python $DIR_INSTANCE/exe/python_support/systemReplicationStatus.py
| Database | Host        | Service Name | Site Name   | Secondary   | Secondary     | Replication |
|          |             |              |             | Host        | Site Name     | Status      |
| -------- | ----------- | ------------ | ----------- | ----------- | ------------- | ----------- |
| SYSTEMDB | wdflbmt7346 | nameserver   | PrimarySite | wdflbmt7347 | SecondarySite | ACTIVE      |
| H10      | wdflbmt7346 | xsengine     | PrimarySite | wdflbmt7347 | SecondarySite | ACTIVE      |
| H10      | wdflbmt7346 | indexserver  | PrimarySite | wdflbmt7347 | SecondarySite | ACTIVE      |

status system replication site "2": ACTIVE
overall system replication status: ACTIVE

Local System Replication State
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

mode: PRIMARY
site id: 1
site name: PrimarySite

The additional parameter systemReplicationStatus.py --localhost restricts the execution of the python script to the host on which is it executed.

The script provides the following return codes:

• 10: No System Replication

• 11: Error

• 12: Unknown

• 13: Initializing

• 14: Syncing

• 15: Active

You can also get system replication-specific information directly from system views.

Specific alerts are issued by the primary system to warn you of potential problems.

The Connection Closed and Configuration Parameter Mismatch alerts are raised when a system replication connection is closed, or when there is a system replication configuration parameter mismatch.

The Logreplay Backlog alert is raised when the system replication logreplay backlog is increased. In this case, logreplay is delayed on the secondary site, causing a longer takeover time.

To identify the reason for the increased system replication logreplay backlog, check the state of the services on the secondary system. To get more information, monitor the secondary site. Possible causes for the increased system replication logreplay backlog can be, for example, a slow or non-functioning log replay, or a non-running service on the secondary system.

The Increased Log Shipping Backlog alert is raised when the system replication log shipping backlog is increased. In this case, the log shipping to the secondary system is delayed or does not work properly, causing data loss on the secondary system in the case where a takeover is executed.

To identify the reason for the increased system replication log shipping backlog, check the status of the secondary system. Possible causes for the increased system replication log shipping backlog can be a slow network performance, connection problems, or other internal issues (for example, in the sync or syncmem replication modes).

Database parameters should be the same in the primary and secondary systems and are checked automatically. The configuration parameter checker reports on any differences between primary, secondary, and tier 3 secondary systems. In such a case, the parameter checker generates an alert.

With parameter replication activated, any changes made on the primary are automatically replicated to the secondary sites. Without this parameter replication activated, changes should be manually duplicated on the other system.

Parameter replication is off by default. It can be enabled and disabled on the primary site by using the following parameter:

[inifile_checker]/replicate = true | false

The parameter checker is on by default. It can be enabled and disabled on the primary site by using the following parameter:

[inifile_checker]/enable = true | false

Some parameters may have different settings on the primary and the secondary sites on purpose. One example is the global_allocation_limit parameter, where the secondary is used for other systems. By adding these parameters to the exclusion list, you can exclude them from checking.