3. aku#

Introducing aku#

Altibase Kubernetes Utility (AKU) is a utility that helps you perform tasks such as synchronizing data in Altibase with the start and termination of Pods or resetting synchronization information when scaling in a Statefulset in Kubernetes. Aku supports data replication among Pods but does not support Altibase's data scale-out feature.

StatefulSets are one of Kubernetes' workloads for supporting stateful applications like databases, and scaling means creating or terminating pods. A Pod is a resource in Kubernetes that contains containers, and Altibase server runs on these containers.

When scaling up or down on a StatefulSet, you can use aku to start or terminate pods that meet the following conditions. You should add the command in the appropriate location so that aku runs on the Altibase container.

When scaling up#

You can use aku, if you want to create a Pods with the same data as Altibase server in an existing Pod.

The pod generated at first is called a "Master pod", and the pod generated by further scaling up is called a "Slave pod".

When scaling down#

You can use aku, if you want to initialize the replication information of Altibase server when Pods are terminated.

Component#

⚠️ Aku should be running with a same version in all Pods, and the aku configuration files should have same values. Basically, aku and aku configuration file are included in the Altibase container image.

Altibase Server and aku should be executed within the same container.

aku#

aku is located in $ALTIBASE_HOME/bin. To execute aku, you need to set $ALTIBASE_HOME and add $ALTIBASE_HOME/bin to $PATH.

aku.conf#

aku.conf is the configuration file for aku. When executing aku, it reads aku.conf file to obtain the information needed for Altibase data synchronization. Altibase provides aku.conf.sample in $ALTIBASE_HOME/conf directory. Before running aku, generate the aku.conf file in the same directory using aku.conf.sample.

The aku.conf.sample file is as follows:

⚠️ Line after # symbol is commented out.

# aku.conf.sample

#***************************************************************** 
# Copyright 2022, Altibase Corporation or its subsidiaries.
# All rights reserved.
# Property File for Altibase AKU Utility
#*****************************************************************

#=================================================================
# Kubernetes setting Properties
#=================================================================
AKU_STS_NAME                  = altibase-sts    # statefulsets name
AKU_SVC_NAME                  = altibase-svc    # services name
AKU_SERVER_COUNT              = 4               # the number of Pods
#=================================================================
# Common Properties
#=================================================================
AKU_SYS_PASSWORD              = manager
AKU_PORT_NO                   = 20300
AKU_REPLICATION_PORT_NO       = 20301
AKU_QUERY_TIMEOUT             = 3600
AKU_QUERY_RETRY_COUNT         = 5
AKU_QUERY_RETRY_DELAY_MSEC    = 1000
#=================================================================
# aku start/end Properties
#=================================================================
AKU_ADDRESS_CHECK_COUNT       = 30
AKU_FLUSH_AT_START            = 1
AKU_FLUSH_TIMEOUT_AT_START    = 300
AKU_DELAY_START_COMPLETE_TIME = 0

AKU_FLUSH_AT_END              = 1
AKU_REPLICATION_RESET_AT_END  = 1
#=================================================================
# Replication Properties
#=================================================================
REPLICATIONS = (
    REPLICATION_NAME_PREFIX = AKU_REP
    SYNC_PARALLEL_COUNT     = 1
    (
        SYS.T1, SYS.T2 PARTITION P1,
        SYS.T3
    )
)

For details of each properties, refer to aku-properties.

Setup to run aku#

Prerequisite#

To ensure stable usage of aku in a Kubernetes environment, the following conditions must be met:

It should be used only in StatefulSets among Kubernetes workload.
The Pod management policy should be OrderedReady. OrderedReady is the default policy for StatefulSets.
The maximum number of scalable replicas is up to 6.
Altibase server and aku should be executed within the same container.
aku -p start command should be performed after Altibase server has started successfully.
aku may not work normally when you execute aku -p start on multiple pods at the same time. To prevent this, it is required to configure the Startup Probe in Kubernetes.
Startup Probe configuration is needed to verify if aku -p start command has been successfully executed. You can use the presence of the aku_start_completed file in the /tmp directory as an indicator for verification.
Set publishNotReadyAddress to true.
aku -p end command should be performed before stopping the Altibase server.
A Pod should be terminated after aku -p end command completes successfully.
Kubernetes's terminationGracePeriodSeconds should be set to a large value to allow aku to complete its tasks successfully before Pod is terminated.

Altibase Environment Variable and Properties#

Altibase Environment
$ALTIBASE_HOME
- Sets the directory in which Altibase server was installed. This must be set to use aku.
Altibase Properties
set ADMIN_MODE to 1.
set REMOTE_SYSDBA_ENABLE to 1.

aku Properties#

Category	Property name	Default value	Description
Kubernetes Setting Properties	AKU_STS_NAME		The name of the StatefulSet defined in the Kubernetes object specification. It can be set up to 63 bytes.
	AKU_SVC_NAME		The service name that provides the Network Service defined in the Kubernetes object specification. It can be set up to 63 bytes.
	AKU_SERVER_COUNT	4	The maximum number of Altibase servers that can be synchronized using aku. It also refers to the number of Pods that can be scaled up in Kubernetes. It can be set from 1 to 6.
Common Properties	AKU_SYS_PASSWORD		Database SYS user password
	AKU_PORT_NO	20300	Altibase Server Port number. It can be set from 1024 to 65535.
	AKU_REPLICATION_PORT_NO	20301	Altibase Replication Port number. It can be set from 1024 to 65535.
	AKU_QUERY_TIMEOUT	3600	It refers to the Altibase server property QUERY_TIMEOUT. If the execution time of SQL statements executed by aku exceeds AKU_QUERY_TIMEOUT value, the statement is canceled.
	AKU_QUERY_RETRY_COUNT	5	If a query executed on the Altibase server fails, it retries the specified number of times with this value. If this value is set to 0, it does not retry.
	AKU_QUERY_RETRY_DELAY_MSEC	1000 (ms)	If a query executed on the Altibase server fails, it retries after waiting the specified time with this value. . If this value is set to 0, it does not wait and retries.
aku start/end Properties	AKU_ADDRESS_CHECK_COUNT	30	The number of attempts to connect to the local IP for checking if the DNS address of the currently created Pods is registered in the Kubernetes service (indicating whether communication between internal Pods is possible) when running `aku -p start`.
	AKU_FLUSH_AT_START	1	This property determines whether replication gaps should be removed or not, during the execution of the `aku -p start` command. If this value is set to 1, replication gaps are removed by using FLUSH command. If this value is set to 0, replication gaps will not be removed.
	AKU_FLUSH_TIMEOUT_AT_START	300	This property sets the wait_time for the FLUSH WAIT command. If this value is set to 0, it performs FLUSH. If this value is set to 1 or greater, it performs FLUSH WAIT with the specified wait_time. This setting is valid only if AKU_FLUSH_AT_START is set to 1.
	AKU_DELAY_START_COMPLETE_TIME	0(sec)	This property specifies a waiting time (in seconds) during `aku -p start` process on the slave pod, specifically after the data synchronization is completed internally and before changing the Altibase property ADMIN_MODE to 0.
	AKU_FLUSH_AT_END	1	This property determines whether replication gaps should be removed or not, during the execution of the `aku -p end` command on the Slave Pod. If this value is set to 1, replication gaps are removed by using FLUSH ALL command. If this value is set to 0, replication gaps will not be removed.
	AKU_REPLICATION_RESET_AT_END	1	This property determines whether to perform a reset of the replication information or not by using RESET command. If this value is set to 1, the replication information is reset. If this value is set to 0, the replication information is not reset.
Replication Properties	REPLICATIONS/REPLICATION_NAME_PREFIX		This property specifies the prefix of the replication object's name that is created by aku. The maximum value is 37 bytes. Please refer to the Naming rule of the replication object in aku.
	REPLICATIONS/SYNC_PARALLEL_COUNT	1	The number of threads for sending/receiving during replication sync. It can be set from 1 to 100.

Naming rule of the replication object in aku"

The naming rule of the replication object in aku is as follow:

REPLICATION_NAME_PREFIX_[Pod Number][Pod Number]

REPLICATION_NAME_PREFIX: The strings specified as REPLICATIONS/REPLICATION_NAME_PREFIX property.
Pod number: The number located after pod_name_ in the names of pods created by the Kubernetes StatefulSet. Each pod has a unique sequential number assigned to it. In the names of Altibase replication objects, the numbers of the two pods that form the replication pair are included, and the smaller number appears first.

Example: The names of the replication objects created in each Pod when AKU_SERVER_COUNT = 4 and REPLICATION_NAME_PREFIX = "AKU_REP"

Pod Number	Replication object name	Description
pod_name-0	AKU_REP_01	Replication object name between pod_name-0 and pod_name-1
	AKU_REP_02	Replication object name between pod_name-0 and pod_name-2
	AKU_REP_03	Replication object name between pod_name-0 and pod_name-3
pod_name-1	AKU_REP_01	Replication object name between pod_name-0 and pod_name-1
	AKU_REP_12	Replication object name between pod_name-1 and pod_name-2
	AKU_REP_13	Replication object name between pod_name-1 and pod_name-3
pod_name-2	AKU_REP_02	Replication object name between pod_name-0 and pod_name-2
	AKU_REP_12	Replication object name between pod_name-1 and pod_name-2
	AKU_REP_23	Replication object name between pod_name-2 and pod_name-3
pod_name-3	AKU_REP_03	Replication object name between pod_name-0 and pod_name-3
	AKU_REP_13	Replication object name between pod_name-1 and pod_name-3
	AKU_REP_23	Replication object name between pod_name-2 and pod_name-3

⚠️ Do not create/drop/modify the Altibase replication objects created by aku.

How to configure Altibase Replication Targets#

Users can specify tables or partitions to be managed by the Altibase replication objects created by aku in the aku configuration file. To specify a replication target, add the target information in the format [user name].[table name] or [user name].[table name] PARTITION [partition name] to the replication property section. The maximum length for the user name, table name, and partition name is 128 bytes.

Each replication target is separated by commas, and one or more replication target can be written on a single line.

Here's an example of configuring tables T1 to T9 to be split and managed across three replications. For table T2, only partition P1 is set as the replication target.

#=================================================================
# Replication Properties
#=================================================================
REPLICATIONS = (
    REPLICATION_NAME_PREFIX = AKU_REP1
    SYNC_PARALLEL_COUNT     = 1
    (
        SYS.T1, SYS.T2 PARTITION P1,
        SYS.T3
    )
),
(
    REPLICATION_NAME_PREFIX = AKU_REP2
    SYNC_PARALLEL_COUNT     = 1
    (
        SYS.T4, SYS.T5, SYS.T6
    )
),
(
    REPLICATION_NAME_PREFIX = AKU_REP3
    SYNC_PARALLEL_COUNT     = 1
    (
        SYS.T7,
        SYS.T8,
        SYS.T9
    )
)

Usage of aku#

Syntax#

aku
[-h]
[--help]
[-v]
[--version]
[-i]
[--info]
[-p [start | end | clean] ]
[--pod [start | end | clean] ]

Parameters#

-h, --help#

Displays the usage of aku

-v, --version#

Displays the version information of aku. It is recommended to use the same version of aku as Altibase server.

-i, --info#

Displays the following informations defined in aku.conf file.

Altibase Server Information
Server ID
Host
User
Password
Port
Replication Port
Max Server Count : Maximum number of scalable Pods
Replication Information
Replication object's name
Replication Items
Replication Target table and User Name

-p, --pod {pod_action}#

Specify the action to be performed with aku. pod_action options are "start", "end", and "clean".

aku -p start#

It creates Altibase replication objects and synchronizes data. You can use the command when starting Pods.

The following shows the detailed behavior of aku -p start command.

⚠️ aku -p start command should be performed after Altibase server has started successfully.

Creation of Master Pod (pod_name-0)#

Since Altibase replication objects need to be created on all Pods, you should execute aku -p start command even if creating pod_name-0 in a StatefulSet.

The following explains the detailed behavior of aku -p start command while creating pod_name-0 in a StatefulSet.

① Read "aku.conf" file.

② Verify if the aku_started_completed file exists in /tmp directory. In the usual case, if the aku -p start command has not been executed previously, this file does not exist. If it exists, an error message is displayed and aku terminates the process because it is considered a duplicate execution of the aku -p start command.

③ Connect to all pods that are the target servers for replication. Typically, as this is an initially created pod, connection attempts to other pods fail. This is the expected behavior.

④ Create the Altibase replication objects. If a replication object with the same name already exists, the replication creation step will be skipped.

⑤ Start replication associated with all pods that successfully connect from pod_name-0, and start replication associated with pod_name-0 from other connected pods. Typically, as this is the initially created pod, there are no connected pods, so this action is not performed.

⑥ Create a file named "aku_start_completed" in /tmp directory.

Scale up#

When scaling up in a StatefulSet, a Slave Pod is created. A Pod can be created and terminated repeatedly. The process of aku -p start execution is a little different when a Pod is first created and when it is restarted after being terminated.

When creating a Slave Pod for the first time, or restart (Default behavior, AKU_REPLICATION_RESET_AT_END = 1)#

The following explains the detailed behavior of aku -p start command on pod_name-1 when the Slave Pod is created for the first time and is restarted normally.

① Read "aku.conf" file.

② Verify if the aku_started_completed file exists in /tmp directory. In the usual case, if the aku -p start command has not been executed previously, this file does not exist. If it exists, an error message is displayed and aku terminates the process because it is considered a duplicate execution of the aku -p start command.

③ Connect to all pods that are the target servers for replication. Typically, only the connection with pod_name-0 succeeds, and the connection to pod_name-2 or pod_name-3 fails because those pods are not created yet.

④ Create the Altibase replication objects. If pod_name-1 is restarted pod, the replication object with the same name may exist and this step will be skipped.

⑤ Execute 'TRUNCATE table' on the replication target table in pod_name-1.

⑥ Request pod_name-0(Master Pod) to perform replication synchronization.

⑦ Perform replication synchronization from pod_name-0 to pod_name-1 and start the replication.

⑧ Start replication associated with all pods that successfully connect from pod_name-1, and start replication associated with pod_name-1 from other connected pods. Typically, only AKU_REP_01, the replication related to pod_name-0, starts on pod_name-0 and pod_name-1.

⑨ Set the Altibase server property ADMIN_MODE to 0 on pod_name-1 to allow to access for database user.

⑩ Create a file named "aku_start_completed" in /tmp directory.

Case of restarting a Slave Pod that has not reset the replication information (Default behavior, AKU_FLUSH_AT_START = 1)#

The following explanation describes the basic behavior of aku when executing aku -p start on a Slave Pod that has not reset the replication information.

If a Pod is abnormally terminated or terminated with the AKU_REPLICATION_RESET_AT_END property sets to 0, the replication information is not initialized. If the replication information were not initialized, when restarting Pods the previous replication information would exist, so the below data synchronization actions from other nodes are skipped.

Executes 'TRUNCATE table' on the replication target table.
Requests for performing replication synchronization.

In this case, if the AKU_FLUSH_AT_START property sets to 1, data synchronization from itself to other nodes will be performed.

Note that if the replication information is not reset and remains, the XSN of the replication object has a value other than -1. Refer to the Cautions:If the Pod Terminates Before the aku -p end Command Completes.

① Read "aku.conf" file.

② Verify if the aku_started_completed file exists in /tmp directory. In the usual case, if the aku -p start command has not been executed previously, this file does not exist. If it exists, an error message is displayed and aku terminates the process because it is considered a duplicate execution of the aku -p start command.

③ Connect to all pods that are the target servers for replication. Typically, only the connection with pod_name-0 succeeds.

④ Start replication associated with all pods that successfully connect from pod_name-1, and start replication associated with pod_name-1 from other connected pods. Typically, only AKU_REP_01, the replication related to pod_name-0, starts on pod_name-0 and pod_name-1.

⑤ Execute ALTER REPLICATION ~ FLUSH ALL command to the replications associated with all pods that successfully connect from pod_name-1. This command sends all unsynchronized data to other pods from pod_name-1.

⑥ On other connected pods, execute ALTER REPLICATION ~ FLUSH ALL command to replications related to pod_name-1 to send unsynchronized data. If AKU_FLUSH_TIMEOUT_AT_START is not set to 0, execute ALTER_REPLICATION ~ FLUSH WAIT wait_time.

⑦ Set the Altibase server property ADMIN_MODE to 0 on pod_name-1 to allow to access for database user.

⑧ Create a file named "aku_start_completed" in /tmp directory.

Case of restarting a Slave Pod that hasn't reset the replication information (AKU_FLUSH_AT_START = 0)#

The following explanation describes the behavior of aku when executing aku -p start on a Slave Pod that has not reset the replication information, with the property AKU_FLUSH_AT_START set to 0.

① Read "aku.conf" file.

② Verify if the aku_started_completed file exists in /tmp directory. In the usual case, if the aku -p start command has not been executed previously, this file does not exist. If it exists, an error message is displayed and aku terminates the process because it is considered a duplicate execution of the aku -p start command.

③ Connect to all pods that are the target servers for replication. Typically, only the connection with pod_name-0 succeeds.

④ Start replication associated with all pods that successfully connect from pod_name-1, and start replication associated with pod_name-1 from other connected pods. Typically, only AKU_REP_01, the replication related to pod_name-0, starts on pod_name-0 and pod_name-1.

⑤ Set the Altibase server property ADMIN_MODE to 0 on pod_name-1 to allow to access for database user.

⑥ Create a file named "aku_start_completed" in /tmp directory.

aku -p end#

The command should be used when terminating Pods. It performs to stop Altibase replication and reset the replication information.

① Read "aku.conf" file.

② Connect to all Pods, which are connected with the current Pod. Since Pods are terminated sequentially, connection errors can occur when attempting to connect to already deleted Pods. This is the expected behavior.

③ Send the replication change logs to the replication objects of current Pod by executing 'ALTER REPLICATION replication_name FLUSH ALL'. If the AKU_FLUSH_TIMEOUT_AT_START property is set to 0, this step is skipped.

④ Request to perform "ALTER REPLICATION replication_name STOP" on all Pods related to current Pod.

⑤ Request to perform "ALTER REPLICATION replication_name RESET" on all Pods related to current Pod. If the AKU_REPLICATION_RESET_AT_END property is set to 0, this step is skipped.

⚠️ aku -p end command should be performed before stopping the Altibase server.

⑥ Delete the "aku_start_completed" file in the /tmp directory.

aku -p clean#

This command deletes all Altibase replication objects on the pods and the "aku_start_completed" file in the /tmp directory. Users can use this command when synchronization between pods is no longer needed.

Cautions#

aku.conf#

Every aku property which does not have its default value must be specified. If not, the "Property [property_name] should be specified by configuration." error occurs.
Use # symbol to write a comment in the "aku.conf" file. Note that if there is no letter after # symbol, it causes the syntax error.

Storage corruption in Master Pod#

aku does not recover data corruption due to storage corruption in Master Pod.

In the Event of a master pod failure#

Master pod failure refers to a scenario where the aku -p start command fails on the master pod under the following circumstances:

One or more slave pods are running.
In the master pod, some or all of the replication object information between the running slave pod(s) and the master pod is lost.

When a master pod failure occurs, the following steps should be taken to recover the master pod:

Select the slave pod that will serve as the basis for recovery. Execute the aku -p end command to terminate all slave pods except for the one chosen as the recovery basis.
Synchronize data from the slave pod to the master pod to ensure data consistency.
Start replication on the master pod.
Retry the command aku -p start.

Refer to the example below for detailed steps of master pod failure recovery.

Example of Master Pod Failure Recovery#

Assuming a failure occurs in the master pod named pod_name-0 in the following environment:

Configuration Information#

Configuration of the aku property in the master pod:
- AKU_SERVER_COUNT = 4
- REPLICATION_NAME_PREFIX = AKU_REP
Configuration of Altibase server properties in the master pod:
- ADMIN_MODE = 1

Failure Scenario#

Slave pod pod_name-1 is running.
Replication targets tables are T1, T2, and T3.
Information of replication object AKU_REP_01 (the replication between the master pod and slave pod pod_name-1) in the master pod is lost.

The master pod failure situation can be identified through the following log:

$ aku -p start
AKU started with START option.
[AKU][2024/04/19 19:21:31.012670][140276343642368] [INFO][akuRunStart:828][-][-] Start as MASTER Pod.
[AKU][2024/04/19 19:21:31.012991][140276343642368] [ERROR][akuRunStart:1030][-][-] The MASTER server is detected to have failed.     Check and perform a manual recovery.
AKU failed to run.

At this point, when querying the XSN of the replication is lost in the master pod, the value is output as -1:
```
iSQL> SELECT REPLICATION_NAME, XSN FROM SYSTEM_.SYS_REPLICATIONS_;
REPLICATION_NAME                XSN                  
--------------------------------------------------------
AKU_REP_01                      -1
1 rows selected.
```
XSN is the identification number of XLog, which delivers replication information between remote servers and local servers through sender and receiver threads. When initializing replication objects, this value becomes -1. The following recovery procedures can be sequentially performed to resolve the failure of the master pod:

Recovery Procedure#

Synchronize data from slave pods to the master pod to ensure data consistency.

1-1. Delete records of replication target tables on the master pod.

To prevent conflicts during data synchronization, TRUNCATE the replication target tables managed by the master pod. To execute the TRUNCATE command, the Altibase server property REPLICATION_DDL_ENABLE must be set to 1.

-- Change the REPLICATION_DDL_ENABLE property to 1.
iSQL> ALTER SYSTEM SET REPLICATION_DDL_ENABLE = 1;
Alter success.

-- TRUNCATE records of replication target tables.
iSQL> TRUNCATE TABLE T1;
Truncate success.
iSQL> TRUNCATE TABLE T2;
Truncate success.
iSQL> TRUNCATE TABLE T3;
Truncate success.

-- Restore the REPLICATION_DDL_ENABLE property to 0.
iSQL> ALTER SYSTEM SET REPLICATION_DDL_ENABLE = 0;
Alter success.

1-2. Perform data synchronization on the slave pod.

Before synchronizing data, RESET the XSN value to initialize it. Once synchronization is complete, replication will automatically start.

-- Stop replication on the slave pod 
-- (can be omitted if replication has not started).
iSQL> ALTER REPLICATION AKU_REP_01 STOP;
Alter success.

-- Reset the replication object between the master pod and the slave pod.
iSQL> ALTER REPLICATION AKU_REP_01 RESET;
Alter success.

-- Perform replication SYNC 
-- to synchronize data consistency between the slave pod and the master pod.
iSQL> ALTER REPLICATION AKU_REP_01 SYNC;
Alter success.

Start replication on the master pod.

Start replication on the master pod to complete the recovery of lost replication object information.
```
iSQL> ALTER REPLICATION AKU_REP_01 START;
Alter success.
```

Retry the aku -p start command.

Once recovery is completed, the master pod will have the same data as the slave pod, and the XSN value of the replication object AKU_REP_01 will be updated. Now, when the aku -p start command is retried, it will be processed successfully.

# Read aku.conf and create replication objects. 
$ aku -p start
# START option means that aku -p start is executed.
AKU started with START option.
[AKU][2024/04/19 19:21:01.011887][139922191153408] [INFO][akuRunStart:828][-][-] Start as MASTER Pod.

# Remove replication gap between the master pod and the slave pod.
[AKU][2024/04/19 19:21:09.057372][139922191153408] [INFO][akuRunStart:891][-][-] Flush replications.
[AKU][2024/04/19 19:21:09.086363][139922191153408] [INFO][akuRunStart:896][-][-] Replication flush has ended.

# After all procedures are successfully completed, aku is terminated. 
AKU run successfully.

If the Pod Terminates Before the `aku -p end` Command Completes#

If the situation in which a Pod was force-terminated before the aku -p end command completed or terminated with the property AKU_REPLICATION_RESET_AT_END set to 0 continues for a long time, there is a possibility of uninitialized replication information remaining in the terminated Pod as well as in other Pods. When this happens, the other Pods do not delete the online log files that are required for replication to the terminated Pod. If the online log file accumulates a lot, it can lead to disk space exhaustion and result in Altibase server not being able to operate normally. To prevent this situation, if you notice that there are long periods of time in which a Pod was terminated before the aku -p end command completed, you should stop replication and initialize the replication information. Refer to the commands below.

ALTER REPLICATION replication_name STOP;
ALTER REPLICATION replication_name RESET;

Suppose pod_name-1 failed to complete aku -p end and and terminated abnormally while pod_name-0 and pod_name-1 are both operational. Let's check the XSN of SYSTEM_.SYS_REPLICATIONS_ on pod_name-0. You can see that the XSN value of the replication object AKU_REP_01 is not -1. It means that the replication information is not initialized. AKU_REP_01 is the replication object between pod_name-0 and pod_name-1.

iSQL> SELECT REPLICATION_NAME, XSN FROM SYSTEM_.SYS_REPLICATIONS_;
REPLICATION_NAME                XSN                  
--------------------------------------------------------
AKU_REP_03                      -1
AKU_REP_02                      -1
AKU_REP_01                      859070110
3 rows selected.

Execute the statements for stopping and resetting replication of the object(AKU_REP_01) on pod_name-0.

iSQL> ALTER REPLICATION AKU_REP_01 STOP;
Alter sucess.
iSQL> ALTER REPLICATION AKU_REP_01 RESET;
Alter sucess.

And then, let's check the XSN of SYSTEM_.SYS_REPLICATIONS_ on pod_name-0. The XSN value of the replication object AKU_REP_01 was changed to -1.

iSQL> SELECT REPLICATION_NAME, XSN FROM SYSTEM_.SYS_REPLICATIONS_;
REPLICATION_NAME                XSN                  
--------------------------------------------------------
AKU_REP_03                      -1
AKU_REP_02                      -1
AKU_REP_01                      -1
3 rows selected.

Examples#

This section introduces various examples of aku usage.

The aku log in the example codes includes the following information:

[AKU][Date and time][Thread number] [Message type][Code information][Target pod name][Replication name] Message

aku -i#

This is the result of running aku -i and displays the information set in aku.conf. A server with Server ID 0 is that of the first Pod created by the StatefulSet.

$ aku -i
  #########################
 [ Server ]
  Server ID        : 0
  Host             : AKUHOST-0.altibase-svc
  User             : SYS
  Password         : manager
  Port             : 20300
  Replication Port : 20301
  Max Server Count : 4
 #########################
 [ Replications ]
 #### Serve[ID:0] Replication list ####
  Replication Name : AKU_REP_01
  Replication Name : AKU_REP_02
  Replication Name : AKU_REP_03
 #### Serve[ID:1] Replication list ####
  Replication Name : AKU_REP_01
  Replication Name : AKU_REP_12
  Replication Name : AKU_REP_13
 #### Serve[ID:2] Replication list ####
  Replication Name : AKU_REP_02
  Replication Name : AKU_REP_12
  Replication Name : AKU_REP_23
 #### Serve[ID:3] Replication list ####
  Replication Name : AKU_REP_03
  Replication Name : AKU_REP_13
  Replication Name : AKU_REP_23
 #########################
 [ Replication Items ]
  #### Replication [Prefix:AKU_REP] item list ####
  User Name        : SYS
  Table Name       : T1

  User Name        : SYS
  Table Name       : T2
  Partition Name   : P1

  User Name        : SYS
  Table Name       : T3
 #########################

aku -p start on a Master Pod#

This is an output of running aku -p start on a Master Pod (pod_name-0).

$ aku -p start
AKU started with START option.
[AKU][2024/03/18 12:34:58.136944][140708807235840] [INFO][akuRunStart:828][-][-] Start as MASTER Pod.
AKU run successfully.

The Description of the output is as below.

# Read aku.conf and create replication objects. 
# START option means that aku -p start is executed.
AKU started with START option.

# MASTER Pod indicates the first pod. 
[AKU][2024/03/18 12:34:58.136944][140708807235840] [INFO][akuRunStart:828][-][-] Start as MASTER Pod.

# After all procedures are successfully completed, aku is terminated. 
AKU run successfully.

aku -p start on the Slave Pod#

This is an output of aku -p start command on the 4th Pod (pod_name-3).

AKU started with START option.
[AKU][2024/03/18 14:01:59.604647][140678415444224] [INFO][akuRunStart:903][-][-] Start as SLAVE Pod.
[AKU][2024/03/18 14:02:01.005068][140678415444224] [INFO][akuRunStart:959][-][-] Truncate tables for replications.
[AKU][2024/03/18 14:02:01.025100][140678415444224] [INFO][akuRunStart:964][-][-] Table truncation has ended.
[AKU][2024/03/18 14:02:01.025877][140678415444224] [INFO][akuRunStart:975][-][-] Sync tables from MASTER Server.
[AKU][2024/03/18 14:02:05.045135][140678415444224] [INFO][akuRunStart:980][-][-] Replication sync has ended.
AKU run successfully.

The Description of the output is as below.

# Read aku.conf and create replication objects. 
# START option means that aku -p start is executed.
 AKU started with START option.

# SLAVE Pod indicates other pods but the first pod. 
[AKU][2024/03/18 14:01:59.604647][140678415444224] [INFO][akuRunStart:903][-][-] Start as SLAVE Pod.

# To prvent record conflict during SYNC processing, deletes all records on the target table.
[AKU][2024/03/18 14:02:01.005068][140678415444224] [INFO][akuRunStart:959][-][-] Truncate tables for replications.
[AKU][2024/03/18 14:02:01.025100][140678415444224] [INFO][akuRunStart:964][-][-] Table truncation has ended.

# MASTER Server indicates the Altibase server of the Master Pod. The data has been synchronized from the server to the local pod.
[AKU][2024/03/18 14:02:01.025877][140678415444224] [INFO][akuRunStart:975][-][-] Sync tables from MASTER Server.
[AKU][2024/03/18 14:02:05.045135][140678415444224] [INFO][akuRunStart:980][-][-] Replication sync has ended.

# After all procedures are successfully completed, aku is terminated. 
AKU run successfully.

aku -p end on the Slave Pod#

This is an output of aku -p end command with the property AKU_REPLICATION_RESET_AT_END set to 1 on the 4th Pod (pod_name-3). You can see that replication FLUSH and RESET commands are executed.

$ aku -p end
AKU started with END option.
[AKU][2024/03/18 14:02:49.246961][139626938108160] [INFO][akuRunEnd:1090][-][-] Start as SLAVE Pod.
[AKU][2024/03/18 14:02:49.247094][139626938108160] [INFO][akuRunEnd:1095][-][-] Flush replications.
[AKU][2024/03/18 14:02:49.247731][139626938108160] [INFO][akuRunEnd:1100][-][-] Replication flush has ended.
[AKU][2024/03/18 14:02:52.001848][139626938108160] [INFO][akuRunEnd:1114][-][-] Reset replications.
[AKU][2024/03/18 14:02:52.014300][139626938108160] [INFO][akuRunEnd:1119][-][-] Replication reset has ended.
AKU run successfully.

The Description of the output is as below.

# Read aku.conf, stop the replication and reset the replication. 
# END option means that aku -p end is executed.
AKU started with END option.

# SLAVE Pod indicates other pods but the first pod. 
[AKU][2024/03/18 14:02:49.246961][139626938108160] [INFO][akuRunEnd:1090][-][-] Start as SLAVE Pod.

# FLUSH unsent changes to other pods.
[AKU][2024/03/18 14:02:49.247094][139626938108160] [INFO][akuRunEnd:1095][-][-] Flush replications.
[AKU][2024/03/18 14:02:49.247731][139626938108160] [INFO][akuRunEnd:1100][-][-] Replication flush has ended.

# Reset all replication objects created on the aku of the local pod.
[AKU][2024/03/18 14:02:52.001848][139626938108160] [INFO][akuRunEnd:1114][-][-] Reset replications.
[AKU][2024/03/18 14:02:52.014300][139626938108160] [INFO][akuRunEnd:1119][-][-] Replication reset has ended.

# After all procedures are successfully completed, aku is terminated. 
AKU run successfully.

3. aku#

Introducing aku#

When scaling up#

When scaling down#

Component#

aku#

aku.conf#

Setup to run aku#

Prerequisite#

Altibase Environment Variable and Properties#

aku Properties#

How to configure Altibase Replication Targets#

Usage of aku#

Syntax#

Parameters#

-h, --help#

-v, --version#

-i, --info#

-p, --pod {pod_action}#

aku -p start#

Creation of Master Pod (pod_name-0)#

Scale up#

When creating a Slave Pod for the first time, or restart (Default behavior, AKU_REPLICATION_RESET_AT_END = 1)#

Case of restarting a Slave Pod that has not reset the replication information (Default behavior, AKU_FLUSH_AT_START = 1)#

Case of restarting a Slave Pod that hasn't reset the replication information (AKU_FLUSH_AT_START = 0)#

aku -p end#

aku -p clean#

Cautions#

aku.conf#

Storage corruption in Master Pod#

In the Event of a master pod failure#

Example of Master Pod Failure Recovery#

Configuration Information#

Failure Scenario#

Recovery Procedure#

If the Pod Terminates Before the aku -p end Command Completes#

Examples#

aku -i#

aku -p start on a Master Pod#

aku -p start on the Slave Pod#

aku -p end on the Slave Pod#

If the Pod Terminates Before the `aku -p end` Command Completes#