1. Introduction to Altibase Hadoop Connector

This chapter introduces the Altibase Hadoop Connector and provides background knowledge.

Background Knowledge

This chapter introduces the basic concept of transferring data between Hadoop and Altibase with Sqoop.

Hadoop

Hadoop is a system that is suitable for the analysis and management of large amounts of data. Hadoop is one of the solutions seeing increased interest as demands for parallel processing of data warehouses and the management of 'big data' increase and the popularity of cloud and distributed computing grows.

Hadoop is an open source software based on Java, and is a framework which distributively processes large amounts of data spread across clusters composed of multiple computers. Hadoop is composed of HDFS(Hadoop Distributed File System) and MapReduce.

Sqoop

Sqoop is a tool for data transfer between Hadoop and relational databases, and is open source software. Using Sqoop, the user can import data from RDBMS to HDFS and export it back to RDBMS. For the import and export of data, Sqoop uses MapReduce. MapReduce offers Fault Tolerance and Parallel Operation.

Sqoop automates most of this process by relying on a database to represent the data to be imported as a schema. Sqoop uses MapReduce to import and export data. MapReduce provides fault tolerance and parallel operation.

What is the Altibase Hadoop Connector?

The Altibase Hadoop Connector enables the efficient transfer of data between Hadoop and Altibase servers. It also allows Altibase to handle data management and Hadoop to handle data analysis. In other words, the Altibase Hadoop Connector lets the user connect to theAltibase server and export data into HDFS or HIVE for the purpose of processing data from Hadoop.

The Altibase Hadoop Connector operates on Sqoop and supports nearly all functions provided by it. Also, users that have experience with Sqoop will easily be able to use the Altibase Hadoop Connector, as it uses a command line argument structure similar to Sqoop