filmov
tv
How to import data from MySQL to Hive using Sqoop

Показать описание
Import data from RDBMS to HDFS:
Example:
$Sqoop import
--connect jdbc:mysql://localhost/college
--username root
--password mypassword
--tables students
-m 2
--target-dir /user/YT/college/student
This import statement stored all those students rows in target directory location.
Import data from MySQL to Hive:
When Sqoop import table schema to the hive table. It write as –
MySQL --to-- temporary HDFS location --to-- Hive table
How MySQL table schema convert to Hive:
MySQL Datatype --to--Java Data type --to-- Hive Datatype
Sqoop code is written in java. Sqoop internally convert MySQL database to java datatype and java datatype to hive datatype.
What Required for import data into Hive:
For import records into Hive following options are required in Sqoop statement:
1. --hive-import
2. --hive-table table-name
--hive-import keyword tell sqoop to import into Hive.
--hive-table points to the table where data is to be put into Hive.
Import data from MySQL to Hive:
Example:
$Sqoop --import
--connect jdbc:mysql://localhost/college
--username root
--password mypassword
-- table students
--hive-import
--hive-table students
-m 1
So this way how we have data in MySQL, the same type and data stored in Hive by Sqoop.
As we write SQL query in MySQL, the same kinds of query we have in Hive.
Example:
$Sqoop import
--connect jdbc:mysql://localhost/college
--username root
--password mypassword
--tables students
-m 2
--target-dir /user/YT/college/student
This import statement stored all those students rows in target directory location.
Import data from MySQL to Hive:
When Sqoop import table schema to the hive table. It write as –
MySQL --to-- temporary HDFS location --to-- Hive table
How MySQL table schema convert to Hive:
MySQL Datatype --to--Java Data type --to-- Hive Datatype
Sqoop code is written in java. Sqoop internally convert MySQL database to java datatype and java datatype to hive datatype.
What Required for import data into Hive:
For import records into Hive following options are required in Sqoop statement:
1. --hive-import
2. --hive-table table-name
--hive-import keyword tell sqoop to import into Hive.
--hive-table points to the table where data is to be put into Hive.
Import data from MySQL to Hive:
Example:
$Sqoop --import
--connect jdbc:mysql://localhost/college
--username root
--password mypassword
-- table students
--hive-import
--hive-table students
-m 1
So this way how we have data in MySQL, the same type and data stored in Hive by Sqoop.
As we write SQL query in MySQL, the same kinds of query we have in Hive.