Thursday, December 18, 2014

How can we import MySQL data into HDFS | Working with Sqoop

Sqoop:


  • Sqoop is a tool that used to import SQL data from SQL databases to HDFS. It was devloped by cloudera.
  • It uses JDBC to talk to database.
  • Sqoop provides the java code that can import data to HDFS
  • After the java code generation,a map only mapreduce job is run to import the data.
  • By default 4mappers are run with 25% each.
Here i am providing few commands that can list the databases,tables and import the tables.

1: sqoop list-databases --connect jdbc:mysql://localhost/training_db --username root --password root
(This will list the available databases in your connection)

2:sqoop list-tables --connect jdbc:mysql://localhost/training_db --username root --password root

3:sqoop list-tables --connect jdbc:mysql://localhost/training_db --table user_log --fields-terminated-by '\t'  -m 1 --username root --password root


1 comment: