Geek Answers Handbook

How to import / export hbase data via hdfs (hadoop commands)

I saved my workarounds using nutch in Hbase, whose file system is hdfs. Then I copied my data (one hbase table) from hdfs directly to some local directory with the command

hadoop fs -CopyToLocal /hbase/input ~/Documents/output

After that, I copied this data back to another hbase (another system) by running the command

 hadoop fs -CopyFromLocal ~/Documents/input /hbase/mydata

It is saved in hdfs and when I use the list command in the hbase shell, it shows it as another table, that is, "mydata", but when I run the scan command, it says that there is no table named "mydata".

What is the problem with the above procedure? In simple words:

  • I want to copy the hbase table to the local file system using the hadoop command
  • Then I want to save it directly to hdfs on another system with the hadoop command
  • Finally, I want the table to display in hbase and display its data as the original table.
+8
hbase hadoop
source share
2 answers

If you want to export a table from one hbase cluster and import it into another, use any of the following methods:

Using Hadoop

  • Export

     $ bin/hadoop jar <path/to/hbase-{version}.jar> export \ <tablename> <outputdir> [<versions> [<starttime> [<endtime>]]

    NOTE. Copy the output directory in hdfs from source to target cluster

  • Import

     $ bin/hadoop jar <path/to/hbase-{version}.jar> import <tablename> <inputdir>

Note. Both outputs output and birdir are in hdfs.

Using hbase

  • Export

     $ bin/hbase org.apache.hadoop.hbase.mapreduce.Export \ <tablename> <outputdir> [<versions> [<starttime> [<endtime>]]]

  • Copy the output directory in hdfs from the source to the target cluster

  • Import

     $ bin/hbase org.apache.hadoop.hbase.mapreduce.Import <tablename> <inputdir>

    Link: Hbase tool for export and import

+20
source share

If you can instead use the Hbase command to back up hbase tables, you can use the Hbase ExportSnapshot tool, which copies the hfiles, logs and snapshot metadata to another file system (local / hdfs / s3) using map reduction work.

  • Take a picture of the table

    $ ./bin/hbase shell hbase> snapshot 'myTable', 'myTableSnapshot-122112'

  • Export to the desired file system

    $ ./bin/hbase class org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot MySnapshot -copy-to fs://path_to_your_directory

You can export it from the local file system to hdfs: /// srv2: 8082 / hbase and run the restore command from the hbase shell to restore the table from the snapshot.

  $ ./bin/hbase shell hbase> disable 'myTable' hbase> restore_snapshot 'myTableSnapshot-122112'

Link: Hbase Pictures

+4
source share

More articles:


All Articles