Hbase: how to specify a host name for an hbase master

Question

I'm struggling to set up a distributed Hbase cluster with two nodes, one of which is my machine, and the other a virtual machine, using the host-only adapter in VirtualBox.

My problem is that the area server (from the VM machine) cannot connect to the Hbase host running on the host machine. Although in the Hbase shell I can list, create a table ... in the register server on the VM machine ("slave"), the log always shows

org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to connect to master. Retrying. Error was: java.net.ConnectException: Connection refused 

Previously, I successfully configured Hadoop, HDFS, and MapReduce in this cluster with two nodes named "master" and "slave", "master" as the master node, and "master" and "slave" work as slave nodes, these names are bound to the interface vboxnet0 VirtualBox (host names in / etc / hostname are different). I also specify the "slave.host.name" property for each node as "master" and "slave".

It seems that the Hbase master on "master" always starts with the name "localhost" host, from the slave machine, I cannot telnet for the hbase master named "master". So, is there a way to specify the use of the hostname for the Hbase host as "master", I tried to specify some properties of the DNS interface for ZooKeeper, Master, RegionServer to use the internal interface between master and slave, but it still does not work at all.

/ etc / hosts for both, like something like

 127.0.0.1 localhost 127.0.0.1 ubuntu.mymachine # For Hadoop 192.168.56.1 master 192.168.56.101 slave # The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters 

Answer 1

My host file is similar to

127.0.0.1 localhost

192.168.2.118 shashwat.machine.com shashwat

make your host files as follows:

127.0.0.1 localhost

For hadoop

192.168.56.1 master

192.168.56.101 slave

and in hbase conf enter the following entries:

 <property> <name>hbase.rootdir</name> <value>hdfs://master:9000/hbase</value> </property> <property> <name>hbase.master</name> <value>master:60000</value> <description>The host and port that the HBase master runs at.</description> </property> <property> <name>hbase.regionserver.port</name> <value>60020</value> <description>The host and port that the HBase master runs at.</description> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> <property> <name>hbase.tmp.dir</name> <value>/home/cluster/Hadoop/hbase-0.90.4/temp</value> </property> <property> <name>hbase.zookeeper.quorum</name> <value>master</value> </property> <property> <name>dfs.replication</name> <value>2</value> </property> <property> <name>hbase.zookeeper.property.clientPort</name> <value>2181</value> <description>Property from ZooKeeper config zoo.cfg. The port at which the clients will connect. </description> </property> 

If you use localhost anywhere, delete this and replace it with "master", which is the namenode name in your host file ....

one thing you can do

sudo gedit / etc / hostname

this will open the bydefault hostname file. ubuntu will be there, so make it a master. and reboot the system.

To define hbase in the "regionserver" file inside conf dir, enter these entries

master slave

and reboot everything.

Answer 2

The answer provided by @Infinity seems to be about ~ 0.9.4 version.

For version 1.1.4.

according to the source code from

 org.apache.hadoop.hbase.master.HMaster 

configuration should be:

  <property> <name>hbase.master.hostname</name> <value>master.local</value> <!-- master.local is the DNS name in my network pointing to hbase master --> </property> 

After setting this value, region servers can connect to the hbase master; however, in my environment, a region server complained:

 com.google.protobuf.ServiceException: java.net.SocketException: Invalid argument 

The problem disappeared after I installed oracle JDK 8 instead of open-jdk-7 in all my nodes.

So in conclusion, here is my solution:

• use a DNS server instead of setting / etc / hosts, since hbase is very choosing a host name and seems to require a DNS lookup as well as a reverse DNS lookup.

• upgrade jdk to oracle 8

• use the specified setting item above.

Answer 3

There are two things that fix this class of problem for me:

1) Remove all the "localhost" names, only 127.0.0.1 pointing to the name of the hmaster node.

2) run "hostname X" on your hbase node host to make sure the hostname matches what is in / etc / hosts.

Not being a network expert, I can’t say why this is important, but it :)

Answer 4

In most cases, the error comes from Zookeeper, which sends the wrong hostname.

You can check what Zookeeper sends as the main HBase host:

Find the bin Zookeeper folder:

 bin/zkCli.sh -server 127.0.0.1:2181 get /hbase/master 

This should give you the main HBase IP address that Zookeeper answers, so this IP address should be accessible.