Connect to HBase running in Docker

I cannot connect to HBase running in Docker on Windows ( banno / hbase-standalone image). However, I can connect to a locally installed HBase.

banno / hbase-standalone image is launched using:

docker run -d -p 2181:2181 -p 60000:60000 -p 60010:60010 -p 60020:60020 -p 60030:60030 banno/hbase-standalone 

I also configured port forwarding on boot2docker-vm (which is required when working on Windows): enter image description here

I can successfully use telnet for all of these ports on my local host.

Next, here is an example of the code we use in our tests:

 Configuration config = HBaseConfiguration.create(); config.clear(); config.setInt("timeout", 12000); config.set("zookeeper.znode.parent", "/hbase"); config.set("hbase.zookeeper.quorum", "127.0.0.1"); config.set("hbase.zookeeper.property.clientPort", "2181"); config.set("hbase.master", "127.0.0.1:60000"); final Configuration configuration = HBaseConfiguration.create(config); JobDefinition.Buildable.dumpProperties(configuration, newArrayList("hbase.*")); HBaseAdmin.checkHBaseAvailable(config); 

What causes the following exception

 Exception in thread "main" org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.net.UnknownHostException: unknown host: a3e6c240af20 at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1651) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(HConnectionManager.java:1677) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1885) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isMasterRunning(HConnectionManager.java:900) at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:2366) at com.xxx.compute.hadoop.jobs.transaction.OurTest.main(OurTest.java:24) Caused by: com.google.protobuf.ServiceException: java.net.UnknownHostException: unknown host: a3e6c240af20 at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1674) at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1715) at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(HConnectionManager.java:1688) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(HConnectionManager.java:1597) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1623) ... 5 more Caused by: java.net.UnknownHostException: unknown host: a3e6c240af20 at org.apache.hadoop.hbase.ipc.RpcClient$Connection.<init>(RpcClient.java:386) at org.apache.hadoop.hbase.ipc.RpcClient.createConnection(RpcClient.java:352) at org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1526) at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1438) at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1657) ... 10 more 

It is explainable. We are launching Windows, which requires a boot2docker-vm virtual machine running using NAT. The Image Docker container works inside boot2docker-vm , also using NAT. However, the ports are "visible" to the host machines that are running the tests, because the Docker container exports the ports, and boot2docker-vm forwards the ports on the host machine. The name a3e6c240af20 actually comes from the Docker container identifier, so a3e6c240af20 is probably the host name for the Docker container:

 CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES a3e6c240af20 banno/hbase-standalone:latest "/bin/sh -c '/opt/hb 24 minutes ago Up 24 minutes 0.0.0.0:2181->2181/tcp, 0.0.0.0:60000->60000/tcp, 0.0.0.0:60010->60010/tcp, 0.0.0.0:60020->60020/tcp, 0.0.0.0:60030->60030/tcp agitated_wozniak 

I'm not sure how HBase communication works, but it seems to cause RPC calls for the instance. HBase Docker returns its hostname, hoping the client will call it there. But since both boot2docker-vm and Docker containers are running using NAT, the host machine does not see the Docker container.

I tried adding a3e6c240af20 to my hosts file:

 127.0.0.1 a3e6c240af20 

Then I get another error, also during an RPC call, which actually doesn't help me much:

 Exception in thread "main" org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.lang.NullPointerException at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1651) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(HConnectionManager.java:1677) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1885) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isMasterRunning(HConnectionManager.java:900) at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:2366) at com.xxx.compute.hadoop.jobs.transaction.OurTest.main(OurTest.java:24) Caused by: com.google.protobuf.ServiceException: java.lang.NullPointerException at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1674) at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1715) at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(HConnectionManager.java:1688) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(HConnectionManager.java:1597) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$StubMaker.makeStub(HConnectionManager.java:1623) ... 5 more Caused by: java.lang.NullPointerException at org.apache.hadoop.hbase.ipc.RpcClient$Connection.writeRequest(RpcClient.java:1051) at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1440) at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1657) ... 10 more 

Does anyone have a suggestion on how this can be solved?

+7
java windows docker hbase nat
source share
2 answers

Try adding [boot2docker IP] a3e6c240af20 instead of 127.0.0.1 because the HBase Java client must reach your docker host not exactly localhost to access zookeeper (CMIIW). Not very sure that it will work, but it works on my Windows.

+1
source share

I used the docker image oddpoet / hbase-cdh5 to avoid this problem.

 docker run -d -p 2181:2181 -p 60000:60000 -p 60010:60010 -p 60020:60020 -p 60030:60030 -h hbase oddpoet/hbase-cdh5 

fig.yml

 hbase: image: oddpoet/hbase-cdh5 hostname: hbase ports: - "3181:2181" - "60000:60000" - "60010:60010" - "60020:60020" - "60030:60030" 

my configuration file

  conf.set("hbase.zookeeper.quorum", zkPath); conf.set("hbase.zookeeper.property.clientPort","2181"); conf.set("zookeeper.znode.parent", "/hbase"); conf.set("hbase.client.retries.number", "3"); // default 35 conf.set("hbase.rpc.timeout", "10000"); // default 60 secs conf.set("hbase.rpc.shortoperation.timeout", "5000"); // default 10 secs 
0
source share

All Articles