WebHDFS redirect URL for datanode has an unresolvable local hostname when on AWS

I have HDFS running on an EC2 node (pseudo multi-node setup) and I use it to access files through the REST WebHDFS API, doing a GET, for example. this:

http: slash slash ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com....0070/webhdfs/v1/foo/bar.txt?op=OPEN

This gives me a temporary redirect to

http: slash slash ip-yy-yy-yy-yy.us-west-2.compute.internal: 50075 / webhdfs / v1 / foo / bar.txt? op = OPEN & namenoderpcaddress = localhost: 9000 & offset = 0

Here xx-xx-xx-xx is the public, static IP assigned to my instance, and yy-yy-yy-yy is the local ip for the instance.

This makes the redirect unsuccessful because ip-yy-yy-yy-yy.us-west-2.compute.internal cannot be opened from the browser, obviously! I would like the redirect URL to generate a static public IP address assigned to my instance, resolving the default public DNS.

Here is a list of HDFS configuration settings, but I can not understand what causes this situation.

My hdfs-site.xml settings:

dfs.replication: 1
dfs.webhdfs.enabled:true

My core-site.xml configurations:

fs.defaultFS: hdfs://localhost:9000

Any help is appreciated, thanks!

+4
source share

All Articles