Geek Answers Handbook

Retrieving files in Hadoop in a web application

I am new to Hadoop. Now I am trying to make an application in eclipse in which I want to use the data available in HDFS. If we want to connect to the database with Java, we have a JDBC connection. How to do this, what do I need to do to connect directly to HDFS?

+4
source share
2 answers

In Hadoop, firstly, you will need to make sure that Hadoop is up and running. Apache Hadoop provides Java classes - FileSystem for accessing files in HDFS from a Java application. Here is one example: I am accessing / books / pg 5000.txt using FileSystem and IOUtils.

import java.io.InputStream;
import java.net.URI;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;


public class FileSystemCat {

        public static void main(String[] args) throws Exception {
            Configuration conf = new Configuration();
            conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/core-site.xml"));
            conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/hdfs-site.xml"));
          String uri = "/books/pg5000.txt";
        FileSystem fs = FileSystem.get(URI.create(uri), conf);
        InputStream in = null;
        try {
            in = fs.open(new Path(uri));
            IOUtils.copyBytes(in, System.out, 4096, false);
            } finally {
            IOUtils.closeStream(in);
            }
        }
}
+3
source

HDFS (), . Hive Hadoop HiveServer2, Thrift API , HDFS .

: https://cwiki.apache.org/confluence/display/Hive/HiveClient

, HIVE ODBC Hadoop (Cloudera, Microsoft HDInsight, Hortonworks).

+1

More articles:


All Articles