Writing a file to HDFS using Java

Question

I am trying to write a file in HDFS, the file is created, but it is empty in the cluster, however, when I run the code locally, it works like a charm.

here is my code:

FSDataOutputStream recOutputWriter = null; FileSystem fs = null; try { //OutputWriter = new FileWriter(outputFileName,true); Configuration configuration = new Configuration(); fs = FileSystem.get(configuration); Path testOutFile = new Path(outputFileName); recOutputWriter = fs.create(testOutFile); //outputWriter = new FileWriter(outputFileName,true); } catch (IOException e) { e.printStackTrace(); } recOutputWriter.writeBytes("======================================\n"); recOutputWriter.writeBytes("OK\n"); recOutputWriter.writeBytes("======================================\n"); if (recOutputWriter != null) { recOutputWriter.close(); } fs.close(); 

Did I miss something?

Answer 1

To write data to a file after it was created in the cluster, I had to add:

 System.setProperty("HADOOP_USER_NAME", "vagrant"); 

Refrence - Burn files to Hadoop HDFS using Scala