Monday 15 April 2013

hadoop - How to delete intermediate output file from Hdfs -



hadoop - How to delete intermediate output file from Hdfs -

i trying delete intermediate output directory of mapreduce programme using

fileutils.deletedirectory(new file(tempfiles));

but command doesn't delete directories hdfs.

map cut down not write intermediate results on hdfs ,it writes on local disk.

whenever mapper produce output first goes on memory buffer partitioning , sorting takes place when buffer exceeds default capacity spill results local disk .

summary output produced mapper goes local file scheme .

only in 1 status mapper write output hdfs if has been set in driver class not utilize reducer.

in above case there final output won't intermediate.

hadoop mapreduce hdfs delete-file

No comments:

Post a Comment