hadoop - How to delete intermediate output file from Hdfs -
i trying delete intermediate output directory of mapreduce programme using
fileutils.deletedirectory(new file(tempfiles));
but command doesn't delete directories hdfs.
map cut down not write intermediate results on hdfs ,it writes on local disk.
whenever mapper produce output first goes on memory buffer partitioning , sorting takes place when buffer exceeds default capacity spill results local disk .
summary output produced mapper goes local file scheme .
only in 1 status mapper write output hdfs if has been set in driver class not utilize reducer.
in above case there final output won't intermediate.
hadoop mapreduce hdfs delete-file
No comments:
Post a Comment