python - How to compare hdfs file with unix file? -
i want compare hdfs file unix file.i know 1 way using cat command can compare files because of cat command provided hadoop distributed file scheme (hdfs).
let's take 1 illustration :
# hdfs dfs -ls -r /demo -rw-r--r-- 3 root hdfs 129617 2014-10-17 12:22 /demo/abc.log #ls /tmp xyz.log
here want compare abc.log
xyz.log
. improve approach??
you utilize shell redirection + diff.
diff <(hdfs dfs -cat /demo/abc.log) <(cat /tmp/xyz.log)
python linux shell hadoop hdfs
No comments:
Post a Comment