Monday 15 September 2014

high availability - Any command to get active namenode for nameservice in hadoop? -



high availability - Any command to get active namenode for nameservice in hadoop? -

the command:

hdfs haadmin -getservicestate machine-98

works if know machine name. there command like:

hdfs haadmin -getservicestate <nameservice>

which can tell ip/hostname of active namenode?

to print out namenodes utilize command: hdfs getconf -namenodes

to print out secondary namenodes: hdfs getconf -secondarynamenodes

to print out backup namenodes: hdfs getconf -backupnodes

note: these commands tested using hadoop 2.4.0.

update 10-31-2014:

here python script read namenodes involved in hadoop ha config file , determine of them active using hdfs haadmin command. script not tested not have ha configured. tested parsing using sample file based on hadoop ha documentation. sense free utilize , modify needed.

class="lang-py prettyprint-override">#!/usr/bin/env python # coding: utf-8 import xml.etree.elementtree et import subprocess sp if __name__ == "__main__": hdfssiteconfigfile = "/etc/hadoop/conf/hdfs-site.xml" tree = et.parse(hdfssiteconfigfile) root = tree.getroot() hashadoophaelement = false activenamenode = none property in root: if "dfs.ha.namenodes" in property.find("name").text: hashadoophaelement = true nameserviceid = property.find("name").text[len("dfs.ha.namenodes")+1:] namenodes = property.find("value").text.split(",") node in namenodes: #get namenode machine address check if active node n in root: prefix = "dfs.namenode.rpc-address." + nameserviceid + "." elementtext = n.find("name").text if prefix in elementtext: nodeaddress = n.find("value").text.split(":")[0] args = ["hdfs haadmin -getservicestate " + node] p = sp.popen(args, shell=true, stdout=sp.pipe, stderr=sp.pipe) line in p.stdout.readlines(): if "active" in line.lower(): print "active namenode: " + node break; err in p.stderr.readlines(): print "error executing hadoop ha command: ",err break if not hashadoophaelement: print "hadoop high-availability configuration not found!"

hadoop high-availability yarn

No comments:

Post a Comment